Data sources
This page lists what a data source is and how a market uses one: what it is asked for, what it answers, how a fetch fails, and the calls involved. Use a source when your quotes and prints live somewhere else, such as a vendor feed or a database of closes.
Quick example
The example reads its quotes from the desk sheets; load them once and it runs as shown. Its source answers from a sheet held in memory. It shows what a seasoned book demands, fetches and values it, and then shows a fetch refused for a date the source can't answer. An incomplete answer, refused with every gap named, is in the how-to Connect a data source.
use std::sync::Arc;
use chrono::NaiveDate;
use helixio_docs_sheets::{SOFR_CLOSE_DATE, Sheet, USD_SOFR_LADDER, USD_SOFR_PRINTS};
use helixrs::{
Book, DataSource, Demand, Locator, MarketContext, MarketData, MarketDataSource, SourceError,
StripQuotes, Termination, template,
};
type Outcome<T> = Result<T, Box<dyn std::error::Error>>;
fn date((year, month, day): (i32, u32, u32)) -> NaiveDate {
NaiveDate::from_ymd_opt(year, month, day).expect("a real calendar date")
}
// The source answers from the desk's sheet: every pillar of every strip it is
// asked for, observed on the day, and the prints it holds.
struct DeskSheet {
sheet: Vec<(&'static str, &'static str, f64)>,
}
impl MarketDataSource for DeskSheet {
fn fetch(&self, demand: &Demand, as_of: NaiveDate) -> Result<MarketData, SourceError> {
let unreadable = |message: String| SourceError::Source {
source: "desk sheet".to_owned(),
message,
};
let mut strips = Vec::new();
for asked in &demand.strips {
let mut entries = Vec::new();
for (_, locator, quote) in self.sheet.iter().filter(|(t, _, _)| *t == asked.template) {
let at = Locator::parse(locator).map_err(|e| unreadable(e.to_string()))?;
entries.push((at, *quote));
}
if !entries.is_empty() {
strips.push(StripQuotes {
template: asked.template.clone(),
observed: as_of,
entries,
});
}
}
let fixings = USD_SOFR_PRINTS
.iter()
.map(|(series, day, value)| ((*series).to_owned(), date(*day), *value))
.collect();
Ok(MarketData {
strips,
fixings,
fx: Vec::new(),
})
}
}
fn sourced(close: NaiveDate, sheet: Sheet, only: Option<&str>) -> MarketContext {
let sheet = sheet
.iter()
.copied()
.filter(|(template, _, _)| only.is_none_or(|kept| kept == *template))
.collect();
MarketContext::as_of(close).with_data_source(Some(DataSource::new(Arc::new(DeskSheet { sheet }))))
}
fn main() -> Outcome<()> {
let close = date(SOFR_CLOSE_DATE);
// A swap traded on 24 September has accrued three days by the close, so the
// book reads three prints, and the USD recipe reads four strips.
let seasoned = template("USD-SOFR-OIS")?
.swap(date((2026, 9, 24)))?
.with_maturity("5Y".parse::<Termination>()?)?
.with_notional(50_000_000.0)
.with_fixed_rate(0.0466)
.build()?;
let book = Book::new().add("SOFR-5Y", seasoned.clone(), 1.0);
let market = sourced(close, USD_SOFR_LADDER, None);
let demand = market.demand(&book)?;
for strip in &demand.strips {
println!("strip {}", strip.template);
}
for (series, day) in &demand.fixings {
println!("fixing {series} {day}");
}
// Attaching the market, then fetching its date: Rust fetches nothing until
// asked. The report says what came back.
let mut book = book.attach_market(market);
book.fetch_market_data(close)?;
let npv = book.npv(close)?;
let rows = &book.fetch_report().rows;
let (answered, fetched) =
(rows.iter().filter(|(_, _, status)| *status == "answered").count(), rows.len());
println!("answered {answered} of {fetched}");
println!("npv {npv:.2}");
// A valuation on 2 October reads the 1 October print, which the source does
// not hold, so the fetch for that date is refused. The book is left where it
// was: on the close, holding the close's snapshot alone, and still pricing.
let next_day = date((2026, 10, 2));
let fetch_refusal = book
.fetch_market_data(next_day)
.expect_err("the 1 October print is not on the source")
.to_string();
let held = book.market_context().expect("attached");
let (held_as_of, held_dates) = (held.reference_date(), held.dates());
let yes = |found: bool| if found { "yes" } else { "no" };
println!(
"refused, naming the 2026-10-01 print: {}",
yes(fetch_refusal.contains("2026-10-01"))
);
println!("as of {held_as_of}");
println!("dates {}", held_dates.len());
let again = book.npv(close)?;
println!("npv {again:.2}");
Ok(())
}import datetime as dt
import helixio as hlx
from desk_sheets import SOFR_CLOSE_DATE, USD_SOFR_LADDER, USD_SOFR_PRINTS
# The source answers from the desk's sheet: every pillar of every strip it is
# asked for, observed on the day, and the prints it holds.
class DeskSheet:
def __init__(self, sheet):
self.sheet = sheet
def fetch(self, demand, as_of):
asked = demand.strips.to_dict()["template"]
strips = [
(template, as_of, list(self.sheet[template].items()))
for template in asked
if template in self.sheet
]
return hlx.MarketData(strips=strips, fixings=list(USD_SOFR_PRINTS))
# A swap traded on 24 September has accrued three days by the close, so the
# book reads three prints, and the USD recipe reads four strips.
seasoned = (
hlx.templates.get("USD-SOFR-OIS")
.swap(dt.date(2026, 9, 24))
.with_maturity("5Y")
.with_notional(50_000_000.0)
.with_fixed_rate(0.0466)
.build()
)
book = hlx.Book().add("SOFR-5Y", seasoned, 1.0)
market = hlx.MarketContext(SOFR_CLOSE_DATE).with_data_source(DeskSheet(USD_SOFR_LADDER))
demand = market.demand(book)
for template in demand.strips.to_dict()["template"]:
print(f"strip {template}")
for series, day in zip(demand.fixings.to_dict()["series"], demand.fixings.to_dict()["date"]):
print(f"fixing {series} {day}")
# Attaching the market fetches the market's own date; the report says what
# came back.
book.attach_market(market)
npv = book.npv(as_of=SOFR_CLOSE_DATE)
report = book.fetch_report().to_dict()
print(f"answered {report['status'].count('answered')} of {len(report['status'])}")
print(f"npv {npv:.2f}")
# A valuation on 2 October reads the 1 October print, which the source does not
# hold, so the fetch for that date is refused. The book is left where it was:
# on the close, holding the close's snapshot alone, and still pricing — a verb
# with no date values on the close.
NEXT_DAY = dt.date(2026, 10, 2)
try:
book.fetch_market_data(NEXT_DAY)
except hlx.SnapshotError as error:
fetch_refusal = str(error)
else:
raise AssertionError("the 1 October print is not on the source")
held = book.market_context
print(f"refused, naming the 2026-10-01 print: {'yes' if '2026-10-01' in fetch_refusal else 'no'}")
print(f"as of {held.as_of()}")
print(f"dates {len(held.dates())}")
again = book.npv()
print(f"npv {again:.2f}")strip USD-SOFR-3M-FUT
strip USD-SOFR-DEPO
strip USD-SOFR-MeetingOIS
strip USD-SOFR-OIS
fixing USD-SOFR 2026-09-28
fixing USD-SOFR 2026-09-29
fixing USD-SOFR 2026-09-30
answered 7 of 7
npv -102966.25
refused, naming the 2026-10-01 print: yes
as of 2026-10-01
dates 1
npv -102966.25
A refused fetch leaves the book where it was: on the close, holding the close's snapshot, still pricing.
API
| Symbol | Meaning | Rust | Python |
|---|---|---|---|
| the demand | MarketContext::demand → Demand |
MarketContext.demand → Demand |
|
| the source | a type implementing MarketDataSource, wrapped in DataSource::new |
any object with fetch(demand, as_of) |
|
| the answer | MarketData of StripQuotes, prints and FxSpots |
MarketData(strips=…, fixings=…, fx=…) |
|
| build the snapshot | MarketContext::with_market_data |
MarketContext.with_market_data |
|
| the whole cycle | fetch for a book's date | MarketContext::with_data_source, then Book::fetch_market_data |
MarketContext.with_data_source, then Book.fetch_market_data |
| gaps | an incomplete answer, refused | SnapshotError |
SnapshotError |
| Argument | Type | Default | Meaning |
|---|---|---|---|
MarketContext.with_data_source(source) |
object | None |
required | the source this market fetches from; None makes it offline, so it never fetches |
source.fetch(demand, as_of) |
your method | required | answers one Demand for one date with a MarketData, leaving out what it doesn't hold |
MarketData(strips=, fixings=, fx=) |
list[(template, date, list[(locator, quote)])], list[(series, date, value)], list[(pair, rate, date)] |
empty | the answer: whole strips by template, prints by series and date, and FX spots by pair |
MarketContext.demand(book) |
Book |
required | what the book needs that the market doesn't hold |
Book.fetch_market_data(as_of) |
date |
required | fetches the demand for as_of and installs the answer |
Book.fetch_report() |
— | — | each demanded key and whether it was answered |
set_data_source(source) |
object | None |
required | Python only: a process-wide default source every later market copies; None clears it |
Defaults and limits
- What a fetch does: first the market works out what the book needs at a date that it doesn't already hold (the demand). Then the source answers what it holds of that, observed on or before the date. Finally the market installs the answer: it solves the recipes' curves from the answered strips and adds the answered prints. So a fetched market and one marked from a sheet with the same numbers are the same market. Why sources work in whole strips rather than points is covered in how a market is built.
- What a demand names: for one date, strips (each template a recipe needs, as a whole template), fixings (each print a valuation reads), FX spots (each spot the book converts through), and unresolved curves (curves no recipe builds, which a source can't be asked for).
- Gaps are refused together: every key the demand named and the answer didn't carry is a gap.
A snapshot with gaps raises
SnapshotError, naming every gap at once. - A refused fetch changes nothing: the book keeps the date and the snapshots it had.
- Per-market sources: the examples set the source on each market, so two markets in one
process can't share a source by accident.
set_data_sourceexists for the process-wide case.
!!! warning "Python fetches for you; Rust doesn't"
In Python, attaching a sourced market to a book fetches the market's date, and a verb asking
for a date the market doesn't hold fetches it then. In Rust, a verb on a date the market
doesn't hold is refused, naming the date, and the book fetches only when you call
fetch_market_data. Both languages fetch explicitly with fetch_market_data(as_of).
See also
- Source:
crates/helixrs/src/market/source.rs,crates/helixrs/src/trading.rs,crates/helixio/src/sources.rs - Related pages:
- Connect a data source, the task step by step.
- How a market is built, for demand and install taken apart by hand.
- How market data reaches a market, for the provider design.
- Books carry their market, for verbs on a fetched market.
From the 0.0 line at revision bf7f2e2. Every printed number is that revision's own output.