Esc

    The API is under testing and may change before release.

    Data sources

    This page lists what a data source is and how a market uses one: what it is asked for, what it answers, how a fetch fails, and the calls involved. Use a source when your quotes and prints live somewhere else, such as a vendor feed or a database of closes.

    Quick example

    The example reads its quotes from the desk sheets; load them once and it runs as shown. Its source answers from a sheet held in memory. It shows what a seasoned book demands, fetches and values it, and then shows a fetch refused for a date the source can't answer. An incomplete answer, refused with every gap named, is in the how-to Connect a data source.

    Rust
    use std::sync::Arc;
    
    use chrono::NaiveDate;
    use helixio_docs_sheets::{SOFR_CLOSE_DATE, Sheet, USD_SOFR_LADDER, USD_SOFR_PRINTS};
    use helixrs::{
        Book, DataSource, Demand, Locator, MarketContext, MarketData, MarketDataSource, SourceError,
        StripQuotes, Termination, template,
    };
    
    type Outcome<T> = Result<T, Box<dyn std::error::Error>>;
    
    fn date((year, month, day): (i32, u32, u32)) -> NaiveDate {
        NaiveDate::from_ymd_opt(year, month, day).expect("a real calendar date")
    }
    
    // The source answers from the desk's sheet: every pillar of every strip it is
    // asked for, observed on the day, and the prints it holds.
    struct DeskSheet {
        sheet: Vec<(&'static str, &'static str, f64)>,
    }
    
    impl MarketDataSource for DeskSheet {
        fn fetch(&self, demand: &Demand, as_of: NaiveDate) -> Result<MarketData, SourceError> {
            let unreadable = |message: String| SourceError::Source {
                source: "desk sheet".to_owned(),
                message,
            };
            let mut strips = Vec::new();
            for asked in &demand.strips {
                let mut entries = Vec::new();
                for (_, locator, quote) in self.sheet.iter().filter(|(t, _, _)| *t == asked.template) {
                    let at = Locator::parse(locator).map_err(|e| unreadable(e.to_string()))?;
                    entries.push((at, *quote));
                }
                if !entries.is_empty() {
                    strips.push(StripQuotes {
                        template: asked.template.clone(),
                        observed: as_of,
                        entries,
                    });
                }
            }
            let fixings = USD_SOFR_PRINTS
                .iter()
                .map(|(series, day, value)| ((*series).to_owned(), date(*day), *value))
                .collect();
            Ok(MarketData {
                strips,
                fixings,
                fx: Vec::new(),
            })
        }
    }
    
    fn sourced(close: NaiveDate, sheet: Sheet, only: Option<&str>) -> MarketContext {
        let sheet = sheet
            .iter()
            .copied()
            .filter(|(template, _, _)| only.is_none_or(|kept| kept == *template))
            .collect();
        MarketContext::as_of(close).with_data_source(Some(DataSource::new(Arc::new(DeskSheet { sheet }))))
    }
    
    fn main() -> Outcome<()> {
        let close = date(SOFR_CLOSE_DATE);
    
        // A swap traded on 24 September has accrued three days by the close, so the
        // book reads three prints, and the USD recipe reads four strips.
        let seasoned = template("USD-SOFR-OIS")?
            .swap(date((2026, 9, 24)))?
            .with_maturity("5Y".parse::<Termination>()?)?
            .with_notional(50_000_000.0)
            .with_fixed_rate(0.0466)
            .build()?;
        let book = Book::new().add("SOFR-5Y", seasoned.clone(), 1.0);
        let market = sourced(close, USD_SOFR_LADDER, None);
        let demand = market.demand(&book)?;
        for strip in &demand.strips {
            println!("strip   {}", strip.template);
        }
        for (series, day) in &demand.fixings {
            println!("fixing  {series} {day}");
        }
    
        // Attaching the market, then fetching its date: Rust fetches nothing until
        // asked. The report says what came back.
        let mut book = book.attach_market(market);
        book.fetch_market_data(close)?;
        let npv = book.npv(close)?;
        let rows = &book.fetch_report().rows;
        let (answered, fetched) =
            (rows.iter().filter(|(_, _, status)| *status == "answered").count(), rows.len());
        println!("answered {answered} of {fetched}");
        println!("npv      {npv:.2}");
    
        // A valuation on 2 October reads the 1 October print, which the source does
        // not hold, so the fetch for that date is refused. The book is left where it
        // was: on the close, holding the close's snapshot alone, and still pricing.
        let next_day = date((2026, 10, 2));
        let fetch_refusal = book
            .fetch_market_data(next_day)
            .expect_err("the 1 October print is not on the source")
            .to_string();
        let held = book.market_context().expect("attached");
        let (held_as_of, held_dates) = (held.reference_date(), held.dates());
        let yes = |found: bool| if found { "yes" } else { "no" };
        println!(
            "refused, naming the 2026-10-01 print: {}",
            yes(fetch_refusal.contains("2026-10-01"))
        );
        println!("as of    {held_as_of}");
        println!("dates    {}", held_dates.len());
        let again = book.npv(close)?;
        println!("npv      {again:.2}");
    
        Ok(())
    }
    Python
    import datetime as dt
    
    import helixio as hlx
    from desk_sheets import SOFR_CLOSE_DATE, USD_SOFR_LADDER, USD_SOFR_PRINTS
    
    # The source answers from the desk's sheet: every pillar of every strip it is
    # asked for, observed on the day, and the prints it holds.
    class DeskSheet:
        def __init__(self, sheet):
            self.sheet = sheet
    
        def fetch(self, demand, as_of):
            asked = demand.strips.to_dict()["template"]
            strips = [
                (template, as_of, list(self.sheet[template].items()))
                for template in asked
                if template in self.sheet
            ]
            return hlx.MarketData(strips=strips, fixings=list(USD_SOFR_PRINTS))
    
    # A swap traded on 24 September has accrued three days by the close, so the
    # book reads three prints, and the USD recipe reads four strips.
    seasoned = (
        hlx.templates.get("USD-SOFR-OIS")
        .swap(dt.date(2026, 9, 24))
        .with_maturity("5Y")
        .with_notional(50_000_000.0)
        .with_fixed_rate(0.0466)
        .build()
    )
    book = hlx.Book().add("SOFR-5Y", seasoned, 1.0)
    market = hlx.MarketContext(SOFR_CLOSE_DATE).with_data_source(DeskSheet(USD_SOFR_LADDER))
    demand = market.demand(book)
    for template in demand.strips.to_dict()["template"]:
        print(f"strip   {template}")
    for series, day in zip(demand.fixings.to_dict()["series"], demand.fixings.to_dict()["date"]):
        print(f"fixing  {series} {day}")
    
    # Attaching the market fetches the market's own date; the report says what
    # came back.
    book.attach_market(market)
    npv = book.npv(as_of=SOFR_CLOSE_DATE)
    report = book.fetch_report().to_dict()
    print(f"answered {report['status'].count('answered')} of {len(report['status'])}")
    print(f"npv      {npv:.2f}")
    
    # A valuation on 2 October reads the 1 October print, which the source does not
    # hold, so the fetch for that date is refused. The book is left where it was:
    # on the close, holding the close's snapshot alone, and still pricing — a verb
    # with no date values on the close.
    NEXT_DAY = dt.date(2026, 10, 2)
    try:
        book.fetch_market_data(NEXT_DAY)
    except hlx.SnapshotError as error:
        fetch_refusal = str(error)
    else:
        raise AssertionError("the 1 October print is not on the source")
    held = book.market_context
    print(f"refused, naming the 2026-10-01 print: {'yes' if '2026-10-01' in fetch_refusal else 'no'}")
    print(f"as of    {held.as_of()}")
    print(f"dates    {len(held.dates())}")
    again = book.npv()
    print(f"npv      {again:.2f}")
    strip   USD-SOFR-3M-FUT
    strip   USD-SOFR-DEPO
    strip   USD-SOFR-MeetingOIS
    strip   USD-SOFR-OIS
    fixing  USD-SOFR 2026-09-28
    fixing  USD-SOFR 2026-09-29
    fixing  USD-SOFR 2026-09-30
    answered 7 of 7
    npv      -102966.25
    refused, naming the 2026-10-01 print: yes
    as of    2026-10-01
    dates    1
    npv      -102966.25

    A refused fetch leaves the book where it was: on the close, holding the close's snapshot, still pricing.

    API

    Symbol Meaning Rust Python
    Δt\Delta_t the demand MarketContext::demand → Demand MarketContext.demand → Demand
    fetch⁡\operatorname{fetch} the source a type implementing MarketDataSource, wrapped in DataSource::new any object with fetch(demand, as_of)
    AtA_t the answer MarketData of StripQuotes, prints and FxSpots MarketData(strips=…, fixings=…, fx=…)
    install⁡\operatorname{install} build the snapshot MarketContext::with_market_data MarketContext.with_market_data
    the whole cycle fetch for a book's date MarketContext::with_data_source, then Book::fetch_market_data MarketContext.with_data_source, then Book.fetch_market_data
    gaps an incomplete answer, refused SnapshotError SnapshotError
    Argument Type Default Meaning
    MarketContext.with_data_source(source) object | None required the source this market fetches from; None makes it offline, so it never fetches
    source.fetch(demand, as_of) your method required answers one Demand for one date with a MarketData, leaving out what it doesn't hold
    MarketData(strips=, fixings=, fx=) list[(template, date, list[(locator, quote)])], list[(series, date, value)], list[(pair, rate, date)] empty the answer: whole strips by template, prints by series and date, and FX spots by pair
    MarketContext.demand(book) Book required what the book needs that the market doesn't hold
    Book.fetch_market_data(as_of) date required fetches the demand for as_of and installs the answer
    Book.fetch_report() — — each demanded key and whether it was answered
    set_data_source(source) object | None required Python only: a process-wide default source every later market copies; None clears it

    Defaults and limits

    • What a fetch does: first the market works out what the book needs at a date that it doesn't already hold (the demand). Then the source answers what it holds of that, observed on or before the date. Finally the market installs the answer: it solves the recipes' curves from the answered strips and adds the answered prints. So a fetched market and one marked from a sheet with the same numbers are the same market. Why sources work in whole strips rather than points is covered in how a market is built.
    • What a demand names: for one date, strips (each template a recipe needs, as a whole template), fixings (each print a valuation reads), FX spots (each spot the book converts through), and unresolved curves (curves no recipe builds, which a source can't be asked for).
    • Gaps are refused together: every key the demand named and the answer didn't carry is a gap. A snapshot with gaps raises SnapshotError, naming every gap at once.
    • A refused fetch changes nothing: the book keeps the date and the snapshots it had.
    • Per-market sources: the examples set the source on each market, so two markets in one process can't share a source by accident. set_data_source exists for the process-wide case.

    !!! warning "Python fetches for you; Rust doesn't" In Python, attaching a sourced market to a book fetches the market's date, and a verb asking for a date the market doesn't hold fetches it then. In Rust, a verb on a date the market doesn't hold is refused, naming the date, and the book fetches only when you call fetch_market_data. Both languages fetch explicitly with fetch_market_data(as_of).

    See also

    • Source: crates/helixrs/src/market/source.rs, crates/helixrs/src/trading.rs, crates/helixio/src/sources.rs
    • Related pages:
      • Connect a data source, the task step by step.
      • How a market is built, for demand and install taken apart by hand.
      • How market data reaches a market, for the provider design.
      • Books carry their market, for verbs on a fetched market.

    From the 0.0 line at revision bf7f2e2. Every printed number is that revision's own output.