How market data reaches a market

If your desk runs many processes, you'll want them all to read one market-data provider. This page explains how that fits together: what a provider answers, how it meets helixio through a thin source in each process, and where storage and streaming belong. For the source protocol itself, see Data sources.

The library only pulls

The library only pulls. A market asks its source for one date, for exactly what a book needs, and keeps the answer. A provider API is the service behind that source, shared by every process. It turns vendor feeds into the library's terms once, so no process carries ticker tables or unit conversions, and it can answer any past date the same way twice.

The provider behind the source

 book.npv(as_of=d) ──► market ──► source ──► provider API ──► the provider's store ◄── vendor feeds
                                  (per        one request          (a cache, a database:
                                   process)   per date              the provider's choice)
                                                   │
                                                   └──► push notifications (optional):
                                                        "this strip moved", never the values

What the provider answers. For one date, the strips, fixings and FX spots a book's demand names, each observed on or before that date. A strip is keyed by its template and answered with every pillar the provider publishes on it. Values are in the library's units: decimal rates and spreads, futures prices, FX outrights. A key the provider does not hold is left out, and the library names it as a gap. A provider that fails says so as an error, never as a gap. It may also answer recipes and calendars, through the same source.

Storage and streaming are the provider's own. A cache such as Redis, a database of closes, a push channel such as WebSocket: each sits behind the API, and the library sees none of them. A push channel tells a process that a newer observation exists. The process then takes a fresh snapshot by fetching again, so every value still reaches the market through its source.

Why a date is fetched once. A market answers a date it already holds without asking again, so a valuation repeated later sees the same numbers. Taking a fresh snapshot is a deliberate act: the date is dropped from the market and fetched again. Mark a daily close walks through it.

What the library sees of a provider

Of the whole service, helixio sees only a source: an object that fetches a date's market data, and optionally recipes and calendars. Because a missing key is a gap and an outage is an error, a broken provider is never mistaken for a missing quote. The source a provider sits behind is defined and run on Data sources. A source over a provider API differs from that one only in where its answers come from: its fetch turns the demand into one request for the date, and the response into MarketData.

helixio-connect, an additional package in the same project, provides a provider API off the shelf. It is a versioned REST contract for such a provider, and a ready source over it, ConnectSource. It also ships a conformance check for a provider's service and a reference server to start one from. helixio needs none of it: any object with the source methods serves a market.

See also

• Source: crates/helixrs/src/market/source.rs, crates/helixrs/src/market/store.rs
• Related pages:
  • Data sources, for the source protocol, run end to end.
  • The dated market, for on, without and the fixing history.
  • Books carry their market, for verbs that fetch a date.