Market Data Infrastructure at Scale: Ingesting, Normalising, and Distributing Real-Time Financial Data

Every trading system ultimately depends on one thing: market data. Get it late, get it incomplete, or get it wrong, and everything downstream — order routing, risk checks, pricing models, backtesting — is compromised. Yet market data infrastructure is the least glamorous and most under-invested part of most trading platforms. Firms spend millions on execution algorithms and risk models while running market data through a decade-old, single-threaded ticker plant that drops packets when volume spikes.

Market data pipelines at trading firms exhibit a consistent pattern: the existing system works at normal volumes, fails silently during market events, and data gaps go undetected until a trade breaks. The fix is not a faster machine. It is an architecture designed for the scale, diversity, and reliability requirements of modern trading.

Who Is This Guide For?

This guide is for quantitative developers, data platform engineers, and trading infrastructure architects building or modernising market data pipelines. If you consume data from exchanges, normalise it for downstream systems, or distribute it to trading desks — this is for you.

By the End of This, You’ll Know…

• What a modern ticker plant architecture looks like and how it differs from legacy designs
• How to normalise market data across exchanges with different protocols and data models
• How to distribute market data to consuming systems with sub-millisecond latency
• When to build on-premise and when cloud market data infrastructure makes sense
• What the SEC’s market data infrastructure modernisation means for your architecture

The Ticker Plant Problem

A ticker plant ingests raw market data feeds from exchanges — typically UDP multicast streams carrying order book updates, trade reports, and instrument reference data — normalises them into a common format, and distributes them to consuming systems. The challenges are scale, diversity, and correctness.

Scale: A single US equity exchange can produce over 10 million messages per second during peak activity. Across all US equities exchanges, aggregate peak throughput exceeds 50 million messages per second. Your ticker plant must process every message, maintain order books for every instrument, and distribute updates without dropping a single message during microbursts — the sub-second spikes that can be 5-10x the average rate.

Diversity: Each exchange has its own protocol. NASDAQ uses OUCH and ITCH. CME uses MDP 3.0. ICE uses iMpact. Eurex uses Enhanced Order Book Interface (EOBI). European exchanges use FIX and native protocols. Your ticker plant must speak all of them, normalise their different data models — different ways of representing order types, trade conditions, and market states — into a single internal format that downstream systems can consume consistently.

Correctness: The order book is the source of truth for all downstream decisions. A missed message — a dropped trade report, a lost order book update — creates a gap that propagates into every calculation that depends on the order book. Gap detection and recovery must be built into the ticker plant at the protocol level, not handled as an afterthought by consuming systems.

The Modern Ticker Plant Architecture

The architecture that works at production scale uses three layers: ingestion, normalisation, and distribution.

Ingestion Layer: Dedicated feed handlers — one per exchange per market — receive UDP multicast streams on dedicated network interfaces with kernel bypass (DPDK or Solarflare OpenOnload). Each feed handler decodes the exchange-specific binary protocol into in-memory message objects. The handlers run in a tight loop, pinned to dedicated CPU cores with interrupt affinity set to prevent kernel preemption. In 2026, Aeron’s Media Driver has become the standard transport for this layer because it provides efficient UDP multicast reception with NAK-based loss recovery — covered in detail in our low-latency messaging guide.

Normalisation Layer: Decoded messages flow to normalisers — one per exchange — that map exchange-specific message types to a common internal data model. A NASDAQ trade report and a NYSE trade report become the same internal Trade message. An order book update from CME becomes the same internal BookUpdate as one from Eurex. The normalisation layer maintains the canonical order book per instrument, applying incremental updates and detecting gaps. If a sequence number gap is detected, the normaliser requests retransmission from the exchange’s recovery feed — every major exchange provides a TCP-based retransmission service for gap recovery.

Distribution Layer: Normalised market data is distributed to consuming systems. The distribution mechanism depends on latency requirements. For trading desks that need sub-millisecond access, Aeron IPC (inter-process communication over shared memory) delivers normalised messages directly into the consuming application’s address space. For systems that tolerate higher latency, Kafka topics carry normalised market data with persistence and replayability — useful for backtesting, strategy research, and regulatory archiving. A typical deployment runs both paths in parallel: Aeron for real-time trading and Kafka for everything else.

Cloud-Native Market Data: When It Makes Sense

Running a ticker plant in the cloud was considered impossible until about 2023. The latency requirements seemed incompatible with virtualised networking. In 2026, that has changed — for certain use cases.

For market-making desks running nanosecond-level strategies in colocation, on-premise is still the only option. The physics of distance — even a few kilometres of fibre between the cloud region and the exchange — adds microseconds that market-making strategies cannot tolerate.

For everything else — execution routing, quantitative research, backtesting, strategy development — cloud-native market data infrastructure is production-grade. GCP’s C3 bare-metal instances with DPDK-backed networking deliver single-digit microsecond latency for UDP multicast processing. AWS Nitro instances provide comparable performance. The key architectural difference is that in the cloud, you do not receive exchange multicast directly. You subscribe to a consolidated market data feed from a provider — Refinitiv, Bloomberg, Exegy, or BMLL — which ingests exchange data at the source and distributes normalised feeds through cloud-native interfaces.

The cost economics are compelling. Running an on-premise ticker plant for US equities costs roughly $500,000-750,000 per year in hardware, colocation, connectivity, and operations. A cloud market data feed with equivalent coverage costs $200,000-400,000 per year in data licensing and compute. For firms that are not market-making, the cloud option is increasingly the default.

The SEC’s Market Data Infrastructure Reform

The SEC’s market data infrastructure rule, approved in 2025, is changing how market data is distributed in US equity markets. The rule requires exchanges to make depth-of-book data available through competing consolidators rather than exclusively through proprietary feeds. In practice, this means that firms can now access full order book data from all exchanges through a single consolidated feed — the kind of data that previously required separate contracts with each exchange and dedicated ticker plants for each protocol.

This has two implications for infrastructure architects. First, the consolidation layer shifts upstream — you can consume normalised, multi-exchange market data from a single provider rather than building feed handlers for every exchange. Second, the competitive landscape for data providers is opening up, which should reduce costs and improve data quality as providers compete on latency and coverage.

Further Reading

• NYSE Real-Time Market Data — Official feed specifications
• SEC: Market Data Infrastructure Reform Statement
• Diginomica: How Real-Time Data Streams Are Transforming Trading

For more on trading infrastructure, see our guides to low-latency messaging, financial data platforms, and our piece on capital markets infrastructure in 2026.

Related reading: Trading Systems & Market Infrastructure Guide · Aeron vs Kafka vs Chronicle Queue · trading systems practice