Design a Low-Latency Market-Data Feed

Understanding the problem

When a standalone design round appears at IMC, it is low-latency flavored, closer to how the trading systems are built than to "design Twitter." A good one to practice is a market-data feed handler, because it sits at the center of a market maker's stack. It turns an exchange's raw wire traffic into a live order book the trading strategies can read.

Read the budget before drawing anything. It inverts almost every instinct from a normal design. There is no request-per-second target to scale horizontally toward. There is one machine and one feed. The latency figure is measured in microseconds, and the best handlers push the hot path into hundreds of nanoseconds. Throughput still matters. A single busy US equity or options feed bursts into the millions of messages a second at the open. You absorb that burst on one core by doing less per message. The design is a fight for nanoseconds on a single thread, and every choice follows from that. IMC's own article How are FPGAs used in trading? describes exchange market data arriving with time measured in nanoseconds. It adds that IMC began working with FPGA-based trading systems more than a decade ago.

So the usual toolkit is mostly a liability here. A load balancer, a message queue between services, a database on the read path, a network hop to a cache: each one adds microseconds or, worse, unpredictable microseconds. Unpredictable is the enemy. The work is collapsing everything onto one pinned thread with no locks, no allocation, and no system calls on the path a message travels from the wire to the book.

The IMC Trading angle

IMC is a market maker on more than 100 exchanges, so a feed handler is the front door of the business for them. That shapes what the round rewards. IMC describes the flagship working session as a "programming test on an approximation of our actual trading system." It's graded on how you reason about performance and alternatives rather than a polished end product. A design like this probes the same instinct: can you tell which microseconds are on the critical path, and can you defend a data-structure or memory choice against a faster one. The CS-fundamentals round feeds straight into it: threads, cache behavior and memory layout are the levers you pull here.

Functional requirements

Keep the scope tight and centered on one job: keep an accurate book and get updates to the strategies as fast as the wire allows. State what you are deliberately leaving out. On a low-latency problem the things you refuse to do on the hot path are half the signal.

Non-functional requirements

On this problem the non-functional side is the entire design. Give each one a real target instead of an adjective. Latency comes first, stated as a distribution with percentiles. The number that counts is wire-to-book: the time from a packet arriving on the network card to the book being updated and the change published. Aim for single-digit microseconds. The number the strategies feel is the tail, the p99 and p99.9.

Jitter is worse than latency, and saying so separates this from web design. A handler with a steady 3 microseconds beats one that averages 1 microsecond but spikes to 50, because a strategy sizes its risk against the worst case it can see. So the whole design bends toward determinism. Cut out the things that occasionally cost a lot even when they are rare: garbage collection pauses, page faults, lock contention, a cache miss that goes to main memory. A predictable path beats a fast-on-average one.

Correctness is absolute, and it has a sharp definition here: the book must never be silently wrong. A stale book is recoverable. A book that dropped a cancel and now shows liquidity that is gone is a trap that loses money. We may miss a packet, since the network is lossy, but we must always know we missed it. So every message carries a sequence number, and a gap must be detected the instant it happens. Throughput rounds out the set. Absorb the open-of-day and volatility bursts of millions of messages a second on one thread without falling behind, because falling behind is just latency by another name.

• 1-9 µs wire-to-book budget. Judged at p99.9 over the average.
• Jitter is the real enemy. Predictable beats fast-on-average, so kill the rare expensive stall.
• millions/s peak message rate. One feed at the open, absorbed on a single pinned core.
• Zero silent loss. Every sequence number accounted for, and a gap is caught at once.

There's no availability-across-datacenters number and no horizontal-scale target. This runs in a colocated cage a few meters of cable from the exchange's matching engine, and it scales by symbol across the cores of one or two machines. Availability here means the redundant A and B lines the exchange already publishes. Planting that early stops you from designing the wrong system.

The set-up

Core entities

Name the few things the handler moves before drawing boxes. There are four, and unlike a typical design none of them live in a database. They live in memory on one thread, and how they are laid out in memory is most of the performance story.

Now the storage decision, which is where this design stops looking like any other. There is no storage tier. The entire working state is the set of order books. It lives in RAM, sized so the hot data stays in the CPU cache. A relational database, a key-value store, even a network call to Redis would each add microseconds and, worse, jitter, so none of them touch the hot path. The only durable concern is a raw capture of the inbound packets, written to the side by a separate process for later replay and audit. That capture never sits between the wire and the book. Saying "no database" with a reason is a strength.

The book is a small set of price levels updated constantly, so the data-structure choice is the design. Which structure the ladder uses (a plain array indexed by price, a hash map, or a balanced tree) is the first deep dive. It's the same "huge list versus a better data structure" reasoning IMC probes in the fundamentals round.

The interface

There is no REST here; say so. The inputs are packets off a socket. The outputs are updates handed to strategies in the same process or over shared memory, so the "interface" is a set of message types and function calls. Sketch it as the shape of the data in and the data out.

# INBOUND: decoded from the exchange's binary feed (per Channel, sequenced)
Add    { seq, instrument, orderId, side, price, qty }
Cancel { seq, instrument, orderId }                       # remove resting qty
Modify { seq, instrument, orderId, newPrice?, newQty? }   # cancel+add, or in place
Trade  { seq, instrument, price, qty, aggressorSide }     # a match occurred
Gap detected when incoming seq != lastSeq + 1  ->  enter recovery
 
# CORE: the hot path, one pinned thread, no locks, no allocation
onMessage(msg)            # decode -> apply to OrderBook -> publish delta
book.bestBid() / bestAsk()          # O(1) top-of-book read for strategies
 
# OUTBOUND: normalized deltas to local strategy consumers (shared memory ring)
BookUpdate { instrument, side, price, newQty, topOfBookChanged }

Two details in there carry weight later. First, a Modify is usually the same as a cancel followed by an add, which is why the book only really needs to handle two mutations well. Second, the outbound update flags whether the top of book changed, because most strategies only care when the best bid or ask moves, and letting them skip the rest is free latency for them.

High-level design

Build the path a single message travels, then defend every hop as either necessary or removable. The whole system is one straight line from the network card to the strategies, deliberately with as little between them as possible. The diagram codes each box by role: blue for the exchange and the strategies, violet for our processing on the hot path, amber for the lock-free handoff, green for in-memory state. The walkthrough tags each component the same way.

1) Receive with no kernel in the way. Packets arrive from the exchange feed A and B as multicast UDP and land on the NIC + kernel bypass. A normal socket would copy each packet through the kernel and wake the thread on an interrupt. Instead the handler busy-polls the card from user space and reads the bytes directly. Busy-polling means one core spins in a tight loop asking "is there a packet yet," burning CPU on purpose so it never pays the cost of being woken up. That trade sacrifices a whole core to spinning. It buys the single biggest and most predictable latency win on the path, and it is the first thing to say.

2) Decode and clean the stream. The Decoder casts the fixed-offset binary fields into a typed Message right on the received buffer, with no allocation. Then line arbitration merges the two feed lines into one. For each sequence number it keeps whichever of A or B arrived first and discards the duplicate, so a drop on one line is invisible as long as the other delivered. In the same step it checks the sequence is contiguous. What we keep off the path here is any text parsing or object creation. The message is already the right shape in the bytes we received. The bottleneck we are punting: when both lines drop the same packet, the sequence jumps and we have to recover without serving a wrong book, which is a deep dive.

3) Apply to the book. Each in-order Message mutates one side of one order book in RAM. An add increases the quantity at a price level, a cancel decreases it, a level that hits zero is removed, and the best bid and ask are kept reachable in constant time. This step runs millions of times a second. It has to be a handful of memory writes that hit cache, with no tree walk or hash lookup that misses to main memory. The bottleneck we are punting: which data structure the price ladder uses decides whether this step is a few nanoseconds or a few hundred, and that is the first deep dive.

4) Publish without waiting. When the book changes, the handler writes a small BookUpdate into the SPSC ring buffer and moves on. It never blocks on the strategy engines. If a consumer is momentarily slow, the ring absorbs the gap and the handler keeps processing the wire. The one thing it can never do is fall behind the feed. A lock here would be a disaster: if the consumer held it, the hot thread would stall for an unbounded time. The bottleneck we are punting: doing all of this per instrument across many cores without locks or false sharing, and proving the tail stays flat, is the scaling deep dive.

Technology choices

Name a concrete choice for each hop and be ready to defend the category, because the follow-ups here are all "why not the faster or simpler thing." Nearly every default from web design is a source of jitter, so the better answer usually removes a layer.

• Kernel bypass for receive, via Solarflare Onload / ef_vi or DPDK. The category is "get the kernel off the path." A standard socket copies each packet into kernel memory, then wakes your thread with an interrupt and a context switch. Together those cost microseconds and, worse, vary run to run. Bypass maps the card's buffers into user space so the thread reads packets with no system call. The cost you accept is a core spinning full-time and code that is harder to write and debug, which is why you only pay it on the hottest path.
• C++ (or Rust) for the hot path, over Java with a normal collector. The category is "a language with no unpredictable pauses." A garbage collector can stop the world for milliseconds at an unknown moment, which is fatal when your budget is microseconds, so the runtime itself is a jitter source. On the JVM, you run with no allocation on the hot path and off-heap buffers so the collector has nothing to do. That's really imitating C++. Say the tradeoff plainly; the working session grades this kind of performance reasoning.
• A pre-allocated array indexed by price for the order book ladder, with a hash map from order id to its location for cancels. The category is "constant-time updates that stay in cache." A balanced tree gives you sorted prices but every add is a pointer-chasing walk that misses cache; an array keyed by price ticks turns an update into a single indexed write. The failure mode to name is a sparse or huge price range blowing up the array, which the deep dive handles. This is the choice IMC probes most directly.
• A lock-free SPSC ring buffer in shared memory for the handoff to strategies. The category is "hand data across threads without ever blocking the producer." A mutex-guarded queue can stall the hot thread for as long as the consumer holds the lock, which is unbounded. A single-producer, single-consumer ring needs only two atomic counters and a memory fence. If the consumer falls behind, the ring fills and you drop or overwrite oldest by policy. The handler never waits, because falling behind the wire is the one failure with no recovery.
• CPU pinning, NUMA-local memory, and huge pages underneath all of it. The category is "stop the operating system from moving your work around." Pin the handler thread to one core and keep interrupts and other processes off it. Allocate its memory on the same NUMA node as that core so reads don't cross the slow inter-socket link. Use huge pages so the address translation cache (the TLB) misses far less often. Each removes a class of rare, expensive stall that would otherwise show up in the tail.
• Deliberately no message queue, no service mesh, and no database anywhere on the path. At microsecond budgets, a network hop to Kafka or Redis is thousands of times over budget, and a lock or a disk write is a jitter bomb. Reaching for that infrastructure here signals someone who pattern-matched "system design" to "distributed pipeline" without reading the numbers. Say you considered them and rejected them.

The IMC Trading angle

Deliver that last bullet with conviction; it's where IMC's round diverges hardest from a FAANG one. Their interviewer has said the assessment is graded on how you reason about performance and alternatives. Breadth counts for little. A fleet of microservices answering a single-thread nanosecond problem reads as the wrong instinct. Do the reverse. Point at the budget, strip the path to one pinned thread with no locks and no allocation, and defend each removal. Keeping both feet on the ground, one of their stated values, looks exactly like refusing to over-build a system that runs on one core.

Potential deep dives

The skeleton is short on purpose. Almost all the signal is in the next four questions. An interviewer will pick one or two and push hard on the performance reasoning. The first two are where the round is usually decided.

1) What data structure holds the order book?

Start with the access pattern, because the structure has to serve exactly it. Adds and cancels arrive constantly and each targets a specific price. A cancel names an order id rather than a price, so you also have to find where that order lives. And strategies read the best bid and best ask far more often than anything else. So the structure needs three things fast: update the quantity at a price, look up an order by id, and read the top of book. Everything below flows from ranking those.

The tree is an acceptable answer, and for many candidates the dive ends there. The upgrade comes from noticing that prices move in fixed increments called ticks. An active book only ever uses a tight band of price levels near the touch. That regularity means you do not need a general sorted structure at all. You can index prices directly.

The IMC Trading angle

This is the deep dive most aligned with what IMC tests elsewhere in the loop. The fundamentals round asks about a huge list versus a better data structure, about hashing, about sorting tradeoffs. Here that question has a real job. Push on the array-versus-tree tradeoff the way they will: name the big-O, then override it with the cache argument. A logarithmic tree can lose to a linear-looking array once memory latency dominates. Knowing that shows the low-latency mindset the firm runs on.

2) A packet is lost on both lines. How do we recover without a wrong book?

Frame the failure precisely first. The feed is UDP, so packets can be dropped or reordered. The exchange guards against that by publishing two identical lines, A and B. Arbitration hides any loss that hits only one line. The problem is the rarer case where the same sequence number is missing from both. The gap check catches it the moment an incoming sequence is not exactly one past the last. From that instant the book is suspect, because you have missed one or more mutations and cannot know what they were. Acting on a book that is missing a cancel can mean quoting against liquidity that no longer exists.

What scores here is treating a detected gap as an expected event with a defined, bounded response. It's more than an error to log. Keep the two guarantees straight while you explain it: we accept that we may lose packets, and we refuse to ever serve a book we know is incomplete as if it were whole.

3) How do we handle every instrument without locks or false sharing?

Scaling here means more cores on the same machine. The trap is that naive threading makes it slower. One thread cannot keep up with every instrument on a busy day, so you split the work. The rule that makes the split safe is the single-writer principle: assign each instrument to exactly one core, and only that core ever writes that book. No two threads touch the same book, so there is nothing to lock. Locking is what you are designing away. A lock on the hot path can stall a thread for an unbounded time and destroys the tail.

Shard by instrument across cores, and route each feed's messages to the core that owns that instrument. Then name false sharing, a favorite probe. Two cores writing to different variables that happen to sit on the same 64-byte cache line will fight over that line as if they shared the data. It quietly serializes work that looked independent. The fix is to pad and align the per-core structures so each core's hot data lives on its own cache lines. IMC's fundamentals round checks this memory-and-cache knowledge, and here it pays off.

Then measure the tail and the jitter, since average latency hides them. Record wire-to-book timings into a latency histogram and watch p99 and p99.9, ideally with hardware timestamps from the network card so you are measuring the real path and not your own clock reads. When the tail spikes, the causes are a known short list. A page fault calls for pre-faulting and locking memory, a TLB miss for huge pages, and the scheduler stealing your core for pinning the thread and isolating the core. A NUMA-remote access means keeping memory node-local, and some cache misses you can prefetch away. Naming that checklist unprompted is a strong senior signal, because the tail gets engineered down on purpose.

4) Is the fastest design even the right business call?

A strong candidate steps back here. Chasing nanoseconds has real costs, so spend them where they pay. Kernel bypass, no allocation, a spinning core and hand-tuned cache layout all buy speed. They cost engineering time and hardware, and they make code harder to change and reason about. Which paths deserve that?

Not all of them, and saying so shows judgment. The feed handler and the order book sit on the critical path between a market event and a trading decision, so they earn the full treatment. But packet capture, monitoring, the recovery path, and the analytics over captured data are off the hot path. There the right call flips: use ordinary, maintainable code, a normal language, a database, because their latency does not reach the market. Spending the extreme optimizations there would be waste. Match the effort to whether the microseconds sit on the path a trade travels. The firm makes the same call when it decides what to build in C++ and what to leave in Python.

The IMC Trading angle

This is the register the working session is really listening for. IMC says it grades how you reason about performance and alternatives. That means knowing which microseconds matter and defending the trade. Optimize the hot path ruthlessly, then leave the capture-and-analytics side simple on purpose. Raising that before being asked reads as senior.

What is expected at each level

Leveling matters for who gets this round. A standalone low-latency design round shows up mainly for experienced and senior hires. New-grad and early-career loops center on the working session and the CS-fundamentals round instead. There the same thinking (memory layout, cache behavior, data-structure tradeoffs) tends to surface in code rather than as a whiteboard design. Read the level you are interviewing for accordingly.

Early-career (working session and fundamentals)

You are unlikely to be handed a blank low-latency design, but the ideas here are your edge in the rounds you do get. The fundamentals round asks about array versus tree versus list, why a cache miss can beat big-O, and what a thread and a lock cost. With the order-book example ready, you give concrete answers instead of definitions. In the working session they grade how you reason out loud about performance and the alternative you didn't pick. The bar: show the low-latency instinct in your code and your explanations, even without the formal design prompt.

Experienced and senior

This is where a design like this can run as its own long session. Reach a clean single-thread path from wire to book, then drive the data-structure and recovery deep dives yourself. Defend the array-over-tree choice on cache grounds. Treat a detected gap as a bounded snapshot-resync problem, and never leave the book silently wrong. You should reach for kernel bypass, no allocation, and the single-writer split without prompting, and defend each against the simpler option. The bar: you find the latency and correctness bottlenecks yourself and back your choices with the mechanism.

Staff and above

Depth leads and the scope widens past one handler. Expect to reason about the tail across many cores and instruments, including false sharing, NUMA and how you'd prove the p99.9 stays flat under a burst. Expect the sharding scheme across machines too. Then comes the judgment call about which paths deserve extreme optimization and which stay simple. The platform question is fair game. This is one feed among a hundred exchanges, so how does the pattern carry to the next venue, with a different protocol and snapshot mechanism? The bar: you own the latency, correctness, and cost argument end to end, and the interviewer redirects you rather than rescues you.