A Low-Latency Order Book

Understanding the problem

First, clear up what "system design" means at a firm like Jump, because it is not the interview that phrase usually names. There is no "design Twitter" here, no load balancers fronting a fleet of stateless services, no sharded database behind a CDN. The design round for a core developer is a low-latency systems problem. You are handed one machine, a network card, and a firehose of market data, and asked to turn it into a correct, current picture of the market fast enough to act on. So the skill on display is not how you spread work across a data center. It is how you move a single message from the wire into a data structure and back out to a strategy without wasting a microsecond.

Picture the object at the center. An order book for one instrument has two sides. The bids are resting buy interest, ranked from the highest price down. The asks are resting sell interest, ranked from the lowest price up. At each price sits an aggregated quantity, the total size willing to trade there. The best bid and best ask are the two prices closest together, and the strategy watches them constantly, because that top of the book is where the next trade will happen. Your job is not to match orders. The exchange does the matching and publishes the results. Your job is to keep a faithful copy of that book, the same book every other participant sees, updated the instant a message arrives.

That copy is built from a feed of small deltas. The exchange does not resend the whole book on every change. It sends incremental messages: add ten lots at 5000.25, reduce the size at 4999.75, delete the level at 5001.00, print a trade for five lots. Each message carries a sequence number so a receiver can tell whether it has seen every update in order. Apply the deltas in order and your book stays a mirror of the exchange's. Miss one, or apply two out of order, and your book quietly diverges from reality, which for a strategy trading off it is worse than having no book at all.

Two forces set every number in this design before a box goes on the board. The first is that the feed is bursty in the extreme. A liquid futures contract ticks along at a manageable rate most of the day. Then a scheduled economic release or a large sweep fires hundreds of thousands to millions of messages in a few milliseconds. The book has to absorb every one without falling behind. The second is that the value of the book decays in microseconds. A strategy acts on the state of the top of the book, and a book that is even a few microseconds stale is describing a market that has already moved. Put those together and the target is stark. Apply each message and answer a best-bid-or-ask query in single-digit microseconds, at the worst instant of the worst burst. Nothing on the hot path can stall.

The Jump Trading angle

Jump is a futures-first high-frequency firm, founded in Chicago in 1999 by two ex-CME floor traders. Its edge lives in this layer: how fast a message becomes a decision. That heritage shapes the round. The interviewer is usually a working engineer, not a recruiter, and the session runs on a shared editor where you sketch and defend your reasoning out loud. Expect the drill to go a level deeper than a big-tech design round. Past "what structure holds the book," they ask what each operation costs and what the cache does during a burst. They ask why a lock-free handoff beats a mutex, and what you would measure at the p99.9. Have a real answer to "why Jump and not Hudson River or Citadel Securities." At a firm with this specific an identity, a generic "top prop shop" answer says you didn't look closely. The honest answer usually runs through the futures and CME microstructure heritage, and the appetite for engineers who care about a single core's behavior.

Functional requirements

Propose a tight core and name what sits outside it, because half the scoping skill is refusing the work that belongs to a neighboring system. The book builder consumes normalized feed events, maintains the book, and answers queries from the strategy. Matching, order entry, and wire decoding all live next door.

Non-functional requirements

These are the requirements the round is graded on, so put a target on each. Latency leads. Applying one delta is a few operations against an in-memory structure, so the median sits well under a microsecond, and the number worth defending is the tail. The p99.9 apply time is where a cache miss, a branch mispredict, or a stray allocation turns a sub-microsecond operation into a multi-microsecond one, and taming that jitter is most of the engineering. Say plainly that you optimize the worst case here. Picture a strategy that is fast on the easy 99.9 percent of messages and slow on the 0.1 percent that arrive during a news burst. It is slow at exactly the wrong moments.

Throughput rides alongside latency and comes from the same choices. A single writer updating an in-memory book has to sustain the burst rate, on the order of millions of messages a second at peak. There is no lock to contend and no allocation to stall on. Correctness is not negotiable: the book must stay a faithful mirror of the exchange, which means gaps are detected and repaired, and the strategy never reads a torn state. Durability barely registers, and that is a real simplification worth stating out loud. The book is derived from the feed, the feed is replayable from the exchange snapshot, so a process that dies loses nothing it cannot rebuild in milliseconds. Nothing here writes to disk on the hot path, and admitting that early kills a lot of machinery a database-first instinct would drag in.

• Apply latency: p50 well under 1us. One delta is a handful of in-memory operations; the p99.9 tail during a burst, not the median, is the number you defend
• Throughput: millions of msgs/sec in a burst. A single lock-free writer must absorb a news-driven spike without falling behind the wire
• Correctness: a faithful mirror of the exchange. Sequence-checked, gaps detected and resynced from a snapshot, and the strategy never reads a torn book
• Recovery: rebuild from the feed snapshot. The book is derived state; the replayable feed is the source of truth, so a crash costs milliseconds, not data

The set-up

Core entities

Pin the objects down before drawing, because the interviewer will drill straight into how each one is laid out in memory. Four carry the design.

One boundary organizes everything below, so name it now. There is the feed, the durable, replayable, totally ordered stream of what the exchange published, and there is the in-memory Book Builder that turns that stream into a book the strategy reads. The feed is the truth and the book is a fast view computed from it. That split is the whole reason durability is cheap here: the book can vanish and be rebuilt by replaying the feed. Ordering and truth live in the feed, speed lives in memory, and recovery is nothing more than replaying the feed into a fresh book. Almost every deep dive is about one side of that line or the bridge across it.

API

This is a library interface on one machine, not a network API, so there are no RPCs and no round trips to reason about. There is a write path the feed drives and a read path the strategy calls, and the two share the book. The whole design is about the write side applying an event and the read side answering a query, both in microseconds, without the reader ever catching the writer mid-update.

Two parts of this contract carry the round. First, onEvent is the single writer. Nothing else mutates the book, which is what lets the whole thing run without locks. Second, the reads have to be answerable while the writer is mid-flight, because the strategy and the feed run at the same time. A naive answer bolts a lock around the book and calls it done, and it works right up until the lock's cost shows up in the tail. Giving the reader a consistent book without ever blocking the writer is a deep dive of its own. The interface keeps that door open. The reads are pure and small, so an atomically published view can serve them.

High-level design

Draw the shortest path from the wire to a decision, then follow one message down it. A packet arrives at the network card and the feed handler decodes it into normalized events. The events cross a lock-free handoff to the book builder, which applies them to the in-memory book. The strategy reads the updated top of the book. Four stages, deliberately split, so the one writer that owns the book is never sharing a thread with packet decoding or with the strategy's own work. Each color in the diagram marks a role. The Exchange feed arriving at the card is blue. The services are violet: the Feed Handler that decodes and normalizes, the Book Builder that owns and mutates the book, and the Strategy that reads it. The amber SPSC ring is the lock-free queue between the two threads. The green Snapshot recovery is the durable backstop the book is rebuilt from after a gap.

1) Packets arrive and are normalized. The Exchange feed lands at the network card. With kernel bypass, packets go straight into the Feed Handler's memory, skipping the operating system's network stack. That alone saves microseconds per packet. The handler decodes the exchange's binary format, turns each field into a typed BookEvent, and checks that the sequence number is exactly one past the last. The work deliberately kept here, off the book writer, is all the parsing and validation, so the writer only ever sees clean events. The bottleneck being punted: packets are UDP and can arrive dropped or out of order, and detecting and repairing that without corrupting the book is the recovery deep dive.

2) Events cross a lock-free handoff. The handler writes each normalized event into the SPSC ring, and the Book Builder reads them out on its own pinned core. The two threads share only this ring, and because exactly one thread writes it and one reads it, the handoff needs no lock at all, just a pair of atomic indices. This is what lets decoding and book-building run in parallel without either stalling on the other. The bottleneck being punted: getting that ring genuinely wait-free, and keeping the producer's and consumer's indices from thrashing the same cache line, is the lock-free-handoff deep dive.

3) The single writer applies the delta. The Book Builder pulls one event and mutates the book. It adds or reduces size at a price, deletes a level, unlinks a cancelled order, or records a trade. It keeps the best-bid and best-ask handles current as levels empty and fill, and updates each level's aggregate quantity so a depth read stays cheap. Because one thread owns the book and processes events in sequence order, the book is a faithful function of the feed. The bottleneck being punted: which data structure makes read-best, insert, and cancel all cheap at once is the order-book-structure deep dive, and it is the single most-drilled question in the round.

4) The strategy reads a consistent view. The Strategy runs concurrently and reads the top of the book on nearly every message. It never locks the book, because a lock on the writer's hot path is exactly the cost this whole design exists to avoid. Instead the writer atomically publishes a small consistent view of the touch, and the reader picks it up whole. The bottleneck being punted: publishing that view so the reader never catches a half-finished update, without ever making the writer wait, is the consistent-read deep dive.

Throughput, latency, and book size

Put numbers on the hot path, because here the sizing is the design, not a footnote. Start with the budget. A modern pinned core executes on the order of a billion simple operations a second. A message that costs a few hundred instructions and touches a couple of cache lines takes a small fraction of a microsecond when the layout is right. That is what puts the median apply well under a microsecond and lets a single writer sustain several million messages a second. Size against the burst. A scheduled release can compress a normal minute of traffic into a few milliseconds. Provision the ring and the writer for the spike rate. The ring must be deep enough to absorb a short burst without the producer ever finding it full.

Now the book itself. A liquid futures contract rests on the order of tens of thousands of orders across a few thousand price levels. At roughly 64 to 128 bytes per order, that is only a few megabytes. That number matters because it fits in a core's cache many times over, so the levels near the touch, the ones almost every message and every read hit, stay in fast memory. The whole reason the structure is fast is that its hot data never has to leave the cache. Keep the working set small and adjacent, and a burst of updates walks memory the CPU already has. Let the structure sprawl with pointers to far-apart nodes, and each update pays a cache miss. That's a hundred-odd nanoseconds landing squarely in your tail.

• Apply cost: ~hundreds of instructions. A few cache lines touched per delta; sub-microsecond at p50 when the hot levels stay in cache
• Burst rate: millions of msgs/sec at the peak. A news release compresses a minute of traffic into milliseconds; size the ring and writer for the spike
• Book size: ~a few MB per instrument. Tens of thousands of orders over thousands of levels; small enough to stay cache-resident
• Cache miss: ~100ns, and it lands in the tail. The gap between a hot dereference and a pointer chase to main memory is most of the p99.9 story

Technology choices

Every box so far is a placeholder. Put a real technology behind each and be ready to defend it, because the sharpest follow-ups are the failure questions and the "why this and not the obvious thing" questions. For each pick, the reason that kind of tool fits and how the design behaves when it is stressed.

• The Book Builder is custom C++, not a database or a managed runtime. The category fits because the book is hot, fits in RAM, and is touched on every message. A database round trip per update would blow the microsecond budget by orders of magnitude. A garbage-collected language could pause the writer mid-burst at the worst possible instant. C++ gives manual control over memory layout and lifetime, which is the whole game here. When the writer dies, nothing durable is lost, because the book is derived state: a fresh process rebuilds it from the exchange snapshot in milliseconds.
• The handoff off the wire is a lock-free single-producer, single-consumer ring buffer, in the spirit of the LMAX Disruptor. A ring is the right shape because the relationship is exactly one writer and one reader, which is the one case you can synchronize with two atomic indices and no lock at all. Design around its failure mode. If the consumer falls behind and the ring fills, the producer has to drop or block. So the ring is sized for the burst, and the consumer stays fast enough that it never happens. A mutex-guarded queue here would be simpler and would also put a lock acquisition on the hot path, which is precisely the cost to avoid.
• The book structure is an array of price levels indexed by tick for a liquid instrument, with a sorted map as the general fallback. A hash map from order id to node gives constant-time cancel. The array fits because exchange prices move in fixed ticks inside a sensible band. A price becomes an array index and the best price is one dereference. Adjacent prices sit adjacent in memory for the cache. When a price falls outside the band, the fallback structure catches it. This trade, generality for cache-friendliness, is the heart of the structure deep dive.
• The wire path is kernel-bypass networking on a specialized card, such as Solarflare or a DPDK-driven NIC. The category fits because the operating system's network stack copies each packet and can context-switch on it, costing microseconds the budget cannot spare, and kernel bypass delivers packets straight into application memory instead. The failure story is graceful. If a packet is dropped on the wire, which UDP allows, the sequence-number check catches the gap and the recovery path rebuilds. Speed and correctness are handled by different mechanisms.
• The runtime environment is a pinned, isolated core with NUMA-local memory and huge pages, which is less a product than a set of settings that matter as much as the code. Pinning the writer to one core and keeping the operating system's other work off it removes scheduler jitter. Allocating the book in the memory bank local to that core avoids a slow cross-socket hop on every access. Huge pages cut the cost of address translation. These are the knobs a latency shop expects you to name, because the fastest data structure still stalls if the core it runs on keeps getting interrupted.

State one split plainly, because the whole design rests on it. The feed is the durable, ordered, replayable truth. The book is a fast in-memory view computed from it, and the strategy reads that view. Say clearly which state is derived and disposable, the book, and where correctness comes from, the sequence-checked feed and the snapshot behind it. Name the brand if pressed, Solarflare, DPDK, the Disruptor pattern, but it is the category and the failure story that carry the round. An in-memory writer, because the book is hot and small. A lock-free ring, because the handoff is one-to-one. Kernel bypass, because the OS stack is too slow. And the whole thing rebuildable from the feed, because a crash must cost milliseconds and not data.

Potential deep dives

The skeleton is the easy part, and almost all the signal is in what follows. The interviewer picks one or two and pushes until either the latency budget breaks or the correctness does. The five below run in the order the design hangs together. What structure holds the book so every operation is cheap. How the handoff off the wire stays lock-free. How the strategy reads a live book without ever seeing it torn. How the book survives a dropped or reordered feed. And how the p99.9 tail is tamed by shaping code and memory for the hardware.

1) The order-book data structure, and the "why that structure" drill

Start with the book, because everything else is only as fast as the structure the writer walks. List the operations before naming any structure, because that list is what the interviewer wants to hear you reason from. Read the best price on each side, constantly, since the strategy asks on nearly every message. Insert size at a price, or create a new level if none exists there. Reduce or delete size at a price. Cancel a specific resting order by id. And read the aggregate size at a level for a depth query. One fact decides the layout. On an order-by-order feed, cancels and modifies outnumber trades by a wide margin. A cancel that has to search is the difference between a book that keeps up in a burst and one that falls behind. Every one of those operations has to be cheap at once, and the naive structures each make one of them slow.

What pushes past the sorted map is mechanical sympathy, the habit of shaping data the way the hardware reads it fastest. The move rests on a fact about markets: prices are not arbitrary real numbers. They move in fixed ticks inside a sensible band around the current price, so a price can be an index into an array instead of a key to be searched for.

The Jump Trading angle

This drill decides the round. It's why the order book shows up as a live coding exercise before it is ever a whiteboard question. A big-tech interviewer is often satisfied once your solution is correct and the right big-O. This room assumes both and asks the next layer. What does each operation cost? What is the cache doing during a burst? Why an intrusive list instead of a standard-library one? What would you profile before committing? Lay out the operations the structure must make cheap. Then name the three-layer answer: sorted or tick-indexed levels over FIFO queues, with an id index for constant-time cancel. Say plainly that a heap allocation per order is disqualifying at a latency shop. That's why the orders come from a pool. Reach for the array-of-ticks layout only after justifying it from the workload rather than reciting it. That sequence, operations first and structure second, is what the engineers watching you type are listening for.

2) The lock-free handoff off the wire

Decoding packets and updating the book are different jobs. Put them on one thread and a burst of parsing work stalls the book. Split them across threads and they have to hand data back and forth many millions of times a second. The obvious handoff is a queue guarded by a mutex, and it is exactly the wrong tool. A mutex puts a lock acquisition, and the risk of the writer being blocked or descheduled while holding it, right on the hottest path in the system. The whole reason to have two threads is speed, and a lock gives a chunk of it back. The relationship here is special in a way that lets you drop the lock entirely: exactly one thread ever writes the handoff and exactly one ever reads it.

That one-writer-one-reader shape is what a single-producer, single-consumer ring buffer is built for. Picture a fixed-size array with two indices, a write position the producer advances and a read position the consumer advances. The producer writes the next slot and then publishes the new write index; the consumer reads up to the published index and then advances its read index. Only one thread touches each index. So the only synchronization needed is that each side sees the other's index update in the right order. A single atomic with the right memory ordering provides that, with no lock anywhere. The buffer is preallocated once, so the hot path never calls the allocator, and its fixed size is chosen to swallow a burst so the producer never finds it full.

3) Reading a live book without tearing it

The writer and the strategy run at the same time, and that raises a real hazard. Suppose the writer is updating the best bid: it lowers the price at one field and is about to write the new size at another. If the strategy reads in between, it sees a price from the new state and a size from the old one, a book that never existed. Acting on a torn read like that is how a strategy trades on a phantom. So the reader must always see an internally consistent snapshot of the touch, and the writer must never block. Slow the writer to protect the reader and you have lost the latency the design was for.

The Jump Trading angle

Name this hazard before the interviewer does. It shows you treat the reader and writer as truly concurrent instead of taking turns. Reach for the lock, catch yourself, and explain why a lock on the writer is the one thing you can't afford. Then offer the atomic-publish or seqlock answer and say which you'd pick for which read. At a firm built around single-core latency, "the writer never blocks" is the property being graded. The round rewards giving the reader a consistent view without ever violating it. Be exact about the memory ordering that makes it correct.

4) Surviving a dropped or reordered feed

The feed arrives as UDP, which is fast and unreliable by design: a packet can be dropped, duplicated, or delivered out of order. The blast radius of ignoring that is total. Miss one delete and your book keeps a level the market has removed. Apply an update whose predecessor you never saw, and your book is silently wrong from then on. A strategy trading off a wrong book loses money confidently. So the book has to know the instant it is no longer trustworthy and repair itself without stopping the flow of live updates that keep arriving during the repair.

Detection is the sequence number every event carries. The feed handler tracks the expected next sequence, and the instant an arriving event is not one past the last, it knows a gap opened. Exchanges also help on the delivery side. A feed is commonly published on two identical lines, an A feed and a B feed. A receiver listens to both and takes whichever packet arrives first for each sequence number. A packet dropped on one line usually still arrives on the other. That arbitration removes most gaps before they ever reach the book. What is left is the rare gap that both lines missed, and that is what recovery handles.

The Jump Trading angle

Jump's futures and CME heritage shows up here in the technical detail, so knowing how real exchange feeds behave earns points. Think of the two-line A/B delivery and the incremental feed paired with a periodic snapshot channel. Add the sequence numbering that makes gaps detectable. A futures market maker's market-data infrastructure runs on these mechanics. Treat them as first-class and you show you've thought about production feeds as well as the data structure. The instinct to protect is simple. The book announces when it can't be trusted and refuses to serve a strategy a book it knows is wrong. A firm with real money on the line wants that conservatism.

5) Mechanical sympathy and the p99.9 tail

The median apply time is the easy number, and it is not the one you defend. The tail is the p99.9 message, the one that arrives in the thick of a news burst. There a sub-microsecond operation balloons into several microseconds. Those are the moments a strategy most needs the book to be fast. Every spike in that tail has a physical cause, and the low-latency craft is knowing them and removing them one by one. The interviewer's follow-up here is "show me the p99.9 and tell me what causes it." A concrete list separates someone who has profiled this from someone who has only reasoned about it.

Walk the causes in the order they bite. A cache miss is the most common. If the hot levels have drifted out of cache, an update pays a hundred-odd nanoseconds to fetch from main memory. So the book is laid out compact and tick-adjacent to keep the touch resident. An allocation is next. Calling the allocator on the hot path can take unbounded time and fragment memory. Orders come from a preallocated pool, and nothing on the path calls new or delete. A page fault will freeze the thread while the OS maps memory, so the process pre-faults and locks its pages up front. A branch mispredict on a cold path flushes the pipeline, so the hot path is kept straight and branch-light. And the scheduler itself is a source of jitter, so the writer is pinned to an isolated core the OS keeps its other work off of.

Two more settings matter on modern hardware and both are worth naming. Memory is not uniform on a multi-socket machine. Memory attached to another socket costs more to reach than your own. So the book lives in the memory bank local to the writer's core. That's NUMA-aware placement. Address translation has its own cache, the TLB, which thrashes when a large working set is mapped in small pages. Huge pages cover more memory per translation entry and cut those misses. None of this is a heroic algorithm. It is a discipline of removing every source of nondeterministic delay from a path that has to be fast at the worst possible instant, and it is the substance of the low-latency systems round.

The Jump Trading angle

This layer most distinguishes the Jump round from a general system-design interview, so treat the "systems" settings as content. A core-developer interviewer expects to hear kernel bypass, core pinning and isolation, NUMA-local allocation, huge pages, pre-faulting, and a heap-free hot path. At a firm whose edge is latency, the fastest data structure still stalls if its core gets interrupted. It also stalls if the memory it touches is a socket away. Bring the tail up yourself, name its causes in physical terms, and say what you would measure and how. "I optimize the p99.9, and here is what lands in it" sounds like someone who does this work.

What is expected, by round and by seniority

Jump does not publish a level ladder, and the loop forks by track, so calibrate to the round in front of you rather than a named rung. The order book turns up in two forms, and how far you carry it is what separates a passing answer from a strong one.

The live coding version

In the coding round the book is a working-implementation exercise, often in about forty-five minutes on a shared editor. Reach a correct book with the operations that matter: apply add, change, and delete, read the best bid and ask cheaply, and cancel a resting order by id without scanning. Choose the three-layer structure on purpose: sorted or tick-indexed levels over FIFO queues with an id-to-node index. Narrate why each operation is cheap as you write it. The bar: correct, the right complexity on the first pass, and a clear account of the per-operation cost when the interviewer asks.

The systems-design version

In the design round the book is the middle of a larger latency system, and the structure is assumed. Now the signal is in the layers around it. Cover the lock-free handoff off the wire and why a mutex is wrong, and the consistent read that never blocks the writer. Then feed gaps and snapshot recovery, and the p99.9 tail with its physical causes and the settings that remove them. You should reach for kernel bypass, core pinning, and pool allocation without being prompted, and defend each from the latency budget. The bar: you drive the design to the tail and the failure modes yourself, and the interviewer redirects your depth rather than supplying it.

Senior and staff

At more senior levels the scope widens past one book on one core. Expect to reason about running many instruments across cores and sockets with NUMA-aware placement. Then the arbitration and clock questions when a strategy reads several correlated books at once. You may be asked how you'd measure and regression-test tail latency, so a change that adds a microsecond is caught before it ships. And where an FPGA takes over the very hottest path. The bar: you own the latency, correctness, and recovery argument end to end, and you can say not just what you would build but how you would prove it stayed fast.