System Design at Databricks

Ask a Databricks candidate what design they drew and the answer keeps coming back the same: a durable key-value store on a single machine. Put, get, delete on local disk. Around fifty thousand durable writes a second, and not one of them lost when the box reboots mid-write. That prompt tells you what the round is actually about. It lives between plain object modeling and a full distributed system, and it turns on a write-ahead log, snapshots, and crash recovery instead of load balancers and service meshes. Databricks grew out of Apache Spark, Delta Lake, and MLflow. So the round reaches for the problems its own engineers spend their days on: how bytes survive a power loss, and how work spreads across a cluster.

Then comes the format surprise: no whiteboard. The round usually runs in a plain Google Doc, no canvas, no boxes-and-arrows tool. You type the architecture in words and rough text blocks while the interviewer reads along. The picture in your head never reaches the page. So the talk-track does all the work, and whether a choice lands rests on how you explain it. A tidy drawing was never going to carry you here.

What is the Databricks system design interview?

The Databricks system design interview is one onsite round, run in a plain Google Doc rather than on a whiteboard. It skews hard to Databricks' own domain: single-node durable storage on one side, distributed data and compute on the other. With no diagram tool, your talk-track and trade-off reasoning carry the score.

The Databricks angle

With no diagram tool, the trade-off reasoning is the artifact. A whiteboard lets a vague answer hide behind a tidy diagram. A Google Doc strips that away. What survives is the sentence where you say why the write-ahead log goes before the in-memory map, or why the scheduler retries an idempotent task instead of one with side effects. Practice narrating a design out loud against a blank document. If you can only think in pictures, this is the round where it shows, and the engineers grading you have shipped the real versions of these systems.

The two shapes the round reaches for

Databricks design prompts cluster into two families, and both come straight from the platform it builds. The first is single-node durability and storage: make data on one machine survive a crash, with a write-ahead log, snapshots, and recovery on restart. This is the signature, and it shows up more than any other shape. The second is distributed data and compute: coordinate work across many machines, ingest a stream in real time, and schedule jobs over a cluster the way Spark schedules tasks. The four worked designs in this section take one distinct shape each so you drill the full spread instead of four versions of the same problem.

The Databricks angle

The storage prompt blurs the line between low-level design and system design. The durable key-value store often runs in a code editor with real pseudocode rather than purely on the doc. Interviewers push on the parts that decide durability: where the write-ahead log sits relative to the acknowledgment, how a snapshot truncates the log, and what replay does after a crash. Treat it as the bridge problem it is. You need the storage-engine instincts of a low-level build and the throughput reasoning of a distributed design in the same answer.

The four designs in this guide

Start with the durable key-value store. It is the most-asked shape and the cleanest place to build the write-ahead log and recovery instincts the round rewards. The three distributed designs each add a new axis on top.

The framework Databricks rewards

The arc that wins the round is the standard one, and on a blank document it matters more because nothing else organizes the conversation for you. Open by nailing down requirements: what is in scope, what the throughput and durability targets are, and what you are deliberately leaving out. State the read-versus-write skew, since most of these systems lean hard one way and the design lives or dies on the heavy side. Put numbers on the scale before you design anything.

From there, define the entities and the API. Name the objects (a key, a log record, a snapshot, a task) and write the interface as concrete endpoints, then keep referring back to those endpoints as you build. Sketch the high-level design next, walking one requirement at a time and naming the bottleneck you are punting at each step. Then go deep where the interviewer points. The deep dives are the round. A weak high-level pass with one genuinely deep component beats a tour of six shallow ones.

The Databricks angle

Rehearse the round out loud, as a walkthrough, not a sketch. The doc carries no diagram, so a clear ordered narration and honest trade-offs are all the interviewer has to grade. The System Design course drills the underlying mechanics: the storage engines, the streaming patterns, the scheduling primitives. This section stays on the Databricks layer, the four shapes its round asks, worked end to end. Each one is written the way you would have to say it across a shared document, with someone who has built the real thing reading along.

Common questions