InterviewLegend

Design a GPU Cluster Job Scheduler

Frequently asked
Alex Chen50 min readAdded 139 real interview reports

A job scheduler takes work from many users and decides where and when each piece runs. Here the machines are GPU hosts, usually 8 GPUs to a box, and the work is containers: training runs, evaluations, batch inference, test suites. The user says what the job needs and the scheduler finds a place for it, starts it, watches it and cleans up after it.

Versions of this prompt show up in NVIDIA technical screens and onsites, for SWE and Senior SWE roles on infrastructure and cloud teams. In an October 2026 onsite it came up as a task service that runs each task when it's due. A senior first round in November 2026 asked for a job scheduler, with questions on execution guarantees and metrics logging. A mid-2026 screen framed it as a control plane that tracks live host state across a large fleet and dispatches jobs. An older version, from a 2023 onsite, asked for an interface for managing and running jobs on various GPUs. Under every version sit two hard problems. Each job has to run once even when hosts, networks and schedulers fail, and the fleet's state changes thousands of times a second. The single-machine cousin, one queue feeding a worker thread per GPU, is worked in the low-level design lesson. This one spreads the same idea across a fleet.

The NVIDIA angle
Whatever the wording, the interviewer tends to steer toward reliability. Expect questions on how a job runs only once, how two executors avoid taking the same work, and what happens on errors. One reported follow-up asks whether a distributed lock alone guarantees exactly-once email delivery. They want no, with the reason.

NVIDIAPremium lesson · System design round

The NVIDIA design, worked end to end.

  1. The API and data modelSystem design · Frequently asked
  2. The design, drawn out2 diagrams
  3. 5 deep divesWith the tradeoffs
Get the NVIDIA questions

$49 a year · all 107 companies · instant access

139 NVIDIA interview reports · guide updated

NVIDIA interview guide

Know what NVIDIA asks before you walk in.

19 coding · 5 system design · 25 behavioral questions

Get Premium$49 a year · Instant access
Or start freeNo card.

Get their questions first.

Had your NVIDIA interview?How we research

InterviewLegend is independent and not affiliated with or endorsed by any company it covers.