InterviewLegend

Design an AV Camera and Perception Pipeline

Sometimes asked
Alex Chen50 min readAdded 139 real interview reports

A self-driving car sees the world mostly through cameras. Light hits a sensor, the sensor produces raw data, and a chain of stages turns that data into a list of the cars, people and cyclists around the vehicle, with positions and velocities. The planner reads that list many times a second and decides how to steer and brake. This is a different animal from a datacenter design. There's no fleet to scale out and no database. Everything runs on one computer in the car, a system-on-chip (SoC) with CPU cores, a GPU and dedicated accelerators. The hard parts are time and memory. Every frame has a deadline, and an old frame is worse than no frame. Megabytes of pixels move between stages thirty times a second, and the pipeline has to keep working when a camera dies. In practice that means six problems to solve: time sync, zero-copy buffering, back-pressure, the latency budget, multi-camera fusion and degraded modes when a camera or stage fails.

The NVIDIA angle
Prompts like this have been linked to senior roles on NVIDIA's autonomous-vehicle teams, but reports are few. Treat it as practice for an AV loop rather than a known question, and ask the hiring manager what the design slot covers.

NVIDIAPremium lesson · System design round

The NVIDIA design, worked end to end.

  1. The API and data modelSystem design
  2. The design, drawn out1 diagram
  3. 5 deep divesWith the tradeoffs
Get the NVIDIA questions

$49 a year · all 107 companies · instant access

139 NVIDIA interview reports · guide updated

NVIDIA interview guide

Know what NVIDIA asks before you walk in.

19 coding · 5 system design · 25 behavioral questions

Get Premium$49 a year · Instant access
Or start freeNo card.

Get their questions first.

Had your NVIDIA interview?How we research

InterviewLegend is independent and not affiliated with or endorsed by any company it covers.