Design an AV Camera and Perception Pipeline
A self-driving car sees the world mostly through cameras. Light hits a sensor, the sensor produces raw data, and a chain of stages turns that data into a list of the cars, people and cyclists around the vehicle, with positions and velocities. The planner reads that list many times a second and decides how to steer and brake. This is a different animal from a datacenter design. There's no fleet to scale out and no database. Everything runs on one computer in the car, a system-on-chip (SoC) with CPU cores, a GPU and dedicated accelerators. The hard parts are time and memory. Every frame has a deadline, and an old frame is worse than no frame. Megabytes of pixels move between stages thirty times a second, and the pipeline has to keep working when a camera dies. In practice that means six problems to solve: time sync, zero-copy buffering, back-pressure, the latency budget, multi-camera fusion and degraded modes when a camera or stage fails.
NVIDIAPremium lesson · System design round
The NVIDIA design, worked end to end.
- The API and data modelSystem design
- The design, drawn out1 diagram
- 5 deep divesWith the tradeoffs
$49 a year · all 107 companies · instant access
139 NVIDIA interview reports · guide updated
NVIDIA interview guideKnow what NVIDIA asks before you walk in.
19 coding · 5 system design · 25 behavioral questions
Get their questions first.
InterviewLegend is independent and not affiliated with or endorsed by any company it covers.