Design an Embedding and Classification API
Candidates in July 2026 Scale AI onsites, for software engineer and backend infrastructure seats, report getting this as a single line. One calls it the classic question from past reports. Another, on an infra team under the ML research org, guessed that most AI-related roles get it. A one-line prompt means the scoping is yours. A December 2025 software engineer onsite had a variation where the models are out of your hands. Roughly: users upload one file, or up to 1,000 in a single request, each holding text documents. You call a classification service and an embedding service whose internals you can't change. Then you store the results so users can find them by user, by file or by meaning. The design below covers both, and the pipeline version is mostly its batch half.
Traffic comes in two shapes that want opposite things from the same GPUs. An interactive call wants its answer in tens of milliseconds. A batch job of fifty million documents wants the most texts per GPU-second and doesn't care about any one of them. Then there's the rule that a vector only means something next to vectors from the same model version. Change the model and every stored vector goes stale at once. So does every cached result, and so does every classifier trained on the old vectors. Say both of these in the first few minutes.
Scale AIPremium lesson · System design round
The Scale AI design, worked end to end.
- Requirements and scopeSystem design · Signature problem
- The API and data model
- The design, drawn out1 diagram
- 5 deep divesWith the tradeoffs
$49 a year · all 107 companies · instant access
20,000+ candidate reports · guide updated
Scale AI interview guideKnow what Scale AI asks before you walk in.
20 coding · 4 system design · 18 behavioral questions
Get their questions first.
InterviewLegend is independent and not affiliated with or endorsed by any company it covers.