InterviewLegend

Design an Embedding and Classification API

Signature problem
Alex Chen25 min readAdded 22 real interview reports

Candidates in July 2026 Scale AI onsites, for software engineer and backend infrastructure seats, report getting this as a single line. One calls it the classic question from past reports. Another, on an infra team under the ML research org, guessed that most AI-related roles get it. A one-line prompt means the scoping is yours. A December 2025 software engineer onsite had a variation where the models are out of your hands. Roughly: users upload one file, or up to 1,000 in a single request, each holding text documents. You call a classification service and an embedding service whose internals you can't change. Then you store the results so users can find them by user, by file or by meaning. The design below covers both, and the pipeline version is mostly its batch half.

Traffic comes in two shapes that want opposite things from the same GPUs. An interactive call wants its answer in tens of milliseconds. A batch job of fifty million documents wants the most texts per GPU-second and doesn't care about any one of them. Then there's the rule that a vector only means something next to vectors from the same model version. Change the model and every stored vector goes stale at once. So does every cached result, and so does every classifier trained on the old vectors. Say both of these in the first few minutes.

The Scale AI angle
Scale's core business is labeling data at volume, so the classification half of this prompt sits close to home. The MLE write-up of the question names version contracts, batching on accelerators, caching, evaluation and safe model rollout. Pick the scope yourself, then go deep on two or three of those.

Scale AIPremium lesson · System design round

The Scale AI design, worked end to end.

  1. Requirements and scopeSystem design · Signature problem
  2. The API and data model
  3. The design, drawn out1 diagram
  4. 5 deep divesWith the tradeoffs
Get the Scale AI questions

$49 a year · all 107 companies · instant access

20,000+ candidate reports · guide updated

Scale AI interview guide

Know what Scale AI asks before you walk in.

20 coding · 4 system design · 18 behavioral questions

Get Premium$49 a year · Instant access
Or start freeNo card.

Get their questions first.

Had your Scale AI interview?How we research

InterviewLegend is independent and not affiliated with or endorsed by any company it covers.