InterviewLegend

Design a Distributed Rate Limiter

Frequently asked
Alex Chen45 min readAdded 19 real interview reports

A caller with allowance left goes through. One without gets 429 Too Many Requests and a hint about when to try again. The limiter keeps one caller from swamping the servers behind the API, and it's how a premium plan buys more than a free one. Two versions of this prompt have been reported at xAI. One builds up in code: a limiter for a single URL, then several URLs with rules that must all pass, then a limit on the total bytes of request bodies over a time window. The other is the distributed design: tens of millions of users, quotas in tokens, a million checks a second and almost no time to spend on each. This lesson covers both. The second is the first one spread across a fleet.

The xAI angle
The Grok API counts usage in tokens, and that changes the problem. Limits apply in requests and in tokens, and nobody knows a reply's length until the model stops writing. Expect to write the core limiter and then extend it; the two-level rate limiter is the single-process starting point.

xAIPremium lesson · System design round

The xAI design, worked end to end.

  1. The API and data modelSystem design · Frequently asked
  2. The design, drawn out2 diagrams
  3. 5 deep divesWith the tradeoffs
Get the xAI questions

$49 a year · all 107 companies · instant access

20,000+ candidate reports · guide updated

xAI interview guide

Know what xAI asks before you walk in.

24 coding · 4 system design · 15 behavioral questions

Get Premium$49 a year · Instant access
Or start freeNo card.

Get their questions first.

Had your xAI interview?How we research

InterviewLegend is independent and not affiliated with or endorsed by any company it covers.