Design a Distributed Rate Limiter

Picture the limiter guarding Adobe's own public APIs, the ones fronting Creative Cloud, Document Cloud, and the Experience Cloud analytics suite, where a single heavy caller must never starve the rest. The ask reads small. Each request arrives tagged with a key, a user id, an API token, or a tenant id, and for each one you return allow or reject. Every key stays under the budget an operator set for it, perhaps a thousand requests a second for a given tenant. The word that carries the weight is distributed. One box with one counter is a trivial read. Spread that traffic over many gateway nodes, each tallying the same tenant at the same moment, and the count has to move into shared storage. Keeping that shared count correct and fast at the same time is the real problem.

Underneath the plain interface, this is really a coordination problem, and it lives in an awkward spot. Two forces fight. The check rides in front of every request, so it has to cost almost nothing, and yet the count it consults is global, something no lone node can observe by itself. Price the check too high and you have quietly taxed every call the platform serves. Set it too loose and the ceiling stops meaning anything. What you owe the caller is a good-enough global verdict in a fraction of a millisecond, delivered while the very traffic being policed streams through many machines at the same time.

Fix a few numbers in mind before anything gets drawn. Assume a platform absorbing roughly a million requests a second in total, fanned out over a gateway fleet. A single key's budget might be a hundred requests a second for a small tenant, or tens of thousands for a big one. The decision runs on a strict allowance: at most roughly a millisecond at p99, since beyond that the limiter turns into the slow part of the very call it guards. Three facts drive the rest. The check rides in front of every request, so it has to be cheap.