Design the Real-Time Gateway

A Discord server (called a guild in the codebase) is a community with channels, members, and a constant stream of things happening in it: messages, typing indicators, presence changes, members joining. The gateway is the piece that carries all of that to people in real time. Every open Discord app holds a long-lived connection to it, and when anything happens in a server you are in, the gateway pushes it down that connection within a fraction of a second. Discord interviewers point the system design round at their own platform rather than a generic web app. This fan-out fabric is the system their engineering blog is mostly about, so treat it as one of the flagship real-time designs.

The whole job sits between an event occurring in a server and every connected member of that server seeing it, fast, over a connection that has to stay open for hours. This is not a request/response API where a client asks and the server answers. The client is not polling. The connection is always open, the server decides when to push, and the same publish has to work whether it lands in a tiny direct message or a server with half a million members. Three realities drive everything else. Connections are persistent and stateful. Delivery is a fan-out from one event to many recipients. And the two ends of a large server look nothing alike, because most of its members are idle.

The Discord angle

The unit of scale at Discord is the open connection, and saying so in the first two minutes reframes the design. A typical backend handles a request and forgets the client the moment it responds. The gateway does the opposite. It keeps tens of millions of WebSockets open at once, one lightweight process per connection on the Elixir BEAM. Its only job is to push events down those sockets the instant something happens. Discord has reported growing this from 5 million concurrent connections in 2017 to more than 11 million in 2019.