Design a Chat System

"Design a chat system," usually framed as Teams, is the design prompt Microsoft reaches for most after the rate limiter. The task sounds small. One person types a message, another sees it appear, ideally within a second. But that second person might be online with the app open, or away with the laptop shut and the phone in a bag. The system serves both, and the sender never knows or cares which. That split, online versus offline delivery, is the real problem hiding inside one sentence.

Start with rough numbers, because they pick the architecture. Microsoft hides exact figures, so hedge and round. Teams is used across a large share of the world's enterprises. Call it a few hundred million daily users sending on the order of tens of billions of messages a day. Spread evenly that's a few hundred thousand messages a second, and real traffic peaks several times higher when working hours overlap. At any moment a large fraction of those users have the app open, each holding a live connection to a server. So the system keeps on the order of a hundred million persistent connections at once. That connection count is what shapes everything else. A feed is read-heavy and you can precompute it. A chat system has to push a message to a specific recipient the instant it arrives, which means knowing exactly where that recipient is connected right now. Lead with the connection count. Everything else in the design bends around it.

The Microsoft angle

Teams is Microsoft's own product, and that changes the room. The interviewer may have built part of the real thing, or sat next to the team that did, so vague hand-waving stands out fast. They know what gets hard at enterprise scale and they push there. A message that can never be lost in a compliance-bound org. Two people who must see the conversation in the same order. A laptop that was closed for eight hours catching up cleanly when it wakes.