Requirements & scope
Problem Statement & Requirements
Functional Requirements
- 1:1 messaging — send and receive text messages between two users in real-time
- Group messaging — support group chats with up to 500 members (Slack-style) or 1024 (WhatsApp-style)
- Online/offline presence — show whether a user is online, offline, or "last seen"
- Message delivery status — sent, delivered, read (double/blue checkmarks)
- Push notifications — notify offline users of new messages
- Media sharing — images, videos, documents (up to 100 MB)
- Message history — persistent storage, searchable, sync across devices
- End-to-end encryption (E2E) — messages encrypted on sender, decrypted only by recipient
Non-Functional Requirements
- Low latency — message delivery < 100 ms (same region), < 300 ms (cross-region)
- High availability — 99.99% uptime (< 52 min downtime/year)
- Ordering — messages within a conversation must be displayed in order
- Durability — no message loss once server acknowledges
- Scalability — support 500M+ daily active users, 100B+ messages/day
- Multi-device sync — seamless experience across phone, web, desktop
Out of Scope
- Voice/video calling (that's a separate WebRTC design)
- Stories/status updates
- Payment integration
- Bots / slash commands (Slack-specific)
Scale estimations
Scale Estimations
Users & Traffic
| Metric | Value |
|---|---|
| Daily Active Users (DAU) | 500M |
| Average messages sent per user per day | 40 |
| Total messages per day | 500M × 40 = 20B |
| Messages per second (avg) | 20B / 86400 = ~231K msg/s |
| Peak messages per second | ~700K msg/s (3x average) |
| Average concurrent connections | ~150M (30% of DAU) |
Message Sizes
| Metric | Value |
|---|---|
| Average text message | 200 bytes |
| With metadata (sender, timestamp, IDs, status) | 500 bytes |
| Media messages (pointer only, media stored separately) | 500 bytes + media URL |
| Average message with encryption overhead | ~600 bytes |
Storage
| Metric | Value |
|---|---|
| Text storage per day | 20B × 600 bytes = ~12 TB/day |
| Text storage per year | 12 TB × 365 = ~4.4 PB/year |
| Media storage per day (20% of messages have media) | 4B × 500 KB avg = ~2 PB/day |
| Retention | Text: forever; Media: configurable |
Network
| Metric | Value |
|---|---|
| Incoming bandwidth (messages) | 231K × 600 bytes = ~139 MB/s |
| Outgoing bandwidth (fan-out) | ~278 MB/s (avg 2 recipients per msg including 1:1) |
| WebSocket connections (concurrent) | ~150M |
| WebSocket servers needed (50K conn/server) | ~3,000 servers |
Layered architecture
Phase 1 of 4 · Client & edge
Start with the entry points
First, define who uses the system and where requests enter. This is the boundary of the design, not yet a complete request path.
- Chat clients
- Assign stable message IDs and keep a local pending state until acknowledgement.
- WebSocket gateway
- Maintain bidirectional connections and route a user to the gateway holding their session.
This first layer defines the system boundary. The next chapters add application logic and data dependencies. After examining the request paths and tradeoffs, we will add operational safeguards and bring the complete architecture together.
API & contracts
Phase 2 of 4 · Application & compute
Add the application logic
Keep the entry points and add the services that do the work. Give each service a clear responsibility before defining its contracts. Storage and background dependencies come next.
- Message service
- Validate conversation membership and assign ordering within the conversation, not globally.
- Push worker
- Use push notifications for offline users; the message store remains the source of truth.
Communication Protocol — WebSocket Deep Dive
Why WebSocket Over HTTP?
WebSocket Connection Lifecycle
Data model
Phase 3 of 4 · Data & async work
Connect data and background work
The services now need their dependencies: durable state, fast lookups, or work that continues outside a request. Separate these responsibilities rather than treating every component as a database.
- Message queue
- Buffer delivery work so a slow recipient or failed gateway does not block the sender.
- Message store
- Store messages durably and expose cursor-based history for reconnect and multi-device sync.
- Presence store
- Treat presence as soft state that expires when a client stops sending heartbeats.
Data Model
Message Table (Cassandra)
Conversation Table
User Conversation Index (for "list my chats")
Core design decisions
Message Ordering & IDs
Why Not Auto-Increment IDs?
Message Ordering Guarantee
Request flows
Message Flow Diagrams
1:1 Message — Both Users Online
1:1 Message — Recipient Offline
Group Message Flow
Performance & caching
Keep live connection routing separate from message history. A cache can tell a gateway where an online device is connected, but a missing cache entry must not erase an accepted message. Fetch history in bounded pages using a stable conversation cursor, and coalesce noisy presence updates rather than treating every heartbeat like durable chat content.
Advanced design
Presence (Online/Offline Status)
Presence Architecture
Presence at Scale — The Problem
End-to-End Encryption (E2E)
Signal Protocol (used by WhatsApp)
E2E Encryption Flow
Group E2E Encryption
Message Queue & Delivery Guarantees
Kafka as Message Bus
Media Handling
Multi-Device Sync
Chat Search
Edge cases
Handling Edge Cases
Message Delivery Reliability
WebSocket Connection Failure
Tradeoffs
Tradeoffs & Design Decisions Summary
| Decision | Option A | Option B | Chosen | Why |
|---|---|---|---|---|
| Protocol | HTTP Long Polling | WebSocket | WebSocket | Bidirectional, low overhead, real-time push |
| Message DB | MySQL (sharded) | Cassandra | Cassandra | Write-heavy (20B/day), partition by conversation, linear scaling |
| Message Queue | Redis Pub/Sub | Kafka | Kafka | Durability, replay, consumer groups, ordering per partition |
| Presence Store | Database | Redis (TTL) | Redis | In-memory speed, TTL auto-expiration, pub/sub for updates |
| Message IDs | UUID v4 | Snowflake ID | Snowflake | Time-sortable, compact 64-bit, no coordination |
| Delivery | Push only | Push + offline queue | Push + queue | Guaranteed delivery even for offline users |
| Encryption | TLS only (server-side) | E2E (Signal Protocol) | E2E | Privacy — server never sees plaintext |
| Group encryption | N×encrypt per message | Sender Key | Sender Key | O(1) encryption per message instead of O(N) |
| Presence fan-out | Push to all contacts | Hybrid (push active + pull on demand) | Hybrid | Bounded fan-out, scales to 500M users |
| Multi-device | Phone-primary | Server-primary with per-device keys | Server-primary | All devices work independently |
| Media upload | Through API server | Direct to S3 (presigned URL) | Direct S3 | Offloads bandwidth from API servers |
| Search | Server-side Elasticsearch | On-device search | Depends on E2E | E2E → device only; no E2E → Elasticsearch |
| Redirect | 301 vs 302 for links | N/A | N/A | N/A |
Reliability & fault tolerance
Phase 4 of 4 · Operations & resilience
Add operational safeguards
With the request paths and tradeoffs established, add the controls that help detect trouble and keep the system operating. The final diagram retains every layer introduced so far.
- Delivery telemetry
- Track accepted, delivered, and read states separately. Expect duplicate delivery and deduplicate by ID.
A gateway can disappear after storing a message but before replying to its sender. The sender retries with the same message ID, and the service returns the existing result. On reconnect, devices resume from a known position in durable history. Delivery acknowledgements should advance monotonically even when old acknowledgements arrive late.
Production architecture
High-Level Architecture
Detailed Component Architecture
Production Architecture
Further exploration
Take a conversation with three devices belonging to the same person. Read a message on one device, disconnect another, and send from the third. Decide which state belongs to the user, which belongs to a device, and how each catches up without downloading the entire conversation.
Interview playbook
Interview Tips
Start with protocol choice — "For real-time messaging, WebSocket is the clear winner over polling." Briefly explain why, then move on.
Draw the 1:1 message flow first — It's the simplest. Show: sender → WS gateway → message service → persist + Kafka → delivery worker → recipient's WS gateway → recipient.
Address "what if the recipient is offline?" early — This is the follow-up every interviewer asks. Show the offline queue + push notification path.
Presence is a trap topic — Don't spend too long on it. Mention the fan-out problem, say "hybrid pull + push" and move on unless asked to deep-dive.
Group messaging is just fan-out — "A group message is a 1:1 message sent to each member. The interesting part is how to fan out efficiently" → Kafka with conversation-level partitioning.
E2E encryption shows depth — Mention Signal Protocol, X3DH, Double Ratchet. You don't need to derive the math — just show you understand why keys rotate per message (forward secrecy).
Message ordering matters — Explain Snowflake IDs and why Kafka partitioning by conversation_id guarantees FIFO within a conversation.
Multi-device sync is a common follow-up — "Sync cursor per device. On reconnect, replay from cursor."
Don't forget media — "Media goes directly to S3 via presigned URL. Message contains only a pointer. CDN for delivery." This 3-sentence answer covers it.
Scale numbers impress — "500M DAU × 40 messages = 20B messages/day = 231K messages/second. We need ~3,000 WebSocket servers at 50K connections each."