Chat System

Problem Statement & Requirements

Functional Requirements

• 1:1 messaging — send and receive text messages between two users in real-time
• Group messaging — support group chats with up to 500 members (Slack-style) or 1024 (WhatsApp-style)
• Online/offline presence — show whether a user is online, offline, or "last seen"
• Message delivery status — sent, delivered, read (double/blue checkmarks)
• Push notifications — notify offline users of new messages
• Media sharing — images, videos, documents (up to 100 MB)
• Message history — persistent storage, searchable, sync across devices
• End-to-end encryption (E2E) — messages encrypted on sender, decrypted only by recipient

Non-Functional Requirements

• Low latency — message delivery < 100 ms (same region), < 300 ms (cross-region)
• High availability — 99.99% uptime (< 52 min downtime/year)
• Ordering — messages within a conversation must be displayed in order
• Durability — no message loss once server acknowledges
• Scalability — support 500M+ daily active users, 100B+ messages/day
• Multi-device sync — seamless experience across phone, web, desktop

Out of Scope

• Voice/video calling (that's a separate WebRTC design)
• Stories/status updates
• Payment integration
• Bots / slash commands (Slack-specific)

Scale Estimations

Users & Traffic

Message Sizes

Storage

Network

High-Level Architecture

Detailed Component Architecture

Communication Protocol — WebSocket Deep Dive

Why WebSocket Over HTTP?

WebSocket Connection Lifecycle

Data Model

Message Table (Cassandra)

Conversation Table

User Conversation Index (for "list my chats")

Message Ordering & IDs

Why Not Auto-Increment IDs?

Message Ordering Guarantee

Message Flow Diagrams

1:1 Message — Both Users Online

1:1 Message — Recipient Offline

Group Message Flow

Workshop note · added for the website’s common reading format

Start with the dominant access pattern of Chat System. Message queue is one place to inspect capacity and tail latency. Measure before introducing a cache: define the cache key, invalidation policy, stale-data budget, and cold-start behavior.

Batch independent work where the latency budget allows it. Bound queues and concurrency, and verify that an optimization does not move the bottleneck to a dependency.

Presence (Online/Offline Status)

Presence Architecture

Presence at Scale — The Problem

End-to-End Encryption (E2E)

Signal Protocol (used by WhatsApp)

E2E Encryption Flow

Group E2E Encryption

Message Queue & Delivery Guarantees

Kafka as Message Bus

Media Handling

Multi-Device Sync

Chat Search

Handling Edge Cases

Message Delivery Reliability

WebSocket Connection Failure

Tradeoffs & Design Decisions Summary

Workshop note · added for the website’s common reading format

Track accepted, delivered, and read states separately. Expect duplicate delivery and deduplicate by ID.

Set service-level objectives for the user-visible path, then map its dependencies. Define bounded retries with jitter, deadlines, and backpressure. Keep a degraded mode that protects authoritative state, and test recovery instead of treating replication as a backup.

For Chat System, pay special attention to Message queue, Message store, Presence store when deciding failure domains and recovery procedures.

Production Architecture

Workshop note · added for the website’s common reading format

Rebuild Chat System from memory, then change one assumption: ten times more traffic, a new region, or a stricter consistency requirement. Which component must change first—and which does not?

Compare Message service with the same boundary in a related design. Write down one alternative you rejected, what it would simplify, and when you would choose it instead. Follow the source link at the end of this article to explore the original document.

Interview Tips

1. Start with protocol choice — "For real-time messaging, WebSocket is the clear winner over polling." Briefly explain why, then move on.

2. Draw the 1:1 message flow first — It's the simplest. Show: sender → WS gateway → message service → persist + Kafka → delivery worker → recipient's WS gateway → recipient.

3. Address "what if the recipient is offline?" early — This is the follow-up every interviewer asks. Show the offline queue + push notification path.

4. Presence is a trap topic — Don't spend too long on it. Mention the fan-out problem, say "hybrid pull + push" and move on unless asked to deep-dive.

5. Group messaging is just fan-out — "A group message is a 1:1 message sent to each member. The interesting part is how to fan out efficiently" → Kafka with conversation-level partitioning.

6. E2E encryption shows depth — Mention Signal Protocol, X3DH, Double Ratchet. You don't need to derive the math — just show you understand why keys rotate per message (forward secrecy).

7. Message ordering matters — Explain Snowflake IDs and why Kafka partitioning by conversation_id guarantees FIFO within a conversation.

8. Multi-device sync is a common follow-up — "Sync cursor per device. On reconnect, replay from cursor."

9. Don't forget media — "Media goes directly to S3 via presigned URL. Message contains only a pointer. CDN for delivery." This 3-sentence answer covers it.

10. Scale numbers impress — "500M DAU × 40 messages = 20B messages/day = 231K messages/second. We need ~3,000 WebSocket servers at 50K connections each."