News Feed

Problem Statement & Requirements

Functional Requirements

• Publish posts -- users create text posts, links, photos, or videos
• News feed -- a personalized, ranked stream of posts from friends/followed accounts
• Retweet / share -- amplify someone else's post to your followers
• Like, reply, quote-tweet -- engagement actions
• Follow / friend -- asymmetric (Twitter: follow) or symmetric (Facebook: friend)
• Trending topics -- surface popular topics in real-time
• Notifications -- new followers, likes, mentions, replies
• Real-time updates -- new posts appear in feed without manual refresh (for active users)

Non-Functional Requirements

• Low latency -- feed renders in < 500 ms
• High availability -- 99.99% uptime
• Eventually consistent -- slight delay in feed propagation is acceptable
• Massively read-heavy -- read:write ratio ~1000:1
• Scalability -- 500M+ DAU, 1B+ feed reads/day
• Freshness -- breaking news and viral content surface within seconds

Key Differences from Instagram Design

Scale Estimations

Users & Content

Traffic

Storage

Fanout

High-Level Architecture

Detailed Architecture

Workshop note · added for the website’s common reading format

For News Feed, define contracts around the boundary components: Client, API gateway, Post service. Specify authentication, request identity, versioning, pagination or streaming semantics, timeouts, and retry behavior. Name which operations are idempotent and how callers discover an uncertain outcome.

The source discusses these contracts within its subsystem walkthroughs rather than as a standalone endpoint catalog. The examples in this article are design exercises, not published service APIs.

Data Model

Posts Table

Conversation Threading

Ranking System (Deep Dive)

Feature Categories

Ranking Model Architecture

Two-Phase Ranking

Fanout Strategies Compared

Post Publishing Flow

Retweet / Share Flow

Feed Read Flow -- Timeline Assembly

This is the core of the design -- how a user's feed is constructed on read.

Timeline Assembly Flow Diagram

Caching Architecture (Multi-Layer)

Real-Time Feed Updates

Trending Topics

Handling Viral Posts

Workshop note · added for the website’s common reading format

Walk through three failure moments in News Feed: a request times out before its result is known; a dependency becomes slow rather than unavailable; and a process restarts after committing state but before acknowledging it.

For each case, name the authoritative record, define a safe retry, cap resource usage, and describe what the caller sees. Event pipeline should help detect and contain the problem: Track fanout lag and cap expensive viral workloads before queues cascade.

Tradeoffs & Design Decisions Summary

Workshop note · added for the website’s common reading format

Track fanout lag and cap expensive viral workloads before queues cascade.

Set service-level objectives for the user-visible path, then map its dependencies. Define bounded retries with jitter, deadlines, and backpressure. Keep a degraded mode that protects authoritative state, and test recovery instead of treating replication as a backup.

For News Feed, pay special attention to Post store, Feed cache, Graph + features when deciding failure domains and recovery procedures.

Production Architecture

Twitter-Specific: "For You" vs "Following"

Interview Tips

1. Frame the two core problems -- "A news feed has two hard problems: (1) how to build the feed efficiently (fanout strategy) and (2) how to rank it (ML pipeline). Let me address both."

2. Fanout is the opening move -- Start with fanout-on-write vs fanout-on-read. Show the celebrity problem. Land on hybrid. This is the foundation everything else builds on.

3. Timeline assembly is the full picture -- "Building the feed is a 5-stage pipeline: gather candidates, hydrate, rank, filter, return." Walk through each stage. This shows you think about the complete system, not just storage.

4. Ranking shows ML maturity -- "We use a multi-task model that predicts P(like), P(reply), P(retweet), and P(hide). The final score is a weighted combination optimizing for healthy engagement." Name specific features.

5. Caching layers show systems depth -- "There are 6 layers of cache: client SQLite, CDN for media, Redis for feed, Memcached for posts, Memcached for users, TAO for social graph." Name each one and its purpose.

6. Trending is a stream processing problem -- "We use Flink to process every post in real-time, extract entities, count in sliding windows, and compute spike ratios vs baseline." Shows you know stream processing.

7. Retweet is a unique design concern -- "A retweet is a pointer, not a copy. It triggers the same fanout as a new post but for the retweeter's followers. Celebrity retweets = same celebrity fanout problem."

8. Real-time updates are overvalued -- "Only ~5% of users are actively watching their feed. For the other 95%, the feed is assembled on next app open. Don't over-engineer WebSocket for everyone."

9. "For You" vs "Following" is a product decision -- "Algorithmic feeds increase engagement 40-60% but users feel less control. Offering both tabs is the modern compromise."

10. Viral content handling shows production thinking -- "A single viral post can get 100K reads/sec on one cache key. We detect hot keys, replicate across cache shards, and use approximate counters to avoid write storms."