Netflix Architecture: Microservices at Scale

Netflix is the canonical example of microservices architecture at massive scale. This case study explores their architecture, key decisions, and lessons learned.

Scale Numbers (2024)

230M+ subscribers across 190 countries
15,000+ titles in catalog
1M+ requests/second at peak
15% of global internet traffic (peak hours)
4,000+ microservices
1,000+ deployments/day

High-Level Architecture

┌─────────────────────────────────────────────────────────────┐
│                        CDN (Open Connect)                    │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐        │
│  │  ISP 1  │  │  ISP 2  │  │  ISP 3  │  │  ...    │        │
│  └────┬────┘  └────┬────┘  └────┬────┘  └────┬────┘        │
└───────┼────────────┼────────────┼────────────┼──────────────┘
        │            │            │            │
        ▼            ▼            ▼            ▼
┌─────────────────────────────────────────────────────────────┐
│                      AWS Cloud                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐       │
│  │   API GW     │  │   Zuul GW    │  │   Edge       │       │
│  │   (GraphQL)  │  │   (Routing)  │  │   Services   │       │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘       │
└─────────┼─────────────────┼─────────────────┼────────────────┘
          │                 │                 │
    ┌─────▼─────┐    ┌──────▼──────┐    ┌─────▼─────┐
    │  User     │    │  Content    │    │  Playback │
    │  Service  │    │  Catalog    │    │  Service  │
    └─────┬─────┘    └──────┬──────┘    └─────┬─────┘
          │                 │                 │
    ┌─────▼─────────────────▼─────────────────▼─────┐
    │           Data Layer (Cassandra, Redis, S3)   │
    └────────────────────────────────────────────────┘

Key Architectural Decisions

1. Microservices with Bounded Contexts

Each service owns a single business capability:

# Service examples
services:
  user-service:
    owns: [profiles, preferences, accounts, billing]
    data: Cassandra (user profiles), PostgreSQL (billing)
  
  catalog-service:
    owns: [titles, metadata, search, recommendations]
    data: Elasticsearch, Cassandra
  
  playback-service:
    owns: [streaming, licensing, DRM, quality selection]
    data: Redis (session), S3 (manifests)
  
  discovery-service:
    owns: [search, browse, personalization]
    data: Elasticsearch, Cassandra

2. API Gateway Pattern

Zuul (Netflix’s gateway) handles:

Request routing
Authentication/authorization
Rate limiting
Circuit breaking
Request/response transformation
A/B testing routing

// Zuul filter example
public class RateLimitFilter extends ZuulFilter {
    @Override
    public String filterType() { return "pre"; }
    
    @Override
    public int filterOrder() { return 1; }
    
    @Override
    public boolean shouldFilter() { return true; }
    
    @Override
    public Object run() {
        RequestContext ctx = RequestContext.getCurrentContext();
        String userId = ctx.getRequest().getHeader("X-User-Id");
        
        if (!rateLimiter.tryAcquire(userId)) {
            ctx.setSendZuulResponse(false);
            ctx.setResponseStatusCode(429);
            ctx.setResponseBody("Rate limit exceeded");
        }
        return null;
    }
}

3. Circuit Breaker Pattern (Hystrix)

Prevent cascade failures:

@HystrixCommand(
    fallbackMethod = "getFallbackCatalog",
    commandProperties = {
        @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1000"),
        @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "20"),
        @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50"),
        @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "5000")
    }
)
public Catalog getCatalog(String userId) {
    return catalogClient.getRecommendations(userId);
}

public Catalog getFallbackCatalog(String userId) {
    // Return cached/popular content
    return catalogCache.getPopular();
}

4. Chaos Engineering (Simian Army)

Proactively inject failures:

Tool	Purpose
Chaos Monkey	Randomly terminates instances
Latency Monkey	Adds artificial latency
Conformity Monkey	Finds non-compliant instances
Doctor Monkey	Removes unhealthy instances
Janitor Monkey	Cleans up unused resources
Chaos Gorilla	Simulates AZ outage
Chaos Kong	Simulates region outage

5. Data Architecture

Cassandra for:

User viewing history (time-series)
Playback positions
Device registry
A/B test assignments

Redis for:

Session management
Rate limiting counters
Real-time recommendations cache
Feature flags

S3 for:

Video manifests (HLS/DASH)
Thumbnails, artwork
Backup/restore

Elasticsearch for:

Search indexing
Log aggregation
Metrics

Resilience Patterns

1. Bulkhead Pattern

Isolate failures to prevent cascade:

# Thread pool isolation per dependency
hystrix:
  threadpool:
    catalogService:
      coreSize: 20
      maxQueueSize: 100
    userService:
      coreSize: 10
      maxQueueSize: 50
    playbackService:
      coreSize: 50
      maxQueueSize: 200

2. Timeout and Retry Budgets

// Retry with exponential backoff + jitter
@Retryable(
    maxAttempts = 3,
    backoff = @Backoff(delay = 100, multiplier = 2, random = true),
    value = {TimeoutException.class, ConnectException.class}
)
public Catalog fetchCatalog(String userId) {
    return catalogClient.getCatalog(userId);
}

3. Graceful Degradation

public PlaybackManifest getManifest(String titleId, String deviceId) {
    try {
        return manifestService.getPersonalizedManifest(titleId, deviceId);
    } catch (Exception e) {
        log.warn("Personalized manifest failed, using generic", e);
        return manifestService.getGenericManifest(titleId);
    }
}

4. Request Collapsing

// Collapse duplicate concurrent requests
@RequestCollapser(
    scope = RequestCollapser.Scope.REQUEST,
    collapserKey = "getUserProfile"
)
public UserProfile getUserProfile(String userId) {
    return userClient.getProfile(userId);
}

Deployment Pipeline

┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐
│  Code   │──▶│  Build  │──▶│  Test   │──▶│  Canary │──▶│  Prod   │
│  Commit │   │  & Unit │   │  Int.   │   │  Deploy │   │  Rollout│
└─────────┘   └─────────┘   └─────────┘   └─────────┘   └─────────┘
                 │             │             │             │
          ┌──────▼──────┐ ┌───▼────┐ ┌──────▼──────┐ ┌────▼────┐
          │  Spinnaker  │ │ Bakery │ │  Kayenta    │ │  Atlas  │
          │  Pipeline   │ │  (AMI) │ │  (Canary    │ │  (Metrics)│
          └─────────────┘ └────────┘ │   Analysis) │ └─────────┘
                                    └─────────────┘

Spinnaker Pipeline Stages:

Bake: Create immutable AMI
Deploy to Dev: Automated
Integration Tests: Automated
Canary Deploy: 1% → 10% → 100%
Kayenta Analysis: Automated metric comparison
Production Rollout: Gradual with pause points

Key Lessons Learned

Invest in observability early: Metrics, logs, traces, dashboards
Automate everything: Deployments, rollbacks, capacity, failover
Design for failure: Assume everything will fail
Culture of ownership: You build it, you run it
Incremental migration: Strangler fig pattern from monolith
Standardize but allow innovation: Paved road + experimentation
Chaos engineering is not optional: Test failure scenarios regularly

Modern Evolution (2020+)

GraphQL Federation: Unified API layer
gRPC: Service-to-service communication
Kubernetes: Container orchestration (migrating from Titus)
Serverless: AWS Lambda for sporadic workloads
ML Platform: Metaflow for ML workflows