Scalability Patterns: Vertical vs Horizontal

Scalability is a system’s ability to handle increased load by adding resources. There are two fundamental approaches: vertical scaling (scale up) and horizontal scaling (scale out).

Vertical Scaling (Scale Up)

Adding more resources (CPU, RAM, storage) to a single machine.

Pros

• Simple: No application changes needed
• Strong consistency: Single database, no distributed transactions
• Lower operational complexity: One server to manage

Cons

• Hardware limits: Maximum CPU, RAM, storage per machine
• Single point of failure: If the machine dies, everything dies
• Cost: High-end hardware is disproportionately expensive
• Downtime: Often requires restart for upgrades

When to Use

• Early-stage startups with predictable growth
• Legacy applications hard to distribute
• Workloads that don’t parallelize well (some ML training)

# Example: AWS RDS vertical scaling
# r5.large → r5.xlarge → r5.2xlarge → r5.4xlarge
# vCPUs: 2 → 4 → 8 → 16
# Memory: 16GB → 32GB → 64GB → 128GB
# Cost/month: ~$180 → $360 → $720 → $1440

Horizontal Scaling (Scale Out)

Adding more machines to distribute the load.

Pros

• No practical limits: Can scale to thousands of machines
• Fault tolerance: Failure of one node doesn’t bring down system
• Cost-effective: Commodity hardware, linear cost scaling
• Elasticity: Add/remove nodes based on demand

Cons

• Complexity: Distributed systems challenges (CAP, consensus, consistency)
• Application changes: Code must be designed for distribution
• Operational overhead: More servers to monitor, deploy, debug
• Network latency: Inter-node communication adds latency

When to Use

• High-growth applications
• Systems requiring high availability
• Microservices architectures
• Big data processing

Scaling Patterns

1. Load Balancing

Distribute incoming requests across multiple servers.

# NGINX load balancer config
upstream backend {
    least_conn;  # Least connections algorithm
    server 10.0.0.1:8080 weight=3;
    server 10.0.0.2:8080 weight=2;
    server 10.0.0.3:8080 weight=1;
    
    # Health checks
    server 10.0.0.4:8080 backup;
}

server {
    location / {
        proxy_pass http://backend;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

2. Stateless Services

Services that don’t store session state locally.

Client → Load Balancer → [Service A, Service B, Service C]
                              ↓
                        Shared Cache (Redis)
                              ↓
                        Database

3. Database Sharding

Partition data across multiple database instances.

# Simple hash-based sharding
def get_shard(user_id: int, num_shards: int) -> int:
    return hash(user_id) % num_shards

# Consistent hashing (better for dynamic scaling)
import hashlib

class ConsistentHashRing:
    def __init__(self, nodes: list, replicas: int = 150):
        self.ring = {}
        self.sorted_keys = []
        
        for node in nodes:
            for i in range(replicas):
                key = self._hash(f"{node}:{i}")
                self.ring[key] = node
                self.sorted_keys.append(key)
        self.sorted_keys.sort()
    
    def get_node(self, key: str) -> str:
        hash_key = self._hash(key)
        for node_hash in self.sorted_keys:
            if node_hash >= hash_key:
                return self.ring[node_hash]
        return self.ring[self.sorted_keys[0]]

4. Read Replicas

Scale reads by replicating data.

-- Primary handles writes
INSERT INTO orders (...) VALUES (...);

-- Replicas handle reads
SELECT * FROM orders WHERE user_id = ?;  -- Routed to replica

5. Caching Layers

Reduce database load with strategic caching.

Request → CDN → API Gateway → Redis Cache → Database
              ↓              ↓
           Static          Computed
           Assets          Responses

The Scalability Checklist

Layer	Vertical Approach	Horizontal Approach
Compute	Bigger VMs	More containers/pods
Database	Bigger instance	Sharding + read replicas
Cache	More memory	Cluster mode (Redis Cluster)
Storage	Bigger disks	Distributed FS (S3, HDFS)
Network	Higher bandwidth	Load balancers, CDN

Cost Comparison

Approach	Monthly Cost (Example)	Max Throughput
Vertical (r5.4xlarge)	~$1,440	~50K req/s
Horizontal (10 × r5.large)	~$1,800	~200K req/s
Horizontal (auto-scaled)	Variable	Unlimited

Anti-Patterns to Avoid

1. Distributed monolith: Services that must be deployed together
2. Chatty services: Excessive inter-service communication
3. Shared database: Multiple services sharing one database
4. Synchronous chains: A→B→C→D where each call blocks

Further Reading

• Scalability Patterns - Martin Fowler
• The Art of Scalability - Abbott & Fisher
• Designing Data-Intensive Applications - Martin Kleppmann (Ch. 5-6)