Horizontal vs Vertical Scaling

Difficulty: Intermediate

Question

Explain the difference between horizontal and vertical scaling. When would you choose one over the other?

Answer

Vertical Scaling (Scale Up): Add more resources (CPU, RAM, disk) to an existing server. Simpler to implement but has hardware limits and creates a single point of failure.

Horizontal Scaling (Scale Out): Add more servers to distribute the load. More complex but offers virtually unlimited scaling and better fault tolerance.

Key differences: - Cost: Vertical has diminishing returns (exponential cost). Horizontal scales linearly. - Downtime: Vertical often requires downtime to upgrade. Horizontal can add nodes live. - Complexity: Vertical is simpler (one server). Horizontal needs load balancers, stateless design, distributed sessions. - Fault tolerance: Vertical has a single point of failure. Horizontal survives individual node failures.

Load Balancer algorithms: Round Robin, Least Connections, IP Hash (sticky sessions), Weighted distribution.

Code examples

Nginx Load Balancer Config

# nginx.conf - Load balancer setup
upstream backend {
    # Round robin (default)
    server backend1:3000;
    server backend2:3000;
    server backend3:3000;

    # OR: Least connections
    # least_conn;

    # OR: IP hash (sticky sessions)
    # ip_hash;

    # Weighted
    # server backend1:3000 weight=3;
    # server backend2:3000 weight=1;
}

server {
    listen 80;
    location / {
        proxy_pass http://backend;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /static/ {
        root /var/www;
        expires 30d;
    }
}

Nginx acts as a reverse proxy, distributing requests across backend servers. Static files are served directly without hitting the application server.

Scaling Decision Framework

// Scaling stages for a growing application:

// Stage 1: Single Server (0-1K users)
// - App + DB on one server
// - Vertical scaling (bigger server)

// Stage 2: Separate DB (1K-10K users)
// - Dedicated app server + dedicated DB server
// - Add caching layer (Redis)
// - CDN for static assets

// Stage 3: Horizontal Scaling (10K-100K users)
// - Multiple app servers behind load balancer
// - Read replicas for database
// - Session storage in Redis (not server memory)

// Stage 4: Microservices (100K+ users)
// - Split into independent services
// - Message queues (RabbitMQ, Kafka)
// - Database per service
// - API Gateway

// Key rule: keep servers STATELESS
// Bad:  storing sessions in server memory
// Good: storing sessions in Redis/DB

Each stage introduces complexity only when needed. The key principle is keeping servers stateless so any server can handle any request.

Health Check & Auto-Scaling

// AWS Auto Scaling Group pseudo-config
{
  "AutoScalingGroup": {
    "MinSize": 2,
    "MaxSize": 10,
    "DesiredCapacity": 3,
    "HealthCheckType": "ELB",
    "HealthCheckGracePeriod": 300,
    "ScalingPolicies": [
      {
        "Name": "ScaleUp",
        "Trigger": "CPU > 70% for 5 min",
        "Action": "Add 2 instances"
      },
      {
        "Name": "ScaleDown",
        "Trigger": "CPU < 30% for 10 min",
        "Action": "Remove 1 instance"
      }
    ]
  }
}

// Health check endpoint
app.get('/health', (req, res) => {
  res.json({
    status: 'healthy',
    uptime: process.uptime(),
    timestamp: Date.now()
  });
});

Auto-scaling automatically adjusts the number of servers based on load metrics. Health checks ensure traffic only goes to healthy instances.

Key points

• Stateless servers are essential for horizontal scaling - never store sessions in server memory
• Database is usually the first bottleneck - optimize queries and add caching before scaling app servers
• Start simple with vertical scaling; move to horizontal only when you hit real limits

Concepts covered

Horizontal Scaling, Vertical Scaling, Load Balancer, Reverse Proxy, CDN