Docker at Scale

Task 1 - Describe several replicas

Inside your quotes folder create a folder called scale with two files. The compose file runs the API and an nginx load balancer in front of it. Notice what the api service does not have. It has no ports, because several replicas cannot all publish the same port on your machine, and no fixed container name, because several containers cannot share one.

# Several copies of the API behind one load balancer.
# Start it with: docker compose -f scale/compose.yaml up -d --build --scale api=3
services:
  lb:
    image: nginx:1.29-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    ports:
      - "8090:80"
    depends_on:
      - api

  api:
    build: ../quotes-api
    environment:
      APP_VERSION: scaled
    restart: always

The load balancer configuration asks Docker's DNS server for api every few seconds. When a service has several replicas, Docker's DNS answers with the address of every one of them, and nginx shares requests between them.

server {
    listen 80;

    # Ask Docker's DNS for "api" again every few seconds. It answers with the
    # address of every replica, and nginx shares requests between them.
    resolver 127.0.0.11 valid=5s;
    set $api http://api:3000;

    location / {
        proxy_pass $api;
    }
}

restart: always tells Docker to restart a replica whenever its process exits, for any reason.

Task 2 - Start three replicas

Run these from the quotes folder. -f chooses the compose file. --scale sets how many containers to run for a service.

docker compose -f scale/compose.yaml up -d --build --scale api=3
docker compose -f scale/compose.yaml ps

Success condition: four containers, named scale-lb-1 and scale-api-1 to scale-api-3. Now send six requests to the endpoint that reports which container answered.

for i in 1 2 3 4 5 6; do curl -s http://localhost:8090/api/whoami; echo; done

Success condition: the answers come from three different host names. Your machine sees one address, and behind it requests are being shared out.

Task 3 - Crash a replica

Simulate a crash by stopping the main process inside the second replica, and immediately send a stream of requests. %{http_code} makes curl print only the response status.

docker compose -f scale/compose.yaml exec --index 2 api kill 1
for i in $(seq 1 12); do curl -s -o /dev/null -w "%{http_code} " http://localhost:8090/api/health; sleep 0.3; done
docker compose -f scale/compose.yaml ps

Success condition: every request returns 200, and scale-api-2 has a much shorter Up time than the others because Docker restarted it. A user would never have known. Now change the number of replicas while the service is running.

docker compose -f scale/compose.yaml up -d --scale api=5
docker compose -f scale/compose.yaml ps
docker compose -f scale/compose.yaml up -d --scale api=1

Task 4 - Find the limits of Compose

What you just did shows the right ideas, but Compose has hard limits as a production platform. Consider each question before reading on.

• All five replicas ran on your laptop. What happens to the service if the laptop is switched off?
• You set the number of replicas by hand. Who changes it at three in the morning when traffic triples?
• To release a new version you would run up again, which recreates every replica at once. Is there a gap?
• Try scaling the database the same way. Five Postgres containers would each have their own separate data. Which one is right?

Compose manages containers on one host. Surviving the loss of a machine, scaling automatically, and updating without downtime need a system that manages a cluster of machines as one. That is what orchestrators do.

docker compose -f scale/compose.yaml down

Task 5 - How an orchestrator thinks

This simulator behaves like an orchestrator. Scale up and down, crash replicas and roll out a new version. Watch the desired and actual lines and read each explanation.

Task 6 - Read a Kubernetes Deployment

You will not run Kubernetes in this series, but you should be able to read its configuration. Kubernetes is configured with YAML files describing desired state. Read this one, which asks for the quotes API to run in a cluster.

# For reading only. This series does not run Kubernetes.
# A Deployment keeps a set number of identical pods running.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: quotes-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: quotes-api
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  template:
    metadata:
      labels:
        app: quotes-api
    spec:
      containers:
        - name: api
          image: docker.io/jsmith/quotes-api:1.0
          ports:
            - containerPort: 3000
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: quotes-db
                  key: url
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              memory: 256Mi
          readinessProbe:
            httpGet:
              path: /api/health
              port: 3000
          livenessProbe:
            httpGet:
              path: /api/health
              port: 3000
---
# A Service gives the pods one stable name and shares traffic between them.
apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: quotes-api
  ports:
    - port: 3000
      targetPort: 3000

Task 7 - Choose a platform

Kubernetes is powerful and also complex to run well. Most organisations choose from a spectrum.

All three run the same images. That portability is what makes the effort you have put into small, stateless, configurable images pay off. The Continuous Deployment to Azure Container Apps lab later in the module deploys to the middle option.

Task 8 - Evaluate