How Kubernetes Works: The Container Orchestrator

Scenario context

apiVersion: apps/v1
kind: Deployment
metadata:
  name: storefront
spec:
  replicas: 3
  selector:
    matchLabels:
      app: storefront
  template:
    metadata:
      labels:
        app: storefront
    spec:
      containers:
        - nam
e: storefront
          image: acme/storefront:v2.1
          ports:
            - containerPort: 8080

Observable outcome: Observes the current state — how many Pods are actually running, on which Nodes.

Stage 3 — Deep Dive: The Scheduler, Reconciler Pattern, and Custom Resources

• How the Scheduler Places Pods When a new Pod needs to be placed, kube-scheduler runs a two-phase algorithm: 1.
• Filter — eliminate Nodes that cannot fit the Pod (insufficient CPU/RAM, taint mismatches, wrong node labels).
• Score — rank remaining Nodes (prefer more free resources, spread replicas across failure zones).
• The highest-scoring Node wins.

sequenceDiagram
    participant U as kubectl apply
    participant A as API Server
    participant E as etcd
    participant S as Scheduler
    participant K as Kubelet (Node)

    U->>A: POST /apis/apps/v1/deployments
    A->>E: store Deployment spec
    A-->>U: 201 Created
    S->>A: watch for unscheduled Pods
    A-->>S: new Pod (node not assigned)
    S->>S: Filter + Score nodes
    S->>A: bind Pod to Node X
    A->>E: store binding
    K->>A: watch assigned Pods
    A-->>K: Pod spec for Node X
    K->>K: pull image + start container

stateDiagram-v2
    [*] --> Pending : Pod created, awaiting scheduling
    Pending --> Running : Node assigned, container started
    Running --> Succeeded : all containers exited 0
    Running --> Failed : container exited non-zero
    Running --> Terminating : delete signal sent
    Terminating --> [*] : graceful shutdown complete
    Pending --> Failed : image pull error / no node fits

Observable outcome: The highest-scoring Node wins.

Stage 4 — The Request Journey: From Browser to Pod

• Here is the complete path a request takes from the internet to a Pod inside your cluster: External traffic enters through the cloud load balancer, hits the Ingress controller (host/path routing), then reaches the Servic…
• The Service distributes requests across all healthy Pods — even as individual Pods restart and get new IPs.
• Every Pod has a unique cluster-internal IP.
• Services expose stable DNS names inside the cluster: No hardcoded IPs.

flowchart TD
    Browser[Browser / API Client] --> LB["Cloud Load Balancer (AWS ALB / GCP LB)"]
    LB --> Ingress["Ingress Controller (nginx-ingress Pod)"]
    Ingress --> SVC["Service (ClusterIP: 10.96.0.42)"]
    SVC --> P1["Pod 1 (10.244.1.5)"]
    SVC --> P2["Pod 2 (10.244.2.8)"]
    SVC --> P3["Pod 3 (10.244.3.2)"]
    style LB fill:#fff3cd,stroke:#f0ad4e
    style SVC fill:#d4edda,stroke:#28a745

http://payment-service.default.svc.cluster.local:8080

Observable outcome: Services expose stable DNS names inside the cluster: No hardcoded IPs.

Stage 5 — Real-World Application: Running a Production E-Commerce Platform on Kubernetes

• Shopify , Zalando , and Airbnb run Kubernetes clusters handling millions of requests per hour.
• A production slice: three services ( storefront , cart , payment ) — each with a Deployment and Service.
• An Ingress exposes storefront externally.
• A HorizontalPodAutoscaler (HPA) scales storefront automatically on CPU.

flowchart LR
    HPA["HorizontalPodAutoscaler (CPU > 70%  add Pods)"] -.->|controls| SF
    Ingress["Ingress (public)"] --> SF[storefront Service Pods x320]
    SF --> Cart[cart Service Pods x2]
    SF --> Pay[payment Service Pods x2]
    style HPA fill:#d4edda,stroke:#28a745

Observable outcome: A HorizontalPodAutoscaler (HPA) scales storefront automatically on CPU.

Stage 6 — Practical Example: Auto-Scaling Storefront with an HPA

• The HorizontalPodAutoscaler watches the metrics-server and adjusts your Deployment's replica count continuously: With this applied: 1.
• Deploy normally with kubectl apply -f deployment.yaml and kubectl apply -f hpa.yaml .
• Black Friday traffic hits — average CPU crosses 70% — the HPA adds replicas every 15 seconds until CPU returns to ~70%.
• Traffic subsides — HPA scales back down, respecting stabilizationWindowSeconds to avoid flapping.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: storefront-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: storefront
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - typ
e: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Observable outcome: Traffic subsides — HPA scales back down, respecting stabilizationWindowSeconds to avoid flapping.

End-to-end outcome

Before Kubernetes, deploying an app meant SSH-ing into servers and running commands manually.

Alternatives considered

Decision Guide: When Kubernetes Pays Off

Decision Guide: When Kubernetes Pays Off

Failure and degraded modes

Trade-offs & Failure Modes: Trade-offs, Failure Modes, and the Operational Complexity Tax

• Detection: Kubernetes is powerful, but the operational cost is real.
• User or operator impact: Kubernetes is powerful, but the operational cost is real.