Skip to main content
← PROBLEMS / ENGINEERING
An end-to-end domain learning journey

Fixing Kubernetes CFS
CPU Throttling Freezes.

A latency-sensitive Java/Netty payment gateway running in Kubernetes (requests.cpu: '500m', limits.cpu: '2') on 64-vCPU nodes has a median latency of 4ms, but p99 latency spikes to 95-120ms every few seconds. Node CPU utilization is only 35%, yet container_cpu_cfs_throttled_periods_total is 45% and nr_throttled climbs rapidly. The JVM sees all 64 host cores via /proc/cpuinfo (or defaults) and spawns 64 Netty/ForkJoin/G1GC threads. Diagnose the Linux CFS bandwidth mechanism causing the 95ms freezes and fix the Kubernetes and runtime configuration.
Start the journey5 stages
Loading diagram…
Fixing Kubernetes CFS CPU Throttling Freezes architecture sketch diagram
FIRST PRINCIPLES. PRODUCTION SCALE.
The idea to keep with you

A latency-sensitive Java/Netty payment gateway running in Kubernetes (`requests.cpu: '500m'`, `limits.cpu: '2'`) on 64-vCPU nodes has a median latency of 4ms, but p99 latency spikes to 95-120ms every few seconds. Node CPU utilization is only 35%, yet `container_cpu_cfs_throttled_periods_total` is 45% and `nr_throttled` climbs rapidly. The JVM sees all 64 host cores via `/proc/cpuinfo` (or defaults) and spawns 64 Netty/ForkJoin/G1GC threads. Diagnose the Linux CFS bandwidth mechanism causing the 95ms freezes and fix the Kubernetes and runtime configuration.

From patterns to decisions.

A latency-sensitive Java/Netty payment gateway running in Kubernetes (`requests.cpu: '500m'`, `limits.cpu: '2'`) on 64-vCPU nodes has a median latency of 4ms, but p99 latency spikes to 95-120ms every few seconds. Node CPU utilization is only 35%, yet `container_cpu_cfs_throttled_periods_total` is 45% and `nr_throttled` climbs rapidly. The JVM sees all 64 host cores via `/proc/cpuinfo` (or defaults) and spawns 64 Netty/ForkJoin/G1GC threads. Diagnose the Linux CFS bandwidth mechanism causing the 95ms freezes and fix the Kubernetes and runtime configuration.

01
ENGINEERING · STAGE 01L4 · FUNDAMENTALS

Linux CFS gives a 2.

Linux CFS gives a 2 CPU pod 200ms of CPU time per 100ms wall-clock period. 16 parallel threads consume 200ms in 12.5ms, freezing the entire pod for the remaining 87.5ms.

ChoiceWhat it buysWhat it costs
Trade-off 15-minute average CPU hides 100ms-scale CFS throttling completely.

Step 1

Compute cpu.cfs_quota_us (200,000) / cpu.cfs_period_us (100,000).

Step 2

Divide 200ms by 16 active threads to show 12.5ms execution + 87.5ms freeze.

Alternative 1

Inspect /sys/fs/cgroup/cpu.stat (nr_throttled, throttled_usec).

Failure Mode 1

GC pauses and Netty event-loop stalls cascade into upstream timeouts.

# Anti-pattern: bursty multi-threaded JVM with low requests and hard 2-core limit
resources:
  requests:
    cpu: "500m"
  limits:
    cpu: "2" # 200ms quota per 100ms period -> severe throttling!
02
ENGINEERING · STAGE 02L5 · APPLIED DESIGN

Right-size requests.

Right-size requests.cpu: 2, remove limits.cpu (or use Guaranteed static pinning), and pass -XX:ActiveProcessorCount=2 to the JVM.

ChoiceWhat it buysWhat it costs
Trade-off 1Zero CFS throttling; pod is guaranteed 2 cores via `cpu.weight` under contention and can burst when host is idle.

Set `requests.cpu

"2" and remove limits.cpu` for zero CFS throttling.

Add `JAVA_TOOL_OPTIONS` with `-XX

ActiveProcessorCount=2` and bounded GC threads.

Alternative 1

Set limits.cpu: "2" + Guaranteed QoS on a node pool with --cpu-manager-policy=static.

Leaving `requests.cpu

500m without limits.cpu` starves the pod during node contention.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-gateway
spec:
  template:
    spec:
      containers:
        - name: gateway
          image: payment-gateway:2.4.1
          env:
            - name: JAVA_TOOL_OPTIONS
              value: "-XX:ActiveProcessorCount=2 -XX:ParallelGCThreads=2 -XX:ConcGCThreads=1"
          resources:
            requests:
              cpu: "2"
              memory: "4Gi"
            limits:
              memory: "4Gi"
              # cpu limit intentionally omitted to prevent CFS throttling freezes
03
ENGINEERING · STAGE 03L6 · SCALE & TRADE-OFFS

Use either (1) Guaranteed QoS.

Use either (1) Guaranteed QoS with integer CPUs + kubelet --cpu-manager-policy=static (bypasses CFS quota via cpuset), or (2) CustomCPUCFSQuotaPeriod reducing cpuCFSQuotaPeriod from 100ms to 5ms.

ChoiceWhat it buysWhat it costs
Trade-off 1`static` CPU Manager requires integer CPU requests (`2`, not `1500m`) and `Guaranteed` QoS (`requests == limits` for both CPU and memory).

Configure `cpuManagerPolicy

static` in KubeletConfiguration for exclusive core pinning.

Alternatively set `cpuCFSQuotaPeriod

5ms` so any freeze is <= 3.75ms.

Alternative 1

Keep 100ms period and drop limits.cpu for Burstable pods.

Failure Mode 1

Fractional CPU (1500m) silently falls back to the shared CFS pool even when cpuManagerPolicy: static is enabled.

# KubeletConfiguration for strict limits without 90ms freezes
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cpuManagerPolicy: static
cpuCFSQuota: true
cpuCFSQuotaPeriod: 5ms
04
ENGINEERING · STAGE 04L6 · SCALE & TRADE-OFFS

In Go, use uber-go/automaxprocs (reads.

In Go, use uber-go/automaxprocs (reads cgroup CPU quota and sets GOMAXPROCS=2 instead of 64). In Python/Gunicorn, size --workers and OMP_NUM_THREADS to the CPU request.

ChoiceWhat it buysWhat it costs
Trade-off 1Dynamic cgroup inspection prevents worker over-allocation when pod CPU requests are resized.

Step 1

Avoid reading os.cpu_count() (returns 64 host cores) when sizing Gunicorn/Uvicorn/ThreadPoolExecutor.

Step 2

Read /sys/fs/cgroup/cpu.max in Python to cap worker/thread pools dynamically.

Alternative 1

Pass WEB_CONCURRENCY=2 and OMP_NUM_THREADS=2 explicitly via Kubernetes Downward API.

Failure Mode 1

os.cpu_count() in Python ignores cgroup v2 cpu.max and returns 64.

import math
from pathlib import Path

def cgroup_cpu_limit(default: int = 2) -> int:
    cpu_max = Path("/sys/fs/cgroup/cpu.max")
    if cpu_max.exists():
        quota, period = cpu_max.read_text().strip().split()
        if quota != "max":
            return max(1, math.ceil(int(quota) / int(period)))
    return default
05
ENGINEERING · STAGE 05L7 · STAFF / PRINCIPAL

Alert on the ratio of.

Alert on the ratio of throttled periods (container_cpu_cfs_throttled_periods_total / container_cpu_cfs_periods_total > 0.05) and total throttled time (container_cpu_cfs_throttled_seconds_total).

ChoiceWhat it buysWhat it costs
Trade-off 1Catches sub-second throttling freezes immediately regardless of low 5-minute average CPU.

Step 1

Monitor both throttled_periods ratio (>5%) and throttled_seconds rate.

Step 2

Correlate spikes directly against service http_request_duration_seconds p99.

Alternative 1

Read /sys/fs/cgroup/cpu.stat throttled_usec directly via OpenTelemetry hostmetrics.

Failure Mode 1

Division by zero on idle containers with container_cpu_cfs_periods_total == 0 unless filtered.

# PrometheusRule for CFS CPU throttling
- alert: HighContainerCPUThrottling
  expr: |
    (
      rate(container_cpu_cfs_throttled_periods_total{container!=""}[2m])
      /
      rate(container_cpu_cfs_periods_total{container!=""}[2m])
    ) > 0.05
  for: 5m
  labels:
    severity: warning

Theory & Tips and Tricks

Practical heuristics & interview traps

  • 01Interview anchor: Align runtime thread pools (ActiveProcessorCount / GOMAXPROCS) to the CPU quota, use Guaranteed QoS with integer CPUs (static kubelet CPU Manager) or remove limits.cpu on latency-critical pods, and shorten cfs_period_us if limits are mandatory.
  • 02Trade-off tip: static CPU Manager requires integer CPU requests (2, not 1500m) and Guaranteed QoS (requests == limits for both CPU and memory).
  • 03Watch out for: os.cpu_count() in Python ignores cgroup v2 cpu.max and returns 64.
Behavioral Progression · Fixing Kubernetes Cfs Cpu Throttling

Fixing Kubernetes Cfs Cpu Throttling — Normal → Load / Fault → Recovery

Send Request
1. Normal Request Ingress
Traffic: 1x baseline · Latency (p99): 4 ms
Process State
2. Core Execution & State Lookup
Traffic: 1x baseline · Latency (p99): 9 ms
Inject 10x Spike
3. Scale Spike / Failure Injection
Traffic: 10x burst · Latency (p99): 185 ms

Align runtime thread pools (ActiveProcessorCount / GOMAXPROCS) to the CPU quota, use Guaranteed QoS with integer CPUs (static kubelet CPU Manager) or remove limits.cpu on latency-critical pods, and shorten cfsperiodus if limits are mandatory.

Synthesis / One connected design

Putting Fixing Kubernetes CFS CPU Throttling Freezes Together

Align runtime thread pools (`ActiveProcessorCount` / `GOMAXPROCS`) to the CPU quota, use `Guaranteed` QoS with integer CPUs (`static` kubelet CPU Manager) or remove `limits.cpu` on latency-critical pods, and shorten `cfs_period_us` if limits are mandatory.

A candidate architecture, not a universal answer

Putting Fixing Kubernetes CFS CPU Throttling Freezes Together

Step 1
Step 2
  1. Set requests.cpu: "2" and remove limits.cpu for zero CFS throttling.
  2. Add JAVA_TOOL_OPTIONS with -XX:ActiveProcessorCount=2 and bounded GC threads.
In Go, use `uber-go/automaxprocs` (reads cgroup CPU quota and sets `GOMAXPROCS=2` instead of 64). In Python/Gunicorn, size `--workers` and `OMP_NUM_THREADS` to the CPU request.
Interview Readiness Check

In Go, use uber-go/automaxprocs (reads cgroup CPU quota and sets GOMAXPROCS=2 instead of 64). In Python/Gunicorn, size --workers and OMP_NUM_THREADS to the CPU request.

Walk through bottlenecks, back-of-the-envelope math, and failure recovery.

Practice / Explain the consequences

Make the trade-off yourself.

Practice Check

Test your understanding

Question 1 of 3

When is ACID Transaction Fundamentals the right default for a production team?

Key takeaway

Fixing Kubernetes CFS CPU Throttling Freezes

Align runtime thread pools (ActiveProcessorCount / GOMAXPROCS) to the CPU quota, use Guaranteed QoS with integer CPUs (static kubelet CPU Manager) or remove limits.cpu on latency-critical pods, and shorten cfs_period_us if limits are mandatory.