Skip to main content
← PROBLEMS / ENGINEERING
An end-to-end domain learning journey

Resolving Linux Conntrack
Table Exhaustion.

Your API gateway cluster (8 Envoy/Nginx nodes, 16 vCPU, 64 GiB RAM, Linux 5.15) handles 85,000 req/s of mixed external TLS traffic and internal microservice calls. During a flash traffic surge, p99 latency spikes to 8 seconds and 3.5% of requests time out with TCP SYN retransmissions. dmesg on the gateway nodes logs nf_conntrack: table full, dropping packet, and conntrack -S shows insert_failed=482190 and drop=482190 while CPU utilization is only 38%.
Start the journey5 stages
Loading diagram…
Resolving Linux Conntrack Table Exhaustion architecture sketch diagram
FIRST PRINCIPLES. PRODUCTION SCALE.
The idea to keep with you

Your API gateway cluster (8 Envoy/Nginx nodes, 16 vCPU, 64 GiB RAM, Linux 5.15) handles `85,000 req/s` of mixed external TLS traffic and internal microservice calls. During a flash traffic surge, p99 latency spikes to 8 seconds and `3.5%` of requests time out with TCP `SYN` retransmissions. `dmesg` on the gateway nodes logs `nf_conntrack: table full, dropping packet`, and `conntrack -S` shows `insert_failed=482190` and `drop=482190` while CPU utilization is only `38%`.

From patterns to decisions.

Your API gateway cluster (8 Envoy/Nginx nodes, 16 vCPU, 64 GiB RAM, Linux 5.15) handles `85,000 req/s` of mixed external TLS traffic and internal microservice calls. During a flash traffic surge, p99 latency spikes to 8 seconds and `3.5%` of requests time out with TCP `SYN` retransmissions. `dmesg` on the gateway nodes logs `nf_conntrack: table full, dropping packet`, and `conntrack -S` shows `insert_failed=482190` and `drop=482190` while CPU utilization is only `38%`.

01
ENGINEERING · STAGE 01L4 · FUNDAMENTALS

Conntrack exhaustion is a fixed.

Conntrack exhaustion is a fixed table-capacity limit (nf_conntrack_max, default 65,536), not a CPU bottleneck. Closed flows linger for 120s in TIME_WAIT, filling the table.

ChoiceWhat it buysWhat it costs
Trade-off 1Reading `/proc/sys/net/netfilter/nf_conntrack_count` is O(1), whereas `conntrack -L` locks and dumps the entire table.

Step 1

Check sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max.

Step 2

Run conntrack -S to inspect insert_failed and drop counters.

Step 3

Break down states with conntrack -L | awk '{print $4}' | sort | uniq -c (check TIME_WAIT share).

Alternative 1

Scrape node_nf_conntrack_entries from Prometheus node_exporter.

Failure Mode 1

Running conntrack -L on a 1M-entry table in production can briefly stall packet processing.

def read_conntrack_utilization() -> dict[str, float]:
    with open("/proc/sys/net/netfilter/nf_conntrack_count") as f:
        count = int(f.read().strip())
    with open("/proc/sys/net/netfilter/nf_conntrack_max") as f:
        limit = int(f.read().strip())
    return {"count": count, "max": limit, "utilization_pct": round((count / limit) * 100, 2)}
02
ENGINEERING · STAGE 02L5 · APPLIED DESIGN

Implementation & Guardrails (2).

Configure nf_conntrack_max=1048576, hashsize=262144, time_wait=30s, and enable persistent upstream keepalive pools in Nginx.

ChoiceWhat it buysWhat it costs
Trade-off 1Persistent keep-alive pools hold open file descriptors (`nofile` ulimit must be raised) but cut TCP handshakes by >95%.

Step 1

Apply sysctl parameters in /etc/sysctl.d/99-conntrack.conf.

Step 2

Set hashsize=262144 in /sys/module/nf_conntrack/parameters/hashsize.

Step 3

Enable keepalive 512 and HTTP/1.1 persistent connections in Nginx.

Alternative 1

Use Envoy with http2_protocol_options multiplexing over a small number of backend TCP connections.

Failure Mode 1

Forgetting proxy_set_header Connection ""; causes Nginx to forward Connection: close and negate keepalive.

upstream api_backend {
    server 10.20.1.10:8080;
    server 10.20.1.11:8080;
    keepalive 512;
    keepalive_requests 10000;
    keepalive_timeout 60s;
}

server {
    location / {
        proxy_pass http://api_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}
03
ENGINEERING · STAGE 03L6 · SCALE & TRADE-OFFS

Use NOTRACK when traffic does.

Use NOTRACK when traffic does not require NAT (MASQUERADE/DNAT) or stateful firewalling—such as loopback, internal BGP/direct-routed VIPs, or high-QPS UDP telemetry.

ChoiceWhat it buysWhat it costs
Trade-off 1`NOTRACK` eliminates all conntrack RAM and spinlock overhead, but disables `RELATED,ESTABLISHED` stateful firewalling for that traffic.

Step 1

Verify whether the interface/port performs SNAT/DNAT (if yes, NOTRACK will break it).

Step 2

Add -t raw -A PREROUTING -i lo -j NOTRACK and -t raw -A OUTPUT -o lo -j NOTRACK.

Step 3

Ensure ACCEPT rule for UNTRACKED state exists in filter table.

Alternative 1

Adopt Cilium eBPF kube-proxy replacement with Direct Server Return (DSR) to bypass Netfilter completely.

Failure Mode 1

Adding NOTRACK without allowing -m conntrack --ctstate UNTRACKED -j ACCEPT drops the untracked packets in the filter chain.

# Example sysctl.d/99-conntrack.conf paired with raw NOTRACK
net.netfilter.nf_conntrack_max: 1048576
net.netfilter.nf_conntrack_tcp_timeout_time_wait: 30
net.netfilter.nf_conntrack_tcp_timeout_established: 3600
net.netfilter.nf_conntrack_udp_timeout: 15
04
ENGINEERING · STAGE 04L6 · SCALE & TRADE-OFFS

A SYN flood fills nf_conntrack.

A SYN flood fills nf_conntrack with SYN_RECV entries if not mitigated. Enable tcp_syncookies=1, shorten nf_conntrack_tcp_timeout_syn_recv=15, and drop floods in XDP/edge WAF before Netfilter.

ChoiceWhat it buysWhat it costs
Trade-off 1SYN cookies disable some TCP options (like window scaling if timestamps are off) during an active flood, but keep the gateway alive.

Step 1

Enable net.ipv4.tcp_syncookies = 1 and net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 15.

Step 2

Deploy XDP (eBPF at NIC driver level) or upstream AWS Shield / Cloudflare to drop spoofed SYNs before sk_buff reaches Netfilter.

Step 3

Monitor SYN_RECV count in conntrack.

Alternative 1

Use iptables -t raw -j SYNPROXY to validate the 3-way handshake before creating any nf_conntrack entry.

Failure Mode 1

Relying solely on nf_conntrack_max during a volumetric SYN flood still exhausts kernel memory.

def detect_syn_flood(syn_recv_count: int, conntrack_max: int) -> bool:
    # Alert if half-open SYN_RECV entries exceed 20% of conntrack capacity
    return (syn_recv_count / conntrack_max) > 0.20
05
ENGINEERING · STAGE 05L7 · STAFF / PRINCIPAL

Move L4 load balancing to.

Move L4 load balancing to eBPF/XDP (e.g., Cilium or Katran) with Direct Server Return (DSR) and BPF_MAP_TYPE_LRU_HASH per-CPU conntrack maps.

ChoiceWhat it buysWhat it costs
Trade-off 1XDP/DSR requires L3/L2 network topology support (e.g., BGP ECMP + IPIP/GRE encapsulation) and specialized eBPF observability.

Step 1

Replace iptables NAT with eBPF tc/XDP forwarding.

Step 2

Use DSR so return traffic flows directly from backend to client, halving gateway PPS.

Step 3

Replace global nf_conntrack locks with lockless per-CPU eBPF LRU hash maps.

Alternative 1

Scale horizontally behind AWS NLB / GCP Maglev with IP preservation.

Failure Mode 1

Asymmetric routing firewalls dropping DSR return packets if stateful security groups sit in the return path.

def estimate_dsr_savings(ingress_pps: int, egress_pps: int) -> dict[str, int]:
    # DSR bypasses gateway on return path (typically 4x-10x larger than ingress)
    return {"without_dsr_pps": ingress_pps + egress_pps, "with_dsr_pps": ingress_pps}

Theory & Tips and Tricks

Practical heuristics & interview traps

  • 01Interview anchor: Size nf_conntrack_max and nf_conntrack_buckets (max / 4) for peak active + TIME_WAIT flows, shorten nf_conntrack_tcp_timeout_time_wait, enable HTTP keep-alive, or bypass conntrack via NOTRACK/eBPF.
  • 02Trade-off tip: NOTRACK eliminates all conntrack RAM and spinlock overhead, but disables RELATED,ESTABLISHED stateful firewalling for that traffic.
  • 03Watch out for: Relying solely on nf_conntrack_max during a volumetric SYN flood still exhausts kernel memory.
Behavioral Progression · Linux Conntrack Table Exhaustion

Linux Conntrack Table Exhaustion — Normal → Load / Fault → Recovery

Send Request
1. Normal Request Ingress
Traffic: 1x baseline · Latency (p99): 4 ms
Process State
2. Core Execution & State Lookup
Traffic: 1x baseline · Latency (p99): 9 ms
Inject 10x Spike
3. Scale Spike / Failure Injection
Traffic: 10x burst · Latency (p99): 185 ms

Size nfconntrackmax and nfconntrackbuckets (max / 4) for peak active + TIMEWAIT flows, shorten nfconntracktcptimeouttimewait, enable HTTP keep-alive, or bypass conntrack via NOTRACK/eBPF.

Synthesis / One connected design

Putting Resolving Linux Conntrack Table Exhaustion Together

Size `nf_conntrack_max` and `nf_conntrack_buckets` (`max / 4`) for peak active + `TIME_WAIT` flows, shorten `nf_conntrack_tcp_timeout_time_wait`, enable HTTP keep-alive, or bypass conntrack via `NOTRACK`/eBPF.

A candidate architecture, not a universal answer

Putting Resolving Linux Conntrack Table Exhaustion Together

Step 1
Step 2
Step 3
  1. Apply sysctl parameters in /etc/sysctl.d/99-conntrack.conf.
  2. Set hashsize=262144 in /sys/module/nf_conntrack/parameters/hashsize.
  3. Enable keepalive 512 and HTTP/1.1 persistent connections in Nginx.
A `SYN` flood fills `nf_conntrack` with `SYN_RECV` entries if not mitigated. Enable `tcp_syncookies=1`, shorten `nf_conntrack_tcp_timeout_syn_recv=15`, and drop floods in `XDP`/edge WAF before Netfilter.
Interview Readiness Check

A SYN flood fills nf_conntrack with SYN_RECV entries if not mitigated. Enable tcp_syncookies=1, shorten nf_conntrack_tcp_timeout_syn_recv=15, and drop floods in XDP/edge WAF before Netfilter.

Walk through bottlenecks, back-of-the-envelope math, and failure recovery.

Practice / Explain the consequences

Make the trade-off yourself.

Practice Check

Test your understanding

Question 1 of 3

When is ACID Transaction Fundamentals the right default for a production team?

Key takeaway

Resolving Linux Conntrack Table Exhaustion

Size nf_conntrack_max and nf_conntrack_buckets (max / 4) for peak active + TIME_WAIT flows, shorten nf_conntrack_tcp_timeout_time_wait, enable HTTP keep-alive, or bypass conntrack via NOTRACK/eBPF.