Your checkout web app sits behind an AWS Application Load Balancer (default `idle_timeout = 60s`) and an Nginx ingress (`keepalive_timeout = 65s`), forwarding to Spring Boot Tomcat (`server.tomcat.keep-alive-timeout = 60s`) and a Node.js microservice (`server.keepAliveTimeout = 5s`). Under steady traffic, 0.3% of `POST /orders` requests fail randomly with `502 Bad Gateway` and `upstream prematurely closed connection` / `ECONNRESET` in Nginx/ALB logs—even though backend pods show 0 errors and <20% CPU. Diagnose the TCP race condition and fix the timeout hierarchy.
TL;DR30-second simplified summary▾
Your checkout web app sits behind an AWS Application Load Balancer (default idle_timeout = 60s) and an Nginx ingress (keepalive_timeout = 65s), forwarding to Spring Boot Tomcat (server.tomcat.keep-alive-timeout = 60s) and a Node.js microservice (server.keepAliveTimeout = 5s). Under steady traffic, 0.3% of POST /orders requests fail randomly with 502 Bad Gateway and upstream prematurely closed connection / ECONNRESET in Nginx/ALB logs—even though backend pods show 0 errors and <20% CPU. Diagnose the TCP race condition and fix the timeout hierarchy.
Always set the backend's keepAliveTimeout strictly greater than the upstream proxy's idle timeout (e.g., ALB 60s -> Backend 75s), and let the proxy initiate idle closes.
From patterns to decisions.
Your checkout web app sits behind an AWS Application Load Balancer (default `idle_timeout = 60s`) and an Nginx ingress (`keepalive_timeout = 65s`), forwarding to Spring Boot Tomcat (`server.tomcat.keep-alive-timeout = 60s`) and a Node.js microservice (`server.keepAliveTimeout = 5s`). Under steady traffic, 0.3% of `POST /orders` requests fail randomly with `502 Bad Gateway` and `upstream prematurely closed connection` / `ECONNRESET` in Nginx/ALB logs—even though backend pods show 0 errors and <20% CPU. Diagnose the TCP race condition and fix the timeout hierarchy.
The backend closes an idle.
The backend closes an idle TCP socket (FIN) right as the proxy writes a new request to it. The backend kernel replies with TCP RST before the HTTP layer ever sees the request.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Keep-alive eliminates TCP handshake latency, but requires asymmetric timeout ownership. |
Check default timeouts
Node.js (5s), Tomcat (60s), ALB (60s), Nginx (65s).
Step 2
Observe that any backend timeout <= upstream proxy timeout creates a race window.
Alternative 1
Use HTTP/2 upstream where GOAWAY prevents write-after-close races.
Failure Mode 1
Backend logs remain completely clean, misleading engineers into blaming the network.
# Classic 502 race timeline:
# t=5.000s: Node.js sends TCP FIN
# t=5.0001s: Nginx writes POST /orders to the same socket
# t=5.0002s: Node.js kernel responds TCP RST -> Nginx returns 502
Set Nginx server keepalive_timeout to.
Set Nginx server keepalive_timeout to 75s (> ALB 60s), Nginx upstream keepalive_timeout to 55s, and both Tomcat and Node.js keepAliveTimeout to 90s.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | 90s backend timeout holds idle sockets slightly longer, using negligible RAM while eliminating 502s. |
Step 1
Configure Nginx server block (75s) and upstream block (55s).
Step 2
Configure Spring Boot server.tomcat.keep-alive-timeout: 90s.
Step 3
Configure Node.js server.keepAliveTimeout = 90000 and server.headersTimeout = 95000.
Alternative 1
In Nginx, set upstream { keepalive_timeout 55s; keepalive_requests 5000; }.
Failure Mode 1
Forgetting headersTimeout > keepAliveTimeout in Node.js still triggers premature closes.
# Spring Boot application.yaml
server:
tomcat:
keep-alive-timeout: 90s
max-keep-alive-requests: 10000
# Node.js equivalent:
# server.keepAliveTimeout = 90000;
# server.headersTimeout = 95000;
Retrying error on non-idempotent POST.
Retrying error on non-idempotent POST /orders is dangerous (and disabled by default in Nginx) because a socket error after partial write can duplicate orders.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Fixing timeout ordering prevents the error at the source for all HTTP methods including `POST`. |
Step 1
Explain that proxy_next_upstream skips POST unless non_idempotent is explicitly enabled.
Step 2
Even for GET, retrying masks the root cause and adds tail latency.
Alternative 1
Add Idempotency-Key support if POST retries are ever needed for real crashes.
Failure Mode 1
Enabling non_idempotent retries causes double-charges during slow backend responses.
# Nginx upstream block (safe)
upstream backend_pool {
server app-service:8080;
keepalive 128;
keepalive_timeout 55s;
keepalive_requests 5000;
}
Implementation & Guardrails (4).
Tomcat's default maxKeepAliveRequests is 100. At 6,000 RPS across 30 pods (200 RPS/pod), each socket hits 100 requests every 0.5 seconds and closes!
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Higher `maxKeepAliveRequests` maximizes socket reuse; ensure load balancer still spreads traffic across new pods. |
Step 1
Raise server.tomcat.max-keep-alive-requests from 100 to 10000 (or -1).
Step 2
Align Nginx keepalive_requests 5000 so Nginx closes the socket before Tomcat's 10000.
Alternative 1
Use maxKeepAliveRequests = -1 (unlimited) and let Nginx's keepalive_requests 5000 manage rotation.
Failure Mode 1
Leaving maxKeepAliveRequests = 100 recreates the exact same FIN/write race via request count instead of time.
server:
tomcat:
keep-alive-timeout: 90s
max-keep-alive-requests: 10000 # Must exceed Nginx keepalive_requests (5000)
Capture TCP FIN flags with.
Capture TCP FIN flags with tcpdump on the backend port (8080) and verify the source IP of FIN packets is Nginx, not the pod.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Quick 30-second `tcpdump` validates the fix definitively. |
Step 1
Run tcpdump -ni any 'tcp[tcpflags] & tcp-fin != 0 and port 8080'.
Step 2
Confirm upstream prematurely closed connection drops to 0 in Nginx metrics.
Alternative 1
Monitor nginx_ingress_controller_requests{status="502"} in Prometheus.
Failure Mode 1
Seeing RST from backend port 8080 indicates another timeout (e.g., requestTimeout or maxKeepAliveRequests) is still firing.
# Verify FIN direction on backend pod
# tcpdump -ni eth0 'tcp[tcpflags] & (tcp-fin|tcp-rst) != 0 and port 8080'
Theory & Tips and Tricks
Practical heuristics & interview traps
- 01Interview anchor: Always set the backend's
keepAliveTimeoutstrictly greater than the upstream proxy's idle timeout (e.g., ALB 60s -> Backend 75s), and let the proxy initiate idle closes. - 02Trade-off tip: Fixing timeout ordering prevents the error at the source for all HTTP methods including
POST. - 03Watch out for: Leaving
maxKeepAliveRequests = 100recreates the exact sameFIN/write race via request count instead of time.
Putting Aligning HTTP Keep-Alive Timeouts Across Proxies Together
Always set the backend's `keepAliveTimeout` strictly greater than the upstream proxy's idle timeout (e.g., ALB 60s -> Backend 75s), and let the proxy initiate idle closes.
Putting Aligning HTTP Keep-Alive Timeouts Across Proxies Together
- Configure Nginx server block (
75s) and upstream block (55s). - Configure Spring Boot
server.tomcat.keep-alive-timeout: 90s. - Configure Node.js
server.keepAliveTimeout = 90000andserver.headersTimeout = 95000.
Interview Readiness Check
Tomcat's default maxKeepAliveRequests is 100. At 6,000 RPS across 30 pods (200 RPS/pod), each socket hits 100 requests every 0.5 seconds and closes!
Walk through bottlenecks, back-of-the-envelope math, and failure recovery.
Practice: Make the trade-off yourself.
Practice Check
Test your understanding
When is ACID Transaction Fundamentals the right default for a production team?
Aligning HTTP Keep-Alive Timeouts Across Proxies
Always set the backend's keepAliveTimeout strictly greater than the upstream proxy's idle timeout (e.g., ALB 60s -> Backend 75s), and let the proxy initiate idle closes.
