WebSocket Connection Flooding Exhausting File Descriptors & Socket Memory

System Design Interview•~30 min read•medium

Real-time sports score delivery service receiving 200,000 simultaneous WebSocket connections.

1. Understand Requirements & Scope

Real-time sports score delivery service receiving 200,000 simultaneous WebSocket connections. Failure cascade dynamic: Object Churn → Unbounded Off-Heap → GC Freezes → 💥.

Functional Requirements:

Operational Invariant: "Optimize kernel TCP socket buffers and scale OS file descriptors to manage high-cardinality idle WebSocket fleets."
Production Hardening: Memory Tune Linux socket buffer sizes for high- connection/low-bandwidth WebSocket servers: reduce minimum TCP memory buffers ('net.ipv4.tcp_rmem = 4096 87380 16777216'), and offlo
Prevent cascading failure by isolating downstream degradation at the ingress boundary.

2. Back-of-the-Envelope Capacity Estimation

• Incident Scenario Baseline: Real-time sports score delivery service receiving 200,000 simultaneous WebSocket connections.

• Failure Cascade Path: Object Churn → Unbounded Off-Heap → GC Freezes → 💥

• Sub-millisecond rejection / shed latency for overloaded requests

3. High-Level Architecture Flow

A global real-time chat architecture handles millions of concurrent persistent WebSocket connections by isolating connection termination to stateless Chat Gateway nodes, utilizing Redis Pub/Sub for inter-server user routing, and persisting chat history to an append-heavy wide-column store (Cassandra / DynamoDB) partitioned by channel_id.

[User A] -> (WebSocket) -> [Chat Gateway 1]
                                  |
                                  v
                         [ Kafka Message Queue ]
                                  |
                         +--------+--------+
                         |                 |
                         v                 v
                [ DB Writer Service ]   [ Redis Pub/Sub Router ]
                         |                 |
                         v                 v
               [ Cassandra Cluster ]    [Chat Gateway 3] -> (WebSocket) -> [User B]

Fig 1. Architecture data flow and component topology.

4. Production Implementation & Code

// Epoll / Async WebSocket Ingress Event Loop Sizing
// Linux kernel limits in /etc/security/limits.conf
* soft nofile 1048576
* hard nofile 1048576

// Netfilter connection tracking table tuning
net.netfilter.nf_conntrack_max = 2097152
net.ipv4.tcp_mem = 786432 1048576 1572864 // TCP memory buffers (pages)

Hardened Implementation Sequence

Ingress Boundary Defense: Memory Tune Linux socket buffer sizes for high- connection/low-bandwidth WebSocket servers: reduce minimum TCP memory buffers ('net.ipv4.tcp_rmem = 40
Failure Containment: Eliminate unconstrained buffering and propagate physical backpressure.
Invariant Enforcement: Optimize kernel TCP socket buffers and scale OS file descriptors to manage high-cardinality idle WebSocket fleets.

5. Key Decisions & Operational Mitigations

Architectural Decisions & Trade-Offs

•Ingress Admission: Naive unconstrained buffering causes queue bloat and GC exhaustion. Mitigation: Bounded ingress gate with fast non-blocking shedding.
•Resource Strategy: Static capacity allocations collapse under brownouts. Mitigation: Dynamic adaptive limits scaling to real-time physical bounds.
•Outage Impact: Cascading lock convoys and memory saturation. Mitigation: Isolated failure domains keep P99 latency flat and protected.

Operational Hazards & Mitigations

•Cascading Failure: OOMKill (Exit 137) Each TCP socket allocates default 128 KB receive and transmit buffers (net.ipv4.tcp_rmem / wmem). 200,000 connections consume 25 GB of non-swappable kernel socket memory, causing the Linux kernel to drop new packets.
•Breakdown Chain: Object Churn → Unbounded Off-Heap → GC Freezes → 💥
•Shedding Failure: Without fast rejection at the boundary, queued dead work triggers multi-second GC pauses, CPU thrashing, and cascading 504 Gateway Timeouts.

Interactive Knowledge Check • Question 1 of 3↓

Interactive Practice Quiz•Question 1 of 3

intermediate

Circuit Breaker State Machine Transitions

In a Circuit Breaker state machine (Closed, Open, Half-Open), what triggers the transition from OPEN back to HALF-OPEN?