A Java 21 Spring Boot checkout API on AWS Lambda (and Cloud Run for EU) normally handles 150 RPS at 45 ms p95. When marketing push notifications trigger a 3,000 RPS spike within 10 seconds, 400+ concurrent cold starts fire simultaneously. Each cold start takes 3.4 seconds (JVM + Spring context + RDS Postgres TLS handshake), breaching the 2 s API Gateway timeout and exhausting Postgres max_connections (500).
TL;DR30-second simplified summary▾
A Java 21 Spring Boot checkout API on AWS Lambda (and Cloud Run for EU) normally handles 150 RPS at 45 ms p95. When marketing push notifications trigger a 3,000 RPS spike within 10 seconds, 400+ concurrent cold starts fire simultaneously. Each cold start takes 3.4 seconds (JVM + Spring context + RDS Postgres TLS handshake), breaching the 2 s API Gateway timeout and exhausting Postgres max_connections (500).
Combine SnapStart / container image optimization with predictive Provisioned Concurrency (min-instances), lazy initialization, and an external connection pooler (RDS Proxy / PgBouncer).
From patterns to decisions.
A Java 21 Spring Boot checkout API on AWS Lambda (and Cloud Run for EU) normally handles 150 RPS at 45 ms p95. When marketing push notifications trigger a 3,000 RPS spike within 10 seconds, 400+ concurrent cold starts fire simultaneously. Each cold start takes 3.4 seconds (JVM + Spring context + RDS Postgres TLS handshake), breaching the 2 s API Gateway timeout and exhausting Postgres max_connections (500).
Cold starts stack microVM boot.
Cold starts stack microVM boot, JVM/Spring DI scanning, and TLS DB handshakes, while per-sandbox connection pools multiply linearly with concurrency.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Direct JDBC avoids proxy latency (~1 ms) but cannot survive serverless horizontal scaling. |
Decompose the 3.4s
sandbox init + JVM startup + Spring classpath scanning + DB TLS handshake.
Step 2
Multiply 400 sandboxes * 5 default Hikari connections = 2,000 connection attempts > 500 Postgres limit.
Step 3
Show how failed requests trigger client retries, spawning even more cold starts.
Alternative 1
HTTP-based serverless data APIs (higher per-query latency).
Failure Mode 1
Connection exhaustion cascades into 500 errors even on already-warm instances.
# Broken serverless DB config
spring:
datasource:
url: jdbc:postgresql://prod-rds.internal:5432/checkout
hikari:
maximum-pool-size: 10 # 100 Lambdas = 1,000 connections -> Postgres OOM/reject
Combines Lambda SnapStart + CRaC.
Combines Lambda SnapStart + CRaC lifecycle hooks, RDS Proxy with Hikari max=1, and Cloud Run concurrency=80 with startup CPU boost.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | SnapStart adds ~15s to CI/CD publish time to capture the microVM snapshot. |
Step 1
Enable SnapStart on PublishedVersions in SAM/Terraform.
Step 2
Implement CRaC Resource beforeCheckpoint/afterRestore to manage sockets safely.
Step 3
Point Spring Boot datasource to RDS Proxy with maximum-pool-size: 1.
Alternative 1
GraalVM Native Image (~80 ms cold start, but breaks some Spring reflection libraries).
Failure Mode 1
Forgetting to register CRaC hooks leaves closed/stale socket handles in the restored heap.
@Component
public class DbSnapshotHook implements org.crac.Resource {
private final HikariDataSource dataSource;
public DbSnapshotHook(HikariDataSource ds) {
this.dataSource = ds;
Core.getGlobalContext().register(this);
}
@Override public void beforeCheckpoint(Context<? extends Resource> ctx) {
dataSource.evictConnection(null); // Flush idle sockets before snapshot
}
@Override public void afterRestore(Context<? extends Resource> ctx) {
// Connections lazily re-established to RDS Proxy on first query
}
}
Lambda isolates 1 request per.
Lambda isolates 1 request per microVM, whereas Cloud Run routes up to N concurrent HTTP requests into a single multi-threaded container.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Higher concurrency slashes cold starts and cost, but a single OOM crash affects up to 80 in-flight requests. |
Step 1
Show that 3,000 RPS at 50 ms requires 150 concurrency: 150 Lambda sandboxes vs only 2–3 Cloud Run containers.
Step 2
Explain --cpu-boost during Cloud Run startup to halve JVM classloading time.
Step 3
Note thread-safety and heap sizing requirements for 80 concurrent Spring MVC/WebFlux threads.
Alternative 1
Lower concurrency (e.g., 20) for CPU-bound workloads to avoid thread contention.
Failure Mode 1
ThreadLocal leaks in multi-concurrent containers causing cross-request data corruption.
# Cloud Run service spec
spec:
template:
metadata:
annotations:
autoscaling.knative.dev/minScale: "3"
run.googleapis.com/startup-cpu-boost: "true"
spec:
containerConcurrency: 80
containers:
- resources:
limits: { cpu: "2", memory: "2Gi" }
RDS Proxy and PgBouncer in.
RDS Proxy and PgBouncer in transaction mode pin a client to a dedicated backend Postgres connection if session-scoped state is mutated.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Disabling server-side prepared statements adds minimal SQL parse CPU (~0.2 ms) but enables true 10:1 connection multiplexing. |
Identify common pinning triggers
prepared statements, SET commands, temporary tables, or advisory locks.
Step 2
Add prepareThreshold=0 to the PostgreSQL JDBC URL to disable server-side prepared statements.
Step 3
Move any session-level settings into transaction-local SET LOCAL.
Alternative 1
Using PostgreSQL 16+ / PgBouncer 1.21+ with protocol-level prepared statement tracking.
Pinning turns RDS Proxy into a 1
1 pass-through, re-exposing Postgres to connection exhaustion.
spring:
datasource:
url: jdbc:postgresql://checkout-proxy.proxy-xyz.us-east-1.rds.amazonaws.com:5432/checkout?sslmode=require&prepareThreshold=0
hikari:
maximum-pool-size: 1
connection-timeout: 1000
Track AWS Lambda InitDuration /.
Track AWS Lambda InitDuration / Cloud Run container_startup_latency alongside RDS Proxy DatabaseConnectionsPinned.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Synthetic burst tests incur brief staging compute cost but catch accidental SnapStart or DI regressions before production. |
Step 1
Alert if Lambda InitDuration p99 > 500 ms on published SnapStart aliases.
Step 2
Run a step-burst k6 test (150 -> 3,000 RPS in 10 s) in staging before major campaigns.
Step 3
Verify RDS Proxy ClientConnections vs DatabaseConnections ratio stays >= 4:1.
Alternative 1
Canary alias traffic shifting (5% for 15 min) with InitDuration rollback alarms.
Failure Mode 1
Deploying to $LATEST instead of a published version silently bypasses SnapStart.
-- CloudWatch Logs Insights query for cold start InitDuration
filter @type = "REPORT" and ispresent(@initDuration)
| stats pct(@initDuration, 50) as p50, pct(@initDuration, 99) as p99, count(*) as cold_starts by bin(5m)
Theory & Tips and Tricks
Practical heuristics & interview traps
- 01Interview anchor: Combine SnapStart / container image optimization with predictive Provisioned Concurrency (min-instances), lazy initialization, and an external connection pooler (RDS Proxy / PgBouncer).
- 02Trade-off tip: Higher concurrency slashes cold starts and cost, but a single OOM crash affects up to 80 in-flight requests.
- 03Watch out for: Pinning turns RDS Proxy into a 1:1 pass-through, re-exposing Postgres to connection exhaustion.
Putting Mitigating Serverless Cold-Start Latency Spikes Together
Combine SnapStart / container image optimization with predictive Provisioned Concurrency (min-instances), lazy initialization, and an external connection pooler (RDS Proxy / PgBouncer).
Putting Mitigating Serverless Cold-Start Latency Spikes Together
- Enable SnapStart on PublishedVersions in SAM/Terraform.
- Implement CRaC Resource beforeCheckpoint/afterRestore to manage sockets safely.
- Point Spring Boot datasource to RDS Proxy with maximum-pool-size: 1.
Interview Readiness Check
RDS Proxy and PgBouncer in transaction mode pin a client to a dedicated backend Postgres connection if session-scoped state is mutated.
Walk through bottlenecks, back-of-the-envelope math, and failure recovery.
Practice: Make the trade-off yourself.
Practice Check
Test your understanding
When is ACID Transaction Fundamentals the right default for a production team?
Mitigating Serverless Cold-Start Latency Spikes
Combine SnapStart / container image optimization with predictive Provisioned Concurrency (min-instances), lazy initialization, and an external connection pooler (RDS Proxy / PgBouncer).
