Adapting to Virtual Threads for Spring Developers

Scenario context

Platform threads (one OS thread per request) max out at a few hundred concurrent I/O-bound requests. Virtual threads (JDK 21+) allow millions — with zero I/O-blocking cost. Spring Boot 3.2 enables them with a single property. Avoid synchronized blocks wrapping I/O (they pin virtual threads to OS threads) and CPU-bound work inside virtual threads. For I/O-heavy Spring services, virtual threads are the most impactfu…

This walkthrough focuses on Why Platform Threads Break Under Concurrent I/O Load.

Before making a design choice, clarify:

• Every thread managed by Tomcat, @Async , or a raw ExecutorService in classic Spring MVC maps 1:1 to an OS thread.
• OS threads are expensive along three dimensions: - Stack memory: ~1 MB of stack space reserved per thread, regardless of whether the thread is doing work or waiting on I/O - Context-switch overhead: When a thread blocks…
• Increase it to 1,000 and your JVM reserves ~1 GB in thread stacks alone.

Concepts in play: jvm-internals-and-garbage-collection, tree-data-structures, stack-and-queue-data-structures

Stage 1 — When 2,000 Users Break a Service That Handled 200 Fine

• Your Spring Boot service is returning HTTP 503s.
• You open Grafana: CPU at 22%, heap at 40%, memory healthy.
• Then you check the thread monitor: the Tomcat thread pool is at 200/200 active threads.
• Thousands of requests are queuing and timing out.

Observable outcome: Thousands of requests are queuing and timing out.

Stage 2 — Virtual Threads: JVM-Scheduled, Not OS-Scheduled

• A virtual thread is a thread managed by the JVM rather than the OS.
• Internally, the JVM runs a small fixed pool of carrier threads — one OS thread per CPU core.
• Virtual threads are mounted onto a carrier thread when they have CPU work to perform.
• The moment a virtual thread hits a blocking I/O call, the JVM unmounts it from the carrier, suspends its stack in heap memory, and frees the carrier for another virtual thread that has work ready.

Observable outcome: The moment a virtual thread hits a blocking I/O call, the JVM unmounts it from the carrier, suspends its stack in heap memory, and frees the carrier for another virtual thread that has work ready.

Stage 3 — How Virtual Thread Scheduling Differs from Platform Threads

• The diagram below shows why the platform thread model exhausts at 200 concurrent requests while the virtual thread model scales to millions.
• Read each subgraph left-to-right: in the platform model, every request arrow terminates at a blocked OS thread; in the virtual model, all virtual threads are suspended in heap while two carrier threads handle all active…
• In the platform thread model, each of the 200 threads is blocked and idle — consuming memory and OS slots while waiting on I/O.
• Requests 201–300 cannot be served.

flowchart TD
    subgraph PlatformModel[Platform Thread Model - 300 concurrent requests]
        R1[Request 1] --> T1[OS Thread 1 - blocked on JDBC]
        R2[Request 2] --> T2[OS Thread 2 - blocked on HTTP call]
        R3[Request 3] --> T3[OS Thread 3 - blocked on file I/O]
        R200[Request 200] --> T200[OS Thread 200 - blocked on JDBC]
        R201[Request 201] --> Q1[Queued - thread pool exhausted]
        R300[Request 300] --> Q2[Queued - timeout approaching]
    end
    subgraph VirtualModel[Virtual Thread Model - 300 concurrent requests]
        VR1[Request 1] --> VT1[Virtual Thread 1 - suspended in heap]
        VR2[Request 2] --> VT2[Virtual Thread 2 - suspended in heap]
        VR300[Request 300] --> VT300[Virtual Thread 300 - suspended in heap]
        CT1[Carrier Thread 1 - 1 per CPU core] -.->|mounts VT with active work| VT1
        CT2[Carrier Thread 2 - 1 per CPU core] -.->|mounts VT with active work| VT2
    end

Observable outcome: Requests 201–300 cannot be served.

Stage 4 — Worked Example: Thread Pool Exhaustion Before and After Virtual Threads

• To see the benefit concretely, contrast the same OrderService in both models.
• The code is identical — only the threading model beneath it changes.
• With 200 threads and a 60ms average request latency, theoretical throughput is 200 / 0.060 ≈ 3,300 req/s .
• In practice, the moment you have 201 concurrent long-running requests, the 201st queues.

// application.properties — classic Tomcat platform thread configuration
server.tomcat.threads.max=200          // hard ceiling: 200 concurrent blocking requests
server.tomcat.threads.min-spare=10
server.tomcat.accept-count=100         // queue depth before rejecting new connections

// Before: OrderService — each method call holds a platform thread for its entire duration
@Service
public class OrderService {

    private final RestTemplate restTemplate;
    private final JdbcTemplate jdbcTemplate;

    public OrderService(RestTemplate restTemplate, JdbcTemplate jdbcTemplate) {
        this.restTemplate = restTemplate;
        this.jdbcTemplate = jdbcTemplate;
    }

    // PROBLEM: This method occupies platform thread #47 for the full ~60ms.
    // During the 10ms JDBC wait and the 50ms HTTP wait, thread #47 sits idle
    // in SocketInputStream.read() — holding 1MB of stack and an OS slot.
    // At 200 concurrent calls, the pool is exhausted regardless of CPU load.
    public Order processOrder(String orderId) {

        // Blocks platform thread ~10ms waiting for DB network round-trip
        Order order = jdbcTemplate.queryForObject(
            "SELECT * FROM orders WHERE id = ?",
            new Object[]{orderId},
            orderRowMapper
        );

        // Blocks platform thread ~50ms waiting for external HTTP response
        String enrichment = restTemplate.getForObject(
            "https://inventory-service/api/items/" + order.getItemId(),
            String.class
        );

        order.setEnrichment(enrichment);
        return order;
    }
}

// Before: Controller — Tomcat dispatches a platform thread per HTTP request.
// That thread is held until processOrder() returns (~60ms total).
@RestController
@RequestMapping("/orders")
public class OrderController {

    private final OrderService orderService;

    public OrderController(OrderService orderService) {
        this.orderService = orderService;
    }

    @GetMapping("/{orderId}")
    public ResponseEntity<Order> getOrder(@PathVariable String orderId) {
        return ResponseEntity.ok(orderService.processOrder(orderId));
    }
}

Observable outcome: In practice, the moment you have 201 concurrent long-running requests, the 201st queues.

Stage 5 — Enabling Virtual Threads in Spring Boot 3.2

• Spring Boot 3.2 wires virtual threads automatically when this property is set and JDK 21+ is detected: Under the hood, this replaces Tomcat's fixed thread pool executor with one that creates a new virtual thread per inc…
• There is no thread pool ceiling.
• The OrderService code is unchanged — the same blocking JDBC call, the same RestTemplate call — they now run on virtual threads.
• The JVM unmounts the virtual thread during each I/O wait, freeing carrier threads for other requests.

# application.yml — enable virtual threads (Spring Boot 3.2+, JDK 21+ required)
spring:
  threads:
    virtual:
      enabled: true

// Explicit programmatic virtual thread configuration
@Configuration
public class VirtualThreadConfig {

    // Replaces Tomcat's fixed thread pool with a virtual-thread-per-task executor.
    // Tomcat now creates one virtual thread per incoming HTTP request.
    @Bean
    public TomcatProtocolHandlerCustomizer<?> virtualThreadTomcatCustomizer() {
        return protocolHandler ->
            protocolHandler.setExecutor(
                Executors.newVirtualThreadPerTaskExecutor()  // JDK 21 API
            );
    }

    // Replace the @Async executor so background tasks also run on virtual threads.
    // SimpleAsyncTaskExecutor with virtualThreads=true is equivalent but
    // the explicit executor gives you more visibility in monitoring tools.
    @Bean
    @Primary
    public Executor virtualThreadAsyncExecutor() {
        return Executors.newVirtualThreadPerTaskExecutor();
    }
}

Observable outcome: The JVM unmounts the virtual thread during each I/O wait, freeing carrier threads for other requests.

Stage 6 — Under the Hood: How the JVM Parks and Resumes Virtual Threads

• Understanding the internals helps you reason about when virtual threads help, when they hurt, and why the pinning problem exists.
• Virtual Thread Internals: Mounting, Unmounting, and Carrier Threads When jdbcTemplate.queryForObject(...) sends a SQL query over a socket, the JVM intercepts the blocking I/O call and executes this lifecycle: 1.
• The virtual thread calls SocketInputStream.read() 2.
• The JVM detects this is a blocking I/O syscall via its internal socket implementation 3.

// DANGEROUS: synchronized + I/O inside the lock = virtual thread pins its carrier
@Service
public class CachedOrderService {

    private final Map<String, Order> cache = new HashMap<>();
    private final JdbcTemplate jdbcTemplate;

    public CachedOrderService(JdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }

    // BAD: synchronized holds the carrier thread for the full JDBC call duration.
    // With 8 CPU cores (8 carrier threads), only 8 concurrent cached lookups
    // can proceed — far worse than the 200-thread platform model.
    public synchronized Order getOrLoad(String orderId) {
        if (cache.containsKey(orderId)) {
            return cache.get(orderId);
        }
        // This JDBC call (10ms+) runs while synchronized holds the carrier thread pinned.
        Order order = jdbcTemplate.queryForObject(
            "SELECT * FROM orders WHERE id = ?",
            new Object[]{orderId}, orderRowMapper
        );
        cache.put(orderId, order);
        return order;
    }
}

// CORRECT: ReentrantLock allows the virtual thread to unmount during lock wait.
// When another virtual thread holds the lock, this one parks in heap — not the carrier.
@Service
public class CachedOrderServiceFixed {

    private final Map<String, Order> cache = new HashMap<>();
    private final ReentrantLock lock = new ReentrantLock();
    private final JdbcTemplate jdbcTemplate;

    public CachedOrderServiceFixed(JdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }

    // GOOD: lockInterruptibly() allows the JVM to park the virtual thread if the lock
    // is held — it unmounts from the carrier and frees it for other virtual threads.
    public Order getOrLoad(String orderId) throws InterruptedException {
        lock.lockInterruptibly();
        try {
            if (cache.containsKey(orderId)) {
                return cache.get(orderId);
            }
            Order order = jdbcTemplate.queryForObject(
                "SELECT * FROM orders WHERE id = ?",
                new Object[]{orderId}, orderRowMapper
            );
            cache.put(orderId, order);
            return order;
        } finally {
            lock.unlock();
        }
    }
}

java -Djdk.tracePinnedThreads=full -jar your-app.jar

Observable outcome: The JVM detects this is a blocking I/O syscall via its internal socket implementation 3.

End-to-end outcome

After working through the stages, you should be able to explain:

• • One property, measurable impact.
• Enabling virtual threads in a Spring Boot 3.2 service handling 5,000 concurrent users dropped p99 latency by 40% with zero code change in the business logic.

Alternatives considered

Spring Migration Decision Guide

Spring Migration Decision Guide

Failure and degraded modes

Trade-offs: I/O-Bound Gains vs CPU-Bound Limits

• Detection: Virtual threads are a solution for I/O-bound concurrency.
• User or operator impact: Virtual threads are a solution for I/O-bound concurrency.