Skip to main content
← PROBLEMS / CONCURRENCY
An end-to-end domain learning journey

Implement a Thread-Safe
Bounded Blocking Queue.

An in-process telemetry ingestion pipeline buffers parsed log events between 16 network-receiver producer threads (bursting up to 150,000 events/sec) and 8 disk/HTTP-shipper consumer threads. Previously, using an unbounded LinkedBlockingQueue caused 4 GB JVM OOM crashes during downstream outages; switching to a custom synchronized bounded queue introduced deadlocks (threads stuck waiting forever when space was available) and IndexOutOfBoundsException under load. Design and implement a thread-safe, fixed-capacity (N) bounded blocking queue supporting put(item), take(), and timed offer(item, timeout) / poll(timeout) with zero busy-waiting, exact FIFO ordering, and clean interruption handling.
Start the journey5 stages
Loading diagram…
Implement a Thread-Safe Bounded Blocking Queue architecture sketch diagram
FIRST PRINCIPLES. PRODUCTION SCALE.
The idea to keep with you

An in-process telemetry ingestion pipeline buffers parsed log events between 16 network-receiver producer threads (bursting up to 150,000 events/sec) and 8 disk/HTTP-shipper consumer threads. Previously, using an unbounded `LinkedBlockingQueue` caused 4 GB JVM OOM crashes during downstream outages; switching to a custom `synchronized` bounded queue introduced deadlocks (threads stuck waiting forever when space was available) and `IndexOutOfBoundsException` under load. Design and implement a thread-safe, fixed-capacity (`N`) bounded blocking queue supporting `put(item)`, `take()`, and timed `offer(item, timeout)` / `poll(timeout)` with zero busy-waiting, exact FIFO ordering, and clean interruption handling.

From patterns to decisions.

An in-process telemetry ingestion pipeline buffers parsed log events between 16 network-receiver producer threads (bursting up to 150,000 events/sec) and 8 disk/HTTP-shipper consumer threads. Previously, using an unbounded `LinkedBlockingQueue` caused 4 GB JVM OOM crashes during downstream outages; switching to a custom `synchronized` bounded queue introduced deadlocks (threads stuck waiting forever when space was available) and `IndexOutOfBoundsException` under load. Design and implement a thread-safe, fixed-capacity (`N`) bounded blocking queue supporting `put(item)`, `take()`, and timed `offer(item, timeout)` / `poll(timeout)` with zero busy-waiting, exact FIFO ordering, and clean interruption handling.

01
CONCURRENCY · STAGE 01L4 · FUNDAMENTALS

With a single monitor, notify().

With a single monitor, notify() can wake a thread of the same role (e.g., a consumer wakes another sleeping consumer when the queue is empty), leaving all producers asleep forever.

ChoiceWhat it buysWhat it costs
Trade-off 1`notifyAll()` fixes the missed signal on `synchronized` but causes a thundering herd of all waiting threads.

Trace

Queue is full; P1 and P2 sleep. C1 takes 1 item and calls notify(), waking P1. Before P1 runs, P3 steals the slot and fills it. C2 takes 1 item? No, C2 was also sleeping—if P1 calls notify() it might wake P2 instead of C2.

Step 2

Show that if fails on spurious wakeups or stolen slots, causing count > cap or IndexOutOfBounds.

Alternative 1

Separate notFull and notEmpty Condition objects on ReentrantLock.

Failure Mode 1

Thundering herd wakeups degrading CPU when 16 producers wake simultaneously for 1 free slot.

// ANTI-PATTERN: single monitor + if + notify()
synchronized (this) {
    if (count == items.length) wait(); // BUG 1: if instead of while
    enqueue(item);
    notify(); // BUG 2: may wake another producer instead of consumer!
}
02
CONCURRENCY · STAGE 02L5 · APPLIED DESIGN

A circular array with ReentrantLock.

A circular array with ReentrantLock, notFull, notEmpty, while-guarded await(), and awaitNanos for timed operations.

ChoiceWhat it buysWhat it costs
Trade-off 1Non-fair `ReentrantLock` maximizes throughput at 150k ops/s; pass `true` if strict thread-wakeup fairness is required.

Step 1

Validate capacity > 0 and reject null items.

Step 2

Implement enqueue(T x) and dequeue() helpers that wrap indices and signal the opposite Condition.

Step 3

Use awaitNanos(nanos) in a while loop for offer and poll.

Alternative 1

In production Java, use java.util.concurrent.ArrayBlockingQueue directly.

Failure Mode 1

Omitting items[takeIndex] = null leaks large telemetry objects until the ring buffer wraps around.

import java.util.Objects;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;

public class BoundedBlockingQueue<T> {
    private final Object[] items;
    private int putIndex, takeIndex, count;
    private final ReentrantLock lock = new ReentrantLock();
    private final Condition notEmpty = lock.newCondition();
    private final Condition notFull = lock.newCondition();

    public BoundedBlockingQueue(int capacity) {
        if (capacity <= 0) throw new IllegalArgumentException("capacity must be > 0");
        this.items = new Object[capacity];
    }

    public void put(T item) throws InterruptedException {
        Objects.requireNonNull(item);
        lock.lockInterruptibly();
        try {
            while (count == items.length) {
                notFull.await();
            }
            enqueue(item);
        } finally {
            lock.unlock();
        }
    }

    public T take() throws InterruptedException {
        lock.lockInterruptibly();
        try {
            while (count == 0) {
                notEmpty.await();
            }
            return dequeue();
        } finally {
            lock.unlock();
        }
    }

    public boolean offer(T item, long timeout, TimeUnit unit) throws InterruptedException {
        Objects.requireNonNull(item);
        long nanos = unit.toNanos(timeout);
        lock.lockInterruptibly();
        try {
            while (count == items.length) {
                if (nanos <= 0L) return false;
                nanos = notFull.awaitNanos(nanos);
            }
            enqueue(item);
            return true;
        } finally {
            lock.unlock();
        }
    }

    private void enqueue(T item) {
        items[putIndex] = item;
        if (++putIndex == items.length) putIndex = 0;
        count++;
        notEmpty.signal();
    }

    @SuppressWarnings("unchecked")
    private T dequeue() {
        T item = (T) items[takeIndex];
        items[takeIndex] = null; // prevent GC retention
        if (++takeIndex == items.length) takeIndex = 0;
        count--;
        notFull.signal();
        return item;
    }
}
03
CONCURRENCY · STAGE 03L6 · SCALE & TRADE-OFFS

Split into putLock and takeLock.

Split into putLock and takeLock with an AtomicInteger count (the LinkedBlockingQueue design), or use batch-drain (drainTo) so consumers acquire the lock once per 64 events.

ChoiceWhat it buysWhat it costs
Trade-off 1Two-lock linked queues allocate a `Node` per item (more GC), whereas `drainTo` on a ring buffer keeps zero-GC allocation.

Step 1

Two-lock splitting lets a producer enqueue while a consumer dequeues in parallel.

Step 2

Alternatively, drainTo(batch, 64) amortizes lock acquisition by 64x with zero extra node allocations.

Alternative 1

LMAX Disruptor (lock-free CAS ring buffer) for >1M ops/sec.

Failure Mode 1

Deadlock if any method ever acquires both putLock and takeLock in inconsistent order.

# Batch draining in Python consumer reduces lock acquisitions by 64x
def drain_batch(queue, max_items: int = 64) -> list:
    batch = [queue.get()]  # block for at least 1 item
    while len(batch) < max_items and not queue.empty():
        try:
            batch.append(queue.get_nowait())
        except Exception:
            break
    return batch
04
CONCURRENCY · STAGE 04L6 · SCALE & TRADE-OFFS

Blocking the 16 receiver threads.

Blocking the 16 receiver threads applies natural TCP backpressure (TCP receive window shrinks to 0), slowing upstream senders instead of crashing the JVM—unless receivers share threads with health checks.

ChoiceWhat it buysWhat it costs
Trade-off 1Blocking `put()` preserves zero data loss until TCP timeouts fire; `offer()` + 429 protects pod liveness and bounds tail latency.

Step 1

Keep health-check endpoints (/healthz) on a separate thread pool from the 16 blocked receiver threads so Kubernetes doesn't restart the pod.

Step 2

Alternatively, use offer(item, 50ms) and return 429 Too Many Requests / drop-oldest if real-time shedding is preferred over TCP backpressure.

Alternative 1

Spill overflow to a local disk WAL (e.g., Chronicle Queue / RocksDB).

Failure Mode 1

Blocking Netty/Reactor event-loop threads with put() freezes all network I/O on that event loop.

// Fail-fast backpressure with 429 shedding
if (!queue.offer(event, 50, TimeUnit.MILLISECONDS)) {
    metrics.counter("telemetry.dropped.backpressure").increment();
    return Response.status(429).build();
}
05
CONCURRENCY · STAGE 05L7 · STAFF / PRINCIPAL

Python's GIL serializes bytecode execution.

Python's GIL serializes bytecode execution across threads, making queue.Queue(maxsize=N) effective for I/O-bound workers but CPU-limited; for multi-core Python parallelism, use multiprocessing.Queue or asyncio.Queue(maxsize=N).

ChoiceWhat it buysWhat it costs
Trade-off 1`asyncio.Queue` has zero OS thread-switching cost, but is single-threaded and cannot survive blocking C extensions.

Step 1

Explain that queue.Queue internally uses threading.Lock + not_empty/not_full Conditions (same pattern).

Step 2

For async event loops, use asyncio.Queue(maxsize=N) (await q.put(), await q.get()) so coroutines yield without blocking the event loop.

Alternative 1

multiprocessing.Queue or concurrent.futures.ProcessPoolExecutor for CPU-heavy parsing.

Failure Mode 1

Calling threading queue.Queue.get() inside an asyncio coroutine freezes the entire event loop.

import asyncio

async def run_pipeline():
    q = asyncio.Queue(maxsize=65536)
    await q.put({"event": "login"})
    item = await q.get()
    q.task_done()

Theory & Tips and Tricks

Practical heuristics & interview traps

  • 01Interview anchor: Implement a fixed-capacity circular ring buffer protected by a ReentrantLock with separate notFull and notEmpty Condition queues (or two-lock split), always guarding await() inside a while loop.
  • 02Trade-off tip: Two-lock linked queues allocate a Node per item (more GC), whereas drainTo on a ring buffer keeps zero-GC allocation.
  • 03Watch out for: Blocking Netty/Reactor event-loop threads with put() freezes all network I/O on that event loop.
Behavioral Progression · Implement Bounded Blocking Queue

Implement Bounded Blocking Queue — Normal → Load / Fault → Recovery

Send Request
1. Normal Request Ingress
Traffic: 1x baseline · Latency (p99): 4 ms
Process State
2. Core Execution & State Lookup
Traffic: 1x baseline · Latency (p99): 9 ms
Inject 10x Spike
3. Scale Spike / Failure Injection
Traffic: 10x burst · Latency (p99): 185 ms

Implement a fixed-capacity circular ring buffer protected by a ReentrantLock with separate notFull and notEmpty Condition queues (or two-lock split), always guarding await() inside a while loop.

Synthesis / One connected design

Putting Implement a Thread-Safe Bounded Blocking Queue Together

Implement a fixed-capacity circular ring buffer protected by a `ReentrantLock` with separate `notFull` and `notEmpty` `Condition` queues (or two-lock split), always guarding `await()` inside a `while` loop.

A candidate architecture, not a universal answer

Putting Implement a Thread-Safe Bounded Blocking Queue Together

Step 1
Step 2
Step 3
  1. Validate capacity > 0 and reject null items.
  2. Implement enqueue(T x) and dequeue() helpers that wrap indices and signal the opposite Condition.
  3. Use awaitNanos(nanos) in a while loop for offer and poll.
Blocking the 16 receiver threads applies natural TCP backpressure (TCP receive window shrinks to 0), slowing upstream senders instead of crashing the JVM—unless receivers share threads with health checks.
Interview Readiness Check

Blocking the 16 receiver threads applies natural TCP backpressure (TCP receive window shrinks to 0), slowing upstream senders instead of crashing the JVM—unless receivers share threads with health checks.

Walk through bottlenecks, back-of-the-envelope math, and failure recovery.

Practice / Explain the consequences

Make the trade-off yourself.

Practice Check

Test your understanding

Question 1 of 3

When is ACID Transaction Fundamentals the right default for a production team?

Key takeaway

Implement a Thread-Safe Bounded Blocking Queue

Implement a fixed-capacity circular ring buffer protected by a ReentrantLock with separate notFull and notEmpty Condition queues (or two-lock split), always guarding await() inside a while loop.