An in-process telemetry ingestion pipeline buffers parsed log events between 16 network-receiver producer threads (bursting up to 150,000 events/sec) and 8 disk/HTTP-shipper consumer threads. Previously, using an unbounded `LinkedBlockingQueue` caused 4 GB JVM OOM crashes during downstream outages; switching to a custom `synchronized` bounded queue introduced deadlocks (threads stuck waiting forever when space was available) and `IndexOutOfBoundsException` under load. Design and implement a thread-safe, fixed-capacity (`N`) bounded blocking queue supporting `put(item)`, `take()`, and timed `offer(item, timeout)` / `poll(timeout)` with zero busy-waiting, exact FIFO ordering, and clean interruption handling.
From patterns to decisions.
An in-process telemetry ingestion pipeline buffers parsed log events between 16 network-receiver producer threads (bursting up to 150,000 events/sec) and 8 disk/HTTP-shipper consumer threads. Previously, using an unbounded `LinkedBlockingQueue` caused 4 GB JVM OOM crashes during downstream outages; switching to a custom `synchronized` bounded queue introduced deadlocks (threads stuck waiting forever when space was available) and `IndexOutOfBoundsException` under load. Design and implement a thread-safe, fixed-capacity (`N`) bounded blocking queue supporting `put(item)`, `take()`, and timed `offer(item, timeout)` / `poll(timeout)` with zero busy-waiting, exact FIFO ordering, and clean interruption handling.
With a single monitor, notify().
With a single monitor, notify() can wake a thread of the same role (e.g., a consumer wakes another sleeping consumer when the queue is empty), leaving all producers asleep forever.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | `notifyAll()` fixes the missed signal on `synchronized` but causes a thundering herd of all waiting threads. |
Trace
Queue is full; P1 and P2 sleep. C1 takes 1 item and calls notify(), waking P1. Before P1 runs, P3 steals the slot and fills it. C2 takes 1 item? No, C2 was also sleeping—if P1 calls notify() it might wake P2 instead of C2.
Step 2
Show that if fails on spurious wakeups or stolen slots, causing count > cap or IndexOutOfBounds.
Alternative 1
Separate notFull and notEmpty Condition objects on ReentrantLock.
Failure Mode 1
Thundering herd wakeups degrading CPU when 16 producers wake simultaneously for 1 free slot.
// ANTI-PATTERN: single monitor + if + notify()
synchronized (this) {
if (count == items.length) wait(); // BUG 1: if instead of while
enqueue(item);
notify(); // BUG 2: may wake another producer instead of consumer!
}
A circular array with ReentrantLock.
A circular array with ReentrantLock, notFull, notEmpty, while-guarded await(), and awaitNanos for timed operations.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Non-fair `ReentrantLock` maximizes throughput at 150k ops/s; pass `true` if strict thread-wakeup fairness is required. |
Step 1
Validate capacity > 0 and reject null items.
Step 2
Implement enqueue(T x) and dequeue() helpers that wrap indices and signal the opposite Condition.
Step 3
Use awaitNanos(nanos) in a while loop for offer and poll.
Alternative 1
In production Java, use java.util.concurrent.ArrayBlockingQueue directly.
Failure Mode 1
Omitting items[takeIndex] = null leaks large telemetry objects until the ring buffer wraps around.
import java.util.Objects;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;
public class BoundedBlockingQueue<T> {
private final Object[] items;
private int putIndex, takeIndex, count;
private final ReentrantLock lock = new ReentrantLock();
private final Condition notEmpty = lock.newCondition();
private final Condition notFull = lock.newCondition();
public BoundedBlockingQueue(int capacity) {
if (capacity <= 0) throw new IllegalArgumentException("capacity must be > 0");
this.items = new Object[capacity];
}
public void put(T item) throws InterruptedException {
Objects.requireNonNull(item);
lock.lockInterruptibly();
try {
while (count == items.length) {
notFull.await();
}
enqueue(item);
} finally {
lock.unlock();
}
}
public T take() throws InterruptedException {
lock.lockInterruptibly();
try {
while (count == 0) {
notEmpty.await();
}
return dequeue();
} finally {
lock.unlock();
}
}
public boolean offer(T item, long timeout, TimeUnit unit) throws InterruptedException {
Objects.requireNonNull(item);
long nanos = unit.toNanos(timeout);
lock.lockInterruptibly();
try {
while (count == items.length) {
if (nanos <= 0L) return false;
nanos = notFull.awaitNanos(nanos);
}
enqueue(item);
return true;
} finally {
lock.unlock();
}
}
private void enqueue(T item) {
items[putIndex] = item;
if (++putIndex == items.length) putIndex = 0;
count++;
notEmpty.signal();
}
@SuppressWarnings("unchecked")
private T dequeue() {
T item = (T) items[takeIndex];
items[takeIndex] = null; // prevent GC retention
if (++takeIndex == items.length) takeIndex = 0;
count--;
notFull.signal();
return item;
}
}
Split into putLock and takeLock.
Split into putLock and takeLock with an AtomicInteger count (the LinkedBlockingQueue design), or use batch-drain (drainTo) so consumers acquire the lock once per 64 events.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Two-lock linked queues allocate a `Node` per item (more GC), whereas `drainTo` on a ring buffer keeps zero-GC allocation. |
Step 1
Two-lock splitting lets a producer enqueue while a consumer dequeues in parallel.
Step 2
Alternatively, drainTo(batch, 64) amortizes lock acquisition by 64x with zero extra node allocations.
Alternative 1
LMAX Disruptor (lock-free CAS ring buffer) for >1M ops/sec.
Failure Mode 1
Deadlock if any method ever acquires both putLock and takeLock in inconsistent order.
# Batch draining in Python consumer reduces lock acquisitions by 64x
def drain_batch(queue, max_items: int = 64) -> list:
batch = [queue.get()] # block for at least 1 item
while len(batch) < max_items and not queue.empty():
try:
batch.append(queue.get_nowait())
except Exception:
break
return batch
Blocking the 16 receiver threads.
Blocking the 16 receiver threads applies natural TCP backpressure (TCP receive window shrinks to 0), slowing upstream senders instead of crashing the JVM—unless receivers share threads with health checks.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Blocking `put()` preserves zero data loss until TCP timeouts fire; `offer()` + 429 protects pod liveness and bounds tail latency. |
Step 1
Keep health-check endpoints (/healthz) on a separate thread pool from the 16 blocked receiver threads so Kubernetes doesn't restart the pod.
Step 2
Alternatively, use offer(item, 50ms) and return 429 Too Many Requests / drop-oldest if real-time shedding is preferred over TCP backpressure.
Alternative 1
Spill overflow to a local disk WAL (e.g., Chronicle Queue / RocksDB).
Failure Mode 1
Blocking Netty/Reactor event-loop threads with put() freezes all network I/O on that event loop.
// Fail-fast backpressure with 429 shedding
if (!queue.offer(event, 50, TimeUnit.MILLISECONDS)) {
metrics.counter("telemetry.dropped.backpressure").increment();
return Response.status(429).build();
}
Python's GIL serializes bytecode execution.
Python's GIL serializes bytecode execution across threads, making queue.Queue(maxsize=N) effective for I/O-bound workers but CPU-limited; for multi-core Python parallelism, use multiprocessing.Queue or asyncio.Queue(maxsize=N).
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | `asyncio.Queue` has zero OS thread-switching cost, but is single-threaded and cannot survive blocking C extensions. |
Step 1
Explain that queue.Queue internally uses threading.Lock + not_empty/not_full Conditions (same pattern).
Step 2
For async event loops, use asyncio.Queue(maxsize=N) (await q.put(), await q.get()) so coroutines yield without blocking the event loop.
Alternative 1
multiprocessing.Queue or concurrent.futures.ProcessPoolExecutor for CPU-heavy parsing.
Failure Mode 1
Calling threading queue.Queue.get() inside an asyncio coroutine freezes the entire event loop.
import asyncio
async def run_pipeline():
q = asyncio.Queue(maxsize=65536)
await q.put({"event": "login"})
item = await q.get()
q.task_done()
Theory & Tips and Tricks
Practical heuristics & interview traps
- 01Interview anchor: Implement a fixed-capacity circular ring buffer protected by a
ReentrantLockwith separatenotFullandnotEmptyConditionqueues (or two-lock split), always guardingawait()inside awhileloop. - 02Trade-off tip: Two-lock linked queues allocate a
Nodeper item (more GC), whereasdrainToon a ring buffer keeps zero-GC allocation. - 03Watch out for: Blocking Netty/Reactor event-loop threads with
put()freezes all network I/O on that event loop.
Implement Bounded Blocking Queue — Normal → Load / Fault → Recovery
Implement a fixed-capacity circular ring buffer protected by a ReentrantLock with separate notFull and notEmpty Condition queues (or two-lock split), always guarding await() inside a while loop.
Putting Implement a Thread-Safe Bounded Blocking Queue Together
Implement a fixed-capacity circular ring buffer protected by a `ReentrantLock` with separate `notFull` and `notEmpty` `Condition` queues (or two-lock split), always guarding `await()` inside a `while` loop.
Putting Implement a Thread-Safe Bounded Blocking Queue Together
- Validate
capacity > 0and rejectnullitems. - Implement
enqueue(T x)anddequeue()helpers that wrap indices and signal the oppositeCondition. - Use
awaitNanos(nanos)in awhileloop forofferandpoll.
Interview Readiness Check
Blocking the 16 receiver threads applies natural TCP backpressure (TCP receive window shrinks to 0), slowing upstream senders instead of crashing the JVM—unless receivers share threads with health checks.
Walk through bottlenecks, back-of-the-envelope math, and failure recovery.
Make the trade-off yourself.
Practice Check
Test your understanding
When is ACID Transaction Fundamentals the right default for a production team?
Implement a Thread-Safe Bounded Blocking Queue
Implement a fixed-capacity circular ring buffer protected by a ReentrantLock with separate notFull and notEmpty Condition queues (or two-lock split), always guarding await() inside a while loop.
