Concurrency & async models
Core details
Compare models you can name in-room:
| Model | Strength | Pitfall |
|---|---|---|
| Thread-per-request | isolation | pool exhaustion under load |
| Event loop / async I/O | high concurrency cheap waits | accidental CPU block (sync FS, heavy regex) |
| Actor / mailbox | ordered per entity | mailbox backlog unbounded if no backpressure |
| Worker pools + queues | smoothing bursts | queue growth if consumers slower than producers |
Backpressure manifests as bounded queue depth shedding load, 429/503 with Retry-After, or slowing producers—pick intentionally with SLO alignment.
Problem this solves: backend services receive more concurrent work than a single CPU, thread pool, event loop, database pool, or queue can process at once.
Simple mental model: concurrency is not capacity. Capacity is the smallest saturated resource on the request path.
Understanding
Async shines when work is waiting-dominated (I/O). It fails when CPU-bound tasks monopolize cooperative schedulers—moving CPU work to pools/child processes preserves latency for I/O handlers. Unbounded queues pretend infinite capacity—real systems need visible saturation metrics.
Use this quick diagnosis:
| Symptom | First suspect | Measurement |
|---|---|---|
| Fast DB query, slow API | pool wait or event loop lag | pool wait histogram, loop delay |
| CPU high, I/O low | synchronous computation | CPU profile, flame graph |
| Queue grows during spikes | consumers slower than producers | queue depth, age of oldest message |
| One tenant hurts others | missing bulkhead/fairness | per-tenant rate, queue, and latency |
Senior understanding
Demonstrate reading pool wait time and event-loop lag before blaming “the database is slow.” Mention bulkhead patterns isolating noisy neighbors (reporting vs payments). For Node specifically, cross-link /backend/nodejs/fundamentals/event-loop, but keep this page model-agnostic patterns crisp.
Decision rules
| Work type | Good default | Avoid |
|---|---|---|
| Waiting on network/database | async I/O with bounded pools | one thread per slow client without limits |
| CPU-heavy image/PDF/crypto work | worker pool, process pool, or specialized service | blocking request handler |
| Per-account ordered work | actor/mailbox or keyed partition | global queue with accidental reordering |
| Burst absorption | bounded queue + backpressure | unbounded queue that hides overload |
Interview answer structure
“I identify whether the work is waiting-bound or CPU-bound, then choose the scheduling model. Async I/O helps with many waits, but it does not create more CPU. I bound pools and queues, expose saturation metrics, and add backpressure before the dependency becomes the failure amplifier.”
Follow-ups to expect:
- How do you detect event-loop starvation?
- What happens if every request starts ten parallel downstream calls?
- How do you isolate reporting jobs from payment traffic?
- When would you choose workers over a separate service?
Diagram
See also
Mark this page when you finish learning it.
Spotted something unclear or wrong on this page?