Python asyncio: One Blocking Call Freezes Every Request

Python · Intermediate · 6 min read · published 2026-08-12

What this solves: Your async FastAPI/aiohttp service has fast endpoints that suddenly show 2-second p99 latency and failing health checks, even though CPU is at 15%. One synchronous call buried in a handler is stalling the entire event loop.

The Problem

A FastAPI service handles 400 rps with a p50 of 6 ms. Someone adds an endpoint that resizes an uploaded avatar with Pillow — 300 ms of CPU per call, and it gets called maybe 3 times a second. Nobody expects trouble at 1% of throughput.

Within an hour: p99 on the login endpoint goes from 20 ms to 1.8 s. Health checks flap. The load balancer starts ejecting instances. CPU utilization on the box: 14%.

Nothing is saturated. Yet every request is queueing behind a handler it has nothing to do with.

Why the Obvious Fix Falls Short

The first instinct is "make it concurrent" — wrap it in a task:

asyncio.create_task(resize(image))  # feels like it runs in the background

This does nothing. create_task schedules a coroutine, but a coroutine only ever yields control at an await on something that actually suspends. resize() has no such point: once the loop starts running it, it runs to completion. create_task moved when the block happens, not whether it blocks.

The second instinct is to scale out: more workers, more pods. This helps statistically — with 8 uvicorn workers, only 1/8 of requests land on the stalled loop at any moment. But you've turned a systemic bug into an intermittent one, and it comes right back when traffic to the slow endpoint grows. You're also now paying 8× memory to work around a 300 ms function.

The third instinct — sprinkling await asyncio.sleep(0) around the call — yields before and after the block, never during it. The 300 ms is atomic from the loop's point of view.

The real misconception: developers read "async" as "parallel." asyncio is cooperative concurrency on one thread. A handler that doesn't cooperate isn't slow — it's exclusive.

How It Actually Works

The event loop is a single thread running a queue of ready callbacks. Each callback runs to completion. Between callbacks, the loop polls the OS (epoll/kqueue) for sockets that became readable and schedules their callbacks.

If one callback takes 300 ms, the loop doesn't poll for 300 ms. Sockets with data sit unread. New connections sit unaccepted in the kernel backlog. Timers fire late. Latency you measure at the client is mostly queue time, which is why CPU looks idle.

sequenceDiagram
    participant K as Kernel (epoll)
    participant L as Event Loop (1 thread)
    participant H as /healthz callback
    participant R as /resize callback
    participant P as Thread/Process Pool
    K->>L: sockets ready: /resize, /healthz
    L->>R: run resize callback
    Note over L,R: 300ms, no await inside<br/>loop cannot poll or run anything
    K--xL: /healthz data waiting (ignored)
    R-->>L: done
    L->>H: run healthz (already 300ms late)
    Note over L: --- after fix ---
    L->>P: run_in_executor(resize)
    L->>H: healthz runs immediately
    P-->>L: future resolves, resume handler

The fix hands the offending work to another OS thread or process, so the loop's callback returns in microseconds and it goes back to polling. Choice matters:

• I/O-bound blocking (requests, psycopg2, boto3, file reads): ThreadPoolExecutor works, because the GIL is released during the syscall.
• CPU-bound (image resize, big json.loads, crypto, pandas): threads still fight for the GIL and will starve loop callbacks anyway. Use ProcessPoolExecutor or move the job to a queue worker.

Before and After

# BEFORE: 300ms of Pillow work runs inside the event loop callback
@app.post("/avatar")
async def avatar(file: UploadFile):
    data = await file.read()
    img = Image.open(io.BytesIO(data))
    img.thumbnail((256, 256))          # blocks the ENTIRE loop
    out = io.BytesIO(); img.save(out, "WEBP")
    return Response(out.getvalue(), media_type="image/webp")

# AFTER: CPU work in a process pool -> loop keeps polling sockets
POOL = concurrent.futures.ProcessPoolExecutor(max_workers=2)  # bounded on purpose

def _resize(data: bytes) -> bytes:
    img = Image.open(io.BytesIO(data)); img.thumbnail((256, 256))
    out = io.BytesIO(); img.save(out, "WEBP"); return out.getvalue()

@app.post("/avatar")
async def avatar(file: UploadFile):
    data = await file.read()
    loop = asyncio.get_running_loop()
    # await yields control; other handlers run while this resizes elsewhere
    body = await loop.run_in_executor(POOL, _resize, data)
    return Response(body, media_type="image/webp")

When NOT to Use This

• If the work is long (>1–2s) or must survive a deploy, an executor is the wrong home. A restart kills in-flight jobs and clients hold connections open for seconds. Use Celery/RQ/Arq and return a job id.
• If your whole app is blocking (a legacy Django ORM codebase), don't async-ify it and shove everything into executors — you get thread-pool exhaustion plus async complexity for zero gain. Run sync WSGI with a thread/process worker model; FastAPI's plain def handlers already do this via its own threadpool.
• If the block is 5 ms, leave it. Executor round-trip and pickling overhead (process pools pickle args and results) can exceed the work.
• Millisecond-scale CPU work at very high rps: process pools serialize your payload twice; a 40 MB image round-trip may cost more than it saves.

Gotchas

• run_in_executor(None, ...) uses the loop's default thread pool — the same one FastAPI uses for sync handlers and anyio.to_thread. Flood it and your sync endpoints stall. Use a dedicated, bounded executor.
• Unbounded pools are a DoS vector. ThreadPoolExecutor() defaults to min(32, cpu+4) in modern Python, but a naive high number means 200 concurrent DB connections when traffic spikes. Bound it and let callers wait.
• ProcessPoolExecutor + fork copies your process, including open DB sockets and loaded ML models. Create the pool at startup before connections exist, or use mp_context=get_context("spawn").
• Exceptions surface on await, not at submit time — and process pools re-raise only picklable exceptions; custom exceptions with non-picklable attrs turn into confusing PicklingErrors.
• Async DB drivers can still block: TLS handshakes, DNS resolution via socket.getaddrinfo (asyncio runs it in a thread, but a slow resolver still eats pool slots), and gzip decompression in "async" HTTP clients.
• Instrument it permanently. Ship loop.slow_callback_duration = 0.1 with debug mode in staging, and run a loop-lag gauge in production: a task that sleeps 100 ms in a loop and exports actual - expected. Loop lag is the single best leading indicator for this class of bug — long before p99 alerts fire.

Key takeaway: In asyncio, any code between two awaits runs exclusively — measure it, and push anything over ~50ms into a thread pool (I/O) or process pool (CPU) instead of hoping concurrency saves you.

import asyncio, logging
loop = asyncio.get_running_loop()
loop.set_debug(True)
loop.slow_callback_duration = 0.05  # warn on >50ms in a single callback
logging.basicConfig(level=logging.WARNING)

async with self.session.get(url, timeout=aiohttp.ClientTimeout(total=1)) as r:
    data = await r.json()