How to Set an HTTP Client Timeout to Stop Cascading Failures

Why no timeout causes cascading failure

When your API calls a downstream service and that service stalls - slow response, dropped connection, overloaded server - your request handler blocks waiting for a reply that never comes. With ~20 concurrent requests all waiting, every worker thread is occupied. The load balancer still marks you healthy (you're accepting connections), so traffic keeps arriving, and the queue grows until the whole service is dead.

The root cause is the default httpx.Client() with no timeout - which means infinite wait. Reproducing it is trivial:

curl --max-time 8 http://localhost:8000/shipping/123
# hangs for 8 seconds - blocked by the stalled downstream

Set an explicit httpx.Timeout

httpx.Timeout lets you specify timeouts for each phase of the request lifecycle separately. The first positional argument is the default applied to all four phases (connect, read, write, pool); per-phase overrides are keyword arguments:

import httpx
from fastapi import FastAPI
from fastapi.responses import JSONResponse

app = FastAPI()

client = httpx.Client(
    base_url="http://downstream-service",
    timeout=httpx.Timeout(3.0, connect=2.0),
)

@app.get("/shipping/{order_id}")
def get_shipping(order_id: str):
    try:
        resp = client.get(f"/orders/{order_id}")
        resp.raise_for_status()
        return resp.json()
    except httpx.TimeoutException:
        return JSONResponse({"error": "upstream timeout"}, status_code=503)
    except httpx.HTTPError as exc:
        return JSONResponse({"error": str(exc)}, status_code=502)

Here connect=2.0 caps the TCP handshake and 3.0 caps the read phase. The downstream now has at most 3 seconds to respond - then the request fails fast and the worker is freed immediately.

Tune each timeout phase independently

Set read based on what the downstream's SLO actually promises - not an arbitrary large value. An SLO of "p99 response in 2s" means your read timeout should be around 3-4s, not 30.

Catch the right exception

httpx raises httpx.TimeoutException (a subclass of httpx.HTTPError) on any timeout. Always catch it explicitly and return a 503 Service Unavailable - that signals "I am healthy but a dependency is not," which lets upstream retry logic and circuit breakers work correctly.

For the next layer of resilience, combine this with a circuit breaker (open after N consecutive timeouts, stop calling the downstream entirely for a backoff period).