1 min lesson
Why request/response breaks
Match each case in "Why request/response breaks" to the signal and response that fit it.
Step 1 of 2
Why request/response breaksthe wrong default
Long-running
A run can outlive any single HTTP timeout or deploy.
Holding a connection open for the whole run wastes a server slot and dies on the first restart.
Stateful
Context, tool results and partial edits accumulate across many model calls.
That state must survive a crash, not live only in process memory.
Bursty
Load is spiky: a model release or a workday peak floods the queue.
Provision for the burst with backpressure, not for the average with hope.
Expensive
Every LLM call has real per-token cost and latency.
A retry storm or a runaway loop is a bill, not just an error rate.