Skip to lesson
Exit
Routing, Failover & Backpressure: The Core Systems Round1 / 2

1 min lesson

Quality-aware fallback

Use "A fallback model is usually cheaper or faster and lower quality" to say what you would do next.

Step 1 of 2

Quality-aware fallback

A fallback model is usually cheaper or faster and lower quality. That tradeoff is acceptable for Tab, where a slightly weaker completion beats no completion and far less acceptable for an Agent doing a multi-file refactor. Decide per surface and make the decision observable.

Say it like this

"I'd make fallback quality-aware and per-surface. Tab can drop to a smaller, faster model under a provider outage and the user barely notices a slightly weaker completion. Agent shouldn't silently degrade mid-task, so there I'd prefer to fail over to an equivalent-tier model on another provider and if none is healthy, surface a clear retryable error rather than quietly hand back worse work. Either way the fallback rate is a first-class metric, because a silent quality regression is the failure mode you find out about from a churn report instead of a dashboard."

Interview move

If they push on "what if two providers go down at once," don't pretend the ladder is infinite. Say it explicitly: failover converts a hard outage into graceful degradation and past the last rung you shed load and return a clean, retryable error - you do not queue requests past the point they're useful. Knowing where graceful degradation ends is more senior than claiming it never does.

Learn more

Optional practice

Practice: Quality-aware fallback

QWhy is a global retry budget safer than a fixed per-request retry count when a provider has a broad outage?