2 min lesson
Where the Spend Is Worth it
Think through this situation: "An interviewer asks whether you'd hedge (send duplicate requests to two providers) on Tab completions. How do you reason through it?" Give the practical answer in plain words.
Step 1 of 2
The JD's core judgment skill is "making good calls in the gray area: weighing reliability, cost, latency and user experience." These four pull against each other constantly and the job is choosing where to spend on which axis for which surface.
Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.
Cost on the x-axis, user-visible payoff on the y-axis - the same technique lands differently per surface.
Learn more
Advanced table
Every resilience technique buys one axis by spending another
- Tension
- Reliability vs cost
- What you gain
- Hedged / duplicate requests cut tail latency and survive a slow provider
- What it costs
- You pay for the loser of every race - real GPU and provider dollars
- When it bites
- Hedge everything and the bill doubles for a p99 win
- Tension
- Latency vs quality
- What you gain
- A faster, cheaper fallback model keeps the feature responsive
- What it costs
- Output quality may drop versus the primary model
- When it bites
- Fine for Tab; risky for an Agent step a user is trusting
- Tension
- Cost vs UX
- What you gain
- Load shedding protects the fleet during a spike
- What it costs
- Some users get throttled, queued or refused
- When it bites
- Who gets shed and does it fail gracefully or with a broken cursor?
- Tension
- Reliability vs latency
- What you gain
- Retries with backoff recover transient provider errors
- What it costs
- Each retry adds latency to an already-slow request
- When it bites
- Retrying a request that's slow (not failed) makes the p99 worse
| Tension | What you gain | What it costs | When it bites |
|---|---|---|---|
| Reliability vs cost | Hedged / duplicate requests cut tail latency and survive a slow provider | You pay for the loser of every race - real GPU and provider dollars | Hedge everything and the bill doubles for a p99 win |
| Latency vs quality | A faster, cheaper fallback model keeps the feature responsive | Output quality may drop versus the primary model | Fine for Tab; risky for an Agent step a user is trusting |
| Cost vs UX | Load shedding protects the fleet during a spike | Some users get throttled, queued or refused | Who gets shed and does it fail gracefully or with a broken cursor? |
| Reliability vs latency | Retries with backoff recover transient provider errors | Each retry adds latency to an already-slow request | Retrying a request that's slow (not failed) makes the p99 worse |
Every resilience technique buys one axis by spending another.
The surface changes the right answer. Tab has a sub-100ms budget, so a cheap fast model and aggressive shedding are defensible - a missed completion is invisible. An Agent step running a multi-minute task can tolerate a slower, higher-quality model and a retry, because the user is already waiting and correctness matters more than a few hundred milliseconds.
There is rarely a single correct answer here and the interviewers know it. They grade how you reason and how you'd measure the tradeoff, not a memorized verdict.