1 min lesson
Cost versus quality is a per-request decision
Compare two rows from "Cost versus quality is a per-request decision", then say when each one fits.
Step 1 of 2
Cost versus quality is a per-request decision
Every call to a premium model costs real money and real latency. The PM's judgment is knowing when a task deserves the expensive, slow, high-quality path and when it should degrade gracefully to something cheaper and faster.
- Task
- Tab on a trivial line
- Spend the tokens?
- No
- Why
- Cheap, fast model; the value is speed and volume, not depth
- Task
- Multi-file refactor via Agent
- Spend the tokens?
- Yes
- Why
- Correctness compounds across files; a cheap miss costs hours
- Task
- “Explain this function” in Chat
- Spend the tokens?
- Mid
- Why
- Quality matters but the user is verifying anyway; balance cost and speed
- Task
- Bulk, low-stakes suggestions
- Spend the tokens?
- No
- Why
- Degrade gracefully; reserve premium compute for where it changes the outcome
| Task | Spend the tokens? | Why |
|---|---|---|
| Tab on a trivial line | No | Cheap, fast model; the value is speed and volume, not depth |
| Multi-file refactor via Agent | Yes | Correctness compounds across files; a cheap miss costs hours |
| “Explain this function” in Chat | Mid | Quality matters but the user is verifying anyway; balance cost and speed |
| Bulk, low-stakes suggestions | No | Degrade gracefully; reserve premium compute for where it changes the outcome |
Don't spend premium compute where a cheap model would be indistinguishable to the user.
Learn more
Full explanation
Metrics can lie: the acceptance-rate trap
Metrics can lie: the acceptance-rate trap
Acceptance rate is the obvious devtools proxy metric and it's seductive because it's easy to move. It is also easy to game in ways that hurt the product.
You can lift acceptance rate by making suggestions shorter and more conservative, which lowers the actual value delivered. You can also have high acceptance and low retention if the suggestions are accepted then immediately edited away. Pair the proxy with a grounded outcome: code that survives, retention and time-to-task, not just the tap on Tab.
Learn more
Optional practice
Practice: Cost versus quality is a per-request decision
QYour team proposes shipping a change that raises Tab acceptance rate from 22% to 31%. What do you ask before celebrating?