1 min lesson
A wrong suggestion costs more than a missing one
Show why this lesson detail matters. "A missing completion is a non-event."
Step 1 of 3
A wrong suggestion costs more than a missing one
This is the asymmetry that separates devtools judgment from generic product sense. A missing completion is a non-event. A confidently wrong completion that a developer accepts can plant a bug, waste a debugging session and, worst of all, teach them not to trust the tool. So the product's job is to be right when it speaks, quiet when it isn't sure, and to keep the diff always there so the developer can verify before anything lands.
Learn more
Full explanation
Cost versus quality is a per-request decision
Cost versus quality is a per-request decision
Every call to a premium model costs real money and real latency. The PM's judgment is knowing when a task deserves the expensive, slow, high-quality path and when it should degrade gracefully to something cheaper and faster.
- Task
- Tab on a trivial line
- Spend the tokens?
- No
- Why
- Cheap, fast model; the value is speed and volume, not depth
- Task
- Multi-file refactor via Agent
- Spend the tokens?
- Yes
- Why
- Correctness compounds across files; a cheap miss costs hours
- Task
- "Explain this function" in Chat
- Spend the tokens?
- Mid
- Why
- Quality matters but the user is verifying anyway; balance cost and speed
- Task
- Bulk, low-stakes suggestions
- Spend the tokens?
- No
- Why
- Degrade gracefully; reserve premium compute for where it changes the outcome
| Task | Spend the tokens? | Why |
|---|---|---|
| Tab on a trivial line | No | Cheap, fast model; the value is speed and volume, not depth |
| Multi-file refactor via Agent | Yes | Correctness compounds across files; a cheap miss costs hours |
| "Explain this function" in Chat | Mid | Quality matters but the user is verifying anyway; balance cost and speed |
| Bulk, low-stakes suggestions | No | Degrade gracefully; reserve premium compute for where it changes the outcome |
Don't spend premium compute where a cheap model would be indistinguishable to the user.
Metrics can lie: the acceptance-rate trap
Acceptance rate is the obvious devtools proxy metric, and it's seductive because it's easy to move. It is also easy to game in ways that hurt the product.
You can lift acceptance rate by making suggestions shorter and more conservative, which lowers the actual value delivered. You can also have high acceptance and low retention if the suggestions are accepted then immediately edited away. Pair the proxy with a grounded outcome: code that survives, retention and time-to-task, not just the tap on Tab.
Learn more
Optional practice
Test yourself on a wrong suggestion costs more than a missing one
QYour team proposes shipping a change that raises Tab acceptance rate from 22% to 31%. What do you ask before celebrating?