2 min lesson
The production tradeoffs you own
Put yourself in this case: "An FDE owns latency, cost and quality tradeoffs on every model call. True or false: with enough engineering you can optimize all three simultaneously to their maximum." Give the clearest next step.
Step 1 of 2
The production tradeoffs you ownnot delegated, not optional
Once a workflow is live, you own its quality directly. That means tracing every model call, maintaining the eval set, watching the metrics and debugging model behavior when it drifts. It also means every model call you ship is a live decision on three competing axes.
- Axis
- Latency
- What you trade
- Faster responses vs. more reasoning per call
- A real FDE call
- Use a smaller model for inline edits, reserve the big one for the agent-led refactor loop
- Axis
- Cost
- What you trade
- Token spend vs. output quality and retries
- A real FDE call
- Cache repo context and batch file edits so a 4,000-file refactor doesn't reindex per call
- Axis
- Quality
- What you trade
- Accept-rate / correctness vs. speed and spend
- A real FDE call
- Add a verification pass on generated diffs even though it costs an extra call
| Axis | What you trade | A real FDE call |
|---|---|---|
| Latency | Faster responses vs. more reasoning per call | Use a smaller model for inline edits, reserve the big one for the agent-led refactor loop |
| Cost | Token spend vs. output quality and retries | Cache repo context and batch file edits so a 4,000-file refactor doesn't reindex per call |
| Quality | Accept-rate / correctness vs. speed and spend | Add a verification pass on generated diffs even though it costs an extra call |
You can't max all three. The job is choosing the right point for this customer's workflow.
Communicating tradeoffs and not over-promising is part of the engineering job, not a soft extra. Telling a staff engineer "this refactor pass will be 92% correct and here's the verification step that catches the rest" builds more trust than promising 100% and getting caught. Honest bad news with a path forward is the FDE house style.
"I can get inline edits under 300ms, but the full agent-assisted migration runs slower and costs more per file. I'd ship the fast path for daily use and gate the migration behind a verification pass. Here's the cost-per-PR estimate so you can decide the budget."