1 min lesson
Cost versus fidelity is a live tradeoff
Use "Cost versus fidelity is a live tradeoff" to explain each part and the role it plays.
Step 1 of 2
Cost versus fidelity is a live tradeoff
Every rollout calls the grader, so the grader's cost multiplies across the whole training run. A large-model judge that scores beautifully may be too slow to run on every sample, starving the policy of gradient. A cheap heuristic runs fast but rewards the wrong thing.
- Grader latency / cost
- Sets rollout throughput. A 5-second judge on millions of samples can dominate the compute bill.
- Grader fidelity
- How well it tracks true quality. Higher fidelity usually means a bigger, slower model.
- Gate-then-judge
- Run the cheap execution check first; only pay for the expensive judge on samples that survive.
- Reward-model distillation
- Train a small RM on the big judge's labels, then serve the cheap RM in the loop.
Naming this tradeoff out loud - and the gate/distill tricks that ease it - reads as someone who has run the pipeline, not just designed one.
Asked to design a grader for, say, "refactor this module," don't hand-wave "an LLM judges it." Walk the stack: an execution gate to kill broken builds, a rubric-driven judge for readability and minimal diff, a human-labeled eval set to calibrate the judge and a cost plan (distill the judge into a small RM) so it survives the rollout budget. Then name how you'd attack your own grader.