2 min lesson
Reward shaping: a nonlinear length penalty
Name the parts in "Reward shaping: a nonlinear length penalty" and give the practical job of each one.
Step 1 of 3
A spread-out field is what lets you decide a release on a few points of movement. When models bunch near a ceiling, a small delta is lost in the noise; when the field is spread, the same delta is a signal you can act on.
Learn more
Full explanation
Full explanation
Any length term is a reward proxy, so Goodhart still applies. Weight the penalty too hard and the policy will sacrifice correctness to hit the token budget - exactly the kind of metric-up, capability-down hack from the reward-hacking section. The nonlinear shape and a quality grader that dominates the reward are what keep length in its lane: a tie-breaker toward tight solutions, not a target the model chases at the expense of being right.
If asked why you'd build an internal benchmark when SWE-bench exists, give the concrete reasons: SWE-bench is public so it's contaminated and saturating, and it only scores correctness. An internal bench drawn from real engineer queries stays uncontaminated, keeps separating models after the public one saturates, and can score the parts a pass/fail test can't - quality, intent on under-specified prompts, and output length. Then connect it to training: that length signal becomes a nonlinear penalty in the reward, tuned to punish both verbosity and truncation.
Learn more
Optional practice
Practice: Reward shaping: a nonlinear length penalty
QCursorBenchCursor's internal benchmark that scores models on both task performance and token efficiency, not accuracy alone. Press Enter for the full definition. uses deliberately short, under-specified prompts and scores completion-token length alongside quality. Why each choice, and how does the length signal enter training without backfiring?