1 min lesson

Difficulty: train at the edge

Tell someone how to act on this idea: "Which is why difficulty calibration is a training-efficiency lever, not a nicety."

Difficulty: train at the edgewhere the gradient lives

In RL the signal comes from the spread between good and bad rollouts. A task the model always passes or always fails gives no contrast and no learning. The useful datapoint is the one the model gets right sometimes - that's where advantage estimation has something to work with. GRPO-style methods make this concrete: they normalize advantage within a group of rollouts on the same prompt, so a prompt with all-same outcomes contributes nothing - which is why difficulty calibration is a training-efficiency lever, not a nicety.

Learn more

Full explanation

Distribution-matching and synthetic data

The easiest data to get - scraped repos, leetcode-style problems, tasks with clean tests - is not the data that matches a real Cursor session. Real requests are multi-file, underspecified, full of project-specific conventions and often have no tests. Closing that gap is its own research problem.

1Profile the real distribution. Characterize what users actually ask - task types, repo sizes, languages, how much context, how multi-file - so you know what you're matching.
2Generate to fill gaps. Synthesize tasks where real data is thin, using past models to draft problems and candidate solutions in the under-covered regions.
3Verify before you trust. Filter synthetic datapoints through execution checks and graders; a synthetic task with a wrong "gold" answer poisons training.
4Calibrate and recycle. Bin by measured difficulty, prune the trivial and the impossible and bootstrap the next round of harder data from the model you just trained.

Watch out

Synthetic data is a feedback loop and feedback loops drift. If you generate tasks from the current model and grade them with the current grader, you can amplify both their blind spots - training the model to be confidently good at exactly the cases your pipeline can't tell apart. Anchor synthetic data to real distributions and trusted verification and keep a human in the loop on samples.

Interview move

If asked "you have a fixed compute budget, how do you pick training data," don't say "use more data." Say: drop tasks the model already aces (no gradient), drop tasks it never solves (no signal), spend the budget at the edge of ability and weight toward the real user distribution rather than the scrapeable one. Then mention difficulty calibration as a curriculum and verifying any synthetic data before it enters the run.

QWhy does a training task the model solves correctly 100% of the time waste compute in an RL run and what difficulty is most valuable?

Difficulty: train at the edge

Step 1 of 3