Skip to lesson
Exit
Deep Dive: Graders, Rewards & Evals1 / 2

1 min lesson

Data quality, difficulty and distribution

For each case in "Data quality, difficulty and distribution", name the signal and the response you would use.

Step 1 of 2

The grader scores the answer, but the datapoint sets the question. A model trained on the wrong tasks learns the wrong things efficiently - and Cursor explicitly hires for improving datapoint quality and difficulty.

Three properties make a training datapoint worth its compute. They pull against each other, which is why "more data" is rarely the answer and curation is the actual job.

Quality

The task is well-posed and the reward signal on it is trustworthy.

A correct solution exists and is gradeable.

Garbage datapoints teach the grader's noise, not the skill.

Difficulty

Too easy: the model already solves it, so the gradient is near zero - wasted compute.

Too hard: the model never succeeds, so there's no positive signal to learn from.

The value is at the edge of current ability.

Distribution

Matches the real spread of what developers ask Cursor to do.

Not just scrapeable, easy or English-only tasks.

Train on the work users bring, not the work that's convenient.

Learn more

Full explanation

Which Datapoints Earn Their Compute

WHICH DATAPOINTS EARN THEIR COMPUTE

Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.

diagram: quadrant

Spend the fixed budget in the top-right: tasks at the edge of ability that also match what real developers ask. Prune the rest.