1 min lesson
Keep the verification log
Pick two rows from the table in "Keep the verification log" and explain the choice each one supports.
Step 1 of 3
Keep the verification logthe single highest-value artifact
Every time you touch a model suggestion, write one line. This log is both your rehearsal of narration and your debrief data.
- Time
- 0:18
- Suggestion
- Cursor wrote reward fn in one shot
- Verdict
- Rejected
- Why
- Scored a hacked stub as high as a real fix.
- Time
- 0:41
- Suggestion
- Vectorized the advantage calc
- Verdict
- Accepted
- Why
- Matched my loop result on the test batch.
- Time
- 1:05
- Suggestion
- Suggested removing the std-normalization
- Verdict
- Overrode
- Why
- Kept it; variance across prompts was large.
| Time | Suggestion | Verdict | Why |
|---|---|---|---|
| 0:18 | Cursor wrote reward fn in one shot | Rejected | Scored a hacked stub as high as a real fix. |
| 0:41 | Vectorized the advantage calc | Accepted | Matched my loop result on the test batch. |
| 1:05 | Suggested removing the std-normalization | Overrode | Kept it; variance across prompts was large. |
A real onsite reviewer is reading exactly this instinct - that you verify before you trust.
Learn more
Full explanation
Debrief honestly
Debrief honestly
Mark every place you over-trusted a suggestion that turned out wrong.
Mark every place you burned time hand-writing what Cursor could have drafted.
Tighter scope: which slice would they have cut to land sooner?
Faster verification: a quick adversarial test instead of staring at the diff.
Pasting raw model output without verifying it is the clearest failure mode for this role. It is human-in-the-loop research. If the model hands you a grader and you wire it in without checking what it actually rewards, you've demonstrated the exact opposite of the trait they hire for. Your log exists to prove you didn't.
“Cursor wrote this reward function in one shot, but before I trust it I want to see what it scores on three adversarial cases: a passing edit, a reward-hacked stub and a partial multi-file change. If it can't separate those, I'm not using it.”
QHalfway through your timed simulation you realize the full task won't fit the block. What's the highest-signal response?