1 min lesson
RLHF vs RLVR
Put this idea into your own words: "RLHF learns a reward model from human preferences."
Step 1 of 3
Where does the reward come from? That single question splits modern post-training into two camps and coding sits awkwardly across the line between them.
RLHF learns a reward model from human preferences. RLVR reads reward straight off a verifier - a compiler, a test suite, a proof checker. The deep-dive will push on which one you'd reach for and, more sharply, how each one fails.
Learn more
Advanced table
RLVR scales with compute, RLHF with human labor
- Reward source
- RLHF (human feedback)
- Learned model trained on human preference pairs
- RLVR (verifiable reward)
- Deterministic checker: tests, compiler, proof
- Where it shines
- RLHF (human feedback)
- Taste, tone, helpfulness - things with no ground truth
- RLVR (verifiable reward)
- Math, code, anywhere correctness is checkable
- Signal quality
- RLHF (human feedback)
- Noisy, biased by labelers, trained on far less data than the base model
- RLVR (verifiable reward)
- Clean, scalable, reproducible
- Main failure
- RLHF (human feedback)
- Reward model is gameable and drifts off-distribution
- RLVR (verifiable reward)
- Reward is only as honest as the verifier
- Scales by
- RLHF (human feedback)
- Collecting more human labels (slow, costly)
- RLVR (verifiable reward)
- Running more environments (compute, not humans)
| RLHF (human feedback) | RLVR (verifiable reward) | |
|---|---|---|
| Reward source | Learned model trained on human preference pairs | Deterministic checker: tests, compiler, proof |
| Where it shines | Taste, tone, helpfulness - things with no ground truth | Math, code, anywhere correctness is checkable |
| Signal quality | Noisy, biased by labelers, trained on far less data than the base model | Clean, scalable, reproducible |
| Main failure | Reward model is gameable and drifts off-distribution | Reward is only as honest as the verifier |
| Scales by | Collecting more human labels (slow, costly) | Running more environments (compute, not humans) |
RLVR scales with compute, RLHF with human labor - a big reason top-tier reasoning runs lean on RLVR.