2 min lesson
PPO, GRPO and the modern stack
Look at both sides of "PPO, GRPO and the modern stack" and name the signal that decides which one fits.
Step 1 of 2
PPO was the workhorse of reinforcement learning from human feedback, or RLHF. Group Relative Policy Optimization, or GRPO, made large-scale RL on reasoning and code cheap enough to run at scale, and cheaper RL matters at Cursor, whose charter names compute-efficient training.
A screen may ask you to contrast them on real memory footprints rather than vibes. Both descend from the policy gradient in section one; they differ in how they estimate the advantage and how many models they keep in memory.
Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.
Step through each dimension. The difference is where the baseline comes from and how much memory you pay for it.