1 min lesson
Sandboxes are the foundation
Walk through the important items in "Sandboxes are the foundation" and give the practical point of each.
Step 1 of 2
Sandboxes are the foundation
None of this runs without a way to execute hundreds of thousands of untrusted agent rollouts safely and fast. Cursor's reported Anyrun-style sandboxing is what makes RL on real coding tasks possible at that scale.
- Isolation: agents run arbitrary code, so each rollout is a hardened sandbox that can't touch the host or other rollouts.
- Throughput: rollouts must spin up fast and run massively in parallel or the trainer starves.
- Reproducible grading: each sandbox runs the tests/verifier deterministically so the reward is trustworthy.
- Cost: a fleet this large is expensive to run, which is part of why training with less compute is a stated research goal.
If the onsite touches infra, reason about the bottleneck out loud: "rollout generation is the long pole, so I'd decouple it from the trainer, run sandboxes async across regions and accept some staleness with importance weighting. Then I'd watch accelerator utilization and the off-policy gap as my two health metrics."
Learn more
Optional practice
Test yourself on Sandboxes are the foundation
QIn a fully async RL pipeline, why does data become off-policy and what's the lever you trade against it?