1 min lesson
Sandboxes are the foundation
Walk through the important items in "Sandboxes are the foundation" and give the practical point of each.
Step 1 of 2
Sandboxes are the foundation
None of this runs without a way to execute hundreds of thousands of untrusted agent rollouts safely and fast. Cursor's reported Anyrun-style sandboxing is what makes RL on real coding tasks possible at all.
- Isolation: agents run arbitrary code, so each rollout is a hardened sandbox that can't touch the host or other rollouts.
- Throughput: rollouts must spin up in well under a second and run massively in parallel or the trainer starves.
- Reproducible grading: each sandbox runs the tests/verifier deterministically so the reward is trustworthy.
- Cost: this is a dominant line item, which is part of why training-with-less-compute is a stated research goal.
When the onsite touches infra, reason about the bottleneck out loud: "rollout generation is the long pole, so I'd decouple it from the trainer, run sandboxes async across regions and accept some staleness with importance weighting - then watch accelerator utilization and the off-policy gap as my two health metrics." Naming utilization and staleness as the things you'd instrument signals you've thought about the system, not just the loss.
Learn more
Optional practice
Practice: Sandboxes are the foundation
QIn a fully async RL pipeline, why does data become off-policy and what's the lever you trade against it?