2 min lesson
Efficient training systems
Use the lesson to respond to this: "In an MoE model, why is a load-balancing auxiliary loss usually necessary?" Keep the answer plain.
Step 1 of 3
Cursor frames its researchers as people who build the infra their experiments need, not just write loss functions. ComposerCursor's own fast coding model, tuned for the editor and priced well below frontier models; the recommended day-to-day model for executing a plan. Press Enter for the full definition.'s training stack - MoE, custom low-precision kernels on Blackwell, async multi-region RL, sandboxed rollouts - is fair game in the interview.
You don't need to have written a CUDA kernel to pass, but systems literacy is part of the bar. You should know why MoE buys capacity, where low-precision risks numerical blow-ups and which parallelism axis bottlenecks an RL rollout-plus-train loop.
Learn more
Full explanation
Low-precision kernels
Low-precision kernelsthroughput vs numerical stability
Cursor has said it wrote custom low-precision kernels to train MoE models on NVIDIA's Blackwell hardware. Lower precision (FP8 and below) roughly doubles throughput and halves memory traffic, but the numerics get sharp: small values underflow, large ones overflow and a single bad reduction can wreck a step.
- Precision
- BF16
- Use
- Default training compute; wide exponent range
- Risk
- More memory/compute than FP8
- Precision
- FP8
- Use
- Matmuls on Blackwell for ~2x throughput
- Risk
- Underflow/overflow; needs per-tensor or block scaling
- Precision
- FP32
- Use
- Master weights, optimizer state, key reductions
- Risk
- Slow/heavy; reserved for stability-critical paths
| Precision | Use | Risk |
|---|---|---|
| BF16 | Default training compute; wide exponent range | More memory/compute than FP8 |
| FP8 | Matmuls on Blackwell for ~2x throughput | Underflow/overflow; needs per-tensor or block scaling |
| FP32 | Master weights, optimizer state, key reductions | Slow/heavy; reserved for stability-critical paths |
Keep accumulation and master weights high-precision; push only the heavy matmuls to FP8 with scaling to stay numerically safe.