2 min lesson
Efficient training systems
Use the lesson to respond to this. "In an MoE model, why is a load-balancing auxiliary loss usually necessary?" Keep the answer plain.
Step 1 of 3
Cursor frames its researchers as people who build the infra their experiments need, not just write loss functions. MoE layout, low-precision matmuls, async RL and sandboxed rollouts are fair game in the interview.
You don't need to have written a CUDA kernel to pass, but systems literacy is part of the bar. You should know why MoE buys capacity, where low-precision risks numerical blow-ups and which parallelism axis bottlenecks an RL rollout-plus-train loop.
Learn more
Full explanation
Low-precision kernels
Low-precision kernelsthroughput vs numerical stability
Frontier MoE training increasingly pushes matmuls to FP8 and below. Lower precision roughly doubles throughput and halves memory traffic, but the numerics get sharp: small values underflow, large ones overflow and a single bad reduction can wreck a step.
- Precision
- BF16
- Use
- Default training compute; wide exponent range
- Risk
- More memory/compute than FP8
- Precision
- FP8
- Use
- Heavy matmuls for ~2x throughput
- Risk
- Underflow/overflow; needs per-tensor or block scaling
- Precision
- FP32
- Use
- Master weights, optimizer state, key reductions
- Risk
- Slow/heavy; reserved for stability-critical paths
| Precision | Use | Risk |
|---|---|---|
| BF16 | Default training compute; wide exponent range | More memory/compute than FP8 |
| FP8 | Heavy matmuls for ~2x throughput | Underflow/overflow; needs per-tensor or block scaling |
| FP32 | Master weights, optimizer state, key reductions | Slow/heavy; reserved for stability-critical paths |
Keep accumulation and master weights high-precision; push only the heavy matmuls to FP8 with scaling to stay numerically safe.