1 min lesson
Switchbacks, interleaving and online-eval designs
Consider this situation: "You want to test a change to a global rate limiter that all Cursor users share. Why is a standard user-level A/B test a poor fit and what design handles it?" Start with the decision, then the evidence.
Step 1 of 3
Some changes can't be split per user at all. A shared inference pool, a retrieval index, a global rate limiter: there is one system and everyone uses it. The design has to move to the unit that can actually be isolated.
These are the designs that distinguish a senior experimentation DS from someone who only knows two-arm user tests. Each one exists to handle a structure where the simple split leaks or is infeasible.