1 min lesson
Peeking and sequential testing
Use "Peeking and sequential testing" to tell the cases apart, then choose a response for each one.
Step 1 of 2
Peeking and sequential testingthe bug that fakes wins
A fixed-horizon test promises one significance level if you look once, at the end. Engineers in a hurry look every morning and ship the moment p drops below 0.05. That naive repeated peeking can inflate the false-positive rate past 30 percent. You either commit to a fixed sample and look once or you use a method built for continuous monitoring.
Always-valid p-values
Sequential / mSPRT-style inference
Look as often as you want, type-I error still controlled
Pay for it with a wider effective threshold
Group-sequential designs
Pre-planned interim looks with alpha-spending
Can stop early for a clear win or for futility
Needs the look schedule fixed up front