2 min lesson
Diff-in-diff in practice
For each case in "Diff-in-diff in practice", name the signal and the response you would use.
Step 1 of 2
Diff-in-diff in practicethe default reach for infra changes
You migrate one region's agent backend and leave a comparable region untouched. Diff-in-differences subtracts each region's pre-period level and compares the post-period change between them, which cancels out fixed differences between regions and any shock that hit both equally. The estimate is only causal if the two regions were trending in parallel before the migration.
metric ~ treated + post + (treated * post) + region_FE + day_FE # the coefficient on (treated * post) is the causal estimate # always plot the pre-period trends first: # if they diverge before treatment, parallel-trends fails # and the DiD number is not credible
Treated unit vs a weighted blend of donor units
Weights fit the pre-period trajectory, not just the level
Strong for one treated region in a staged rollout
Causal identification lives in a narrow band around the cutoff
Powerful when a threshold mechanically assigns treatment
Only speaks to units near the boundary, not the whole population
When asked "did this rollout cause the latency drop?", don't jump to a method name. First say what the ideal randomized experiment would have been, then explain why it's impossible here, then pick the quasi-experiment whose assumption is most defensible given the data you have. That ordering shows you reach for randomization first and treat causal inference as the fallback it is.
Propensity scoring and matching only adjust for confounders you measured. They do nothing about unobserved ones and a confident causal claim from matching on three covariates invites a fair takedown. State the assumption (ignorability) plainly and, ideally, bound how strong an unmeasured confounder would have to be to overturn your result.