1 min lesson
Structure the write-up - decision first, evidence in the appendix
Work through the cases in "Structure the write-up - decision first, evidence in the appendix", pairing each signal with the move that fits.
Step 1 of 2
Structure the write-up - decision first, evidence in the appendix
Top of page: the cause, the blast radiusHow much breaks if a change goes wrong; the scope of potential damage. Press Enter for the full definition. in one number and the recommendation.
Example: “p99 rose 2.1s for the ~8% of users on client 2.14 in eu-west; recommend rollback. Evidence below.”
Segmented charts and the confound checks you ran, in the appendix.
Close with limitations and next steps and keep the analysis reproducible (query + notebook).
Reviewers are senior and time-boxed. If page one is a histogram of latency and the recommendation is on page nine, you've demonstrated you can analyze but not communicate to a decision-maker - which is half the job here. The recommendation, the blast radiusHow much breaks if a change goes wrong; the scope of potential damage. Press Enter for the full definition. and the confidence go in the first three sentences. Everything that justifies them goes underneath.
Learn more
Full explanation
Self-score with a hiring-bar rubric
Self-score with a hiring-bar rubric
- Axis
- Clarity
- Below bar
- Decision buried; reader hunts for the point.
- At hiring bar
- Cause, blast radiusHow much breaks if a change goes wrong; the scope of potential damage. Press Enter for the full definition., recommendation in the first three sentences.
- Axis
- Correctness
- Below bar
- Blamed the global trend; missed a confound.
- At hiring bar
- Localized to a segment, ruled out traffic-mix, change-point confirmed.
- Axis
- Pragmatism
- Below bar
- Explored everything; recommended nothing.
- At hiring bar
- Made a call (roll back / fix forward) with the evidence that decided it.
- Axis
- Communication
- Below bar
- Notebook dump, not reproducible.
- At hiring bar
- Tight write-up, reproducible query/notebook, limitations stated.
| Axis | Below bar | At hiring bar |
|---|---|---|
| Clarity | Decision buried; reader hunts for the point. | Cause, blast radiusHow much breaks if a change goes wrong; the scope of potential damage. Press Enter for the full definition., recommendation in the first three sentences. |
| Correctness | Blamed the global trend; missed a confound. | Localized to a segment, ruled out traffic-mix, change-point confirmed. |
| Pragmatism | Explored everything; recommended nothing. | Made a call (roll back / fix forward) with the evidence that decided it. |
| Communication | Notebook dump, not reproducible. | Tight write-up, reproducible query/notebook, limitations stated. |
Score the version you'd submit today, not the one you imagine after another pass.
QYour global p99 jumped after release N. You segment by client_version and no single version shows a jump, yet the aggregate clearly rose. What's the most likely explanation and your next move?