1 min lesson
When signals conflict
Work through this: "Your survey says 80% of developers are satisfied with the agent, but telemetry shows a high revert rate on agent edits and interviews surface distrust. How do you reconcile this in your readout?"
Step 1 of 2
When signals conflictthe part that reveals seniority
Conflicting data is the normal case, not a failure. Resolve it explicitly by deciding which source is authoritative for which kind of claim, then say so out loud.
- For a claim about…
- What people actually do
- Trust most
- Analytics / experiment
- Because
- Behavior beats self-report every time.
- For a claim about…
- Why they do it
- Trust most
- Qual
- Because
- Only interviews recover the mechanism.
- For a claim about…
- How widespread it is
- Trust most
- Survey / analytics
- Because
- Scale is exactly their strength.
- For a claim about…
- Whether a change caused an effect
- Trust most
- Experiment
- Because
- Randomization is the only clean causal evidence.
| For a claim about… | Trust most | Because |
|---|---|---|
| What people actually do | Analytics / experiment | Behavior beats self-report every time. |
| Why they do it | Qual | Only interviews recover the mechanism. |
| How widespread it is | Survey / analytics | Scale is exactly their strength. |
| Whether a change caused an effect | Experiment | Randomization is the only clean causal evidence. |
Name the authority for each claim type rather than averaging conflicting sources into mush.
Tag every claim with how sure you are and why. “Strong: experiment plus matching telemetry.” “Suggestive: consistent across nine interviews, not yet sized.” “Speculative: one vivid quote.” Over-claiming from thin data is the fastest way to lose a technical team's trust and you rarely win it back.
“Interviews told me developers distrust silent agent edits. The survey showed it's a top-three concern for power users. Telemetry confirmed they revert more after large diffs. So we tested a preview and acceptance rose while reverts held flat. Four sources, one story and I'm confident because they converge.”