Skip to lesson
Exit
Capstone: The Mock Loop1 / 2

2 min lesson

Pre-write the defense

Walk through the important items in "Pre-write the defense" and give the practical point of each.

Step 1 of 2

Pre-write the defensethe panel will push; plan for it

Every method choice is a tradeoff and the researchers on the panel will probe each one. The classic gut-check is "do this in half the time" - they want to see you cut scope without losing the spine of the study.

  • "Why not just a survey?" Surveys capture stated trust, not the moment someone reverts a diff. You need behavior plus the explanation behind it and a survey gives you neither cleanly.
  • "Why moderated, not unmoderated?" Trust failures are emotional and contextual; you need to probe the hesitation in real time. Unmoderated would miss the pause before someone clicks reject.
  • "Half the time - what goes?" Drop the survey, keep telemetry plus five sessions. You lose the ability to size the problem, but you keep the directional answer and the mechanism.
  • "What if your sample is biased toward power users?" Acknowledge it, then show the mitigation: deliberately recruit from the at-risk segment (shared-repo engineers) and flag the readout's generalizability limit.
Say it like this

"I'd anchor on one behavioral metric - revert rate on multi-file Agent edits - and pair it with moderated sessions to explain the reverts. If you gave me 48 hours I'd ship telemetry plus five sessions with a confidence flag; with two weeks I'd add a survey to size it. Either way, every finding maps to one call: do we auto-apply or keep the review gate."