2 min lesson
The agent proposes, you verify
For each case in "The agent proposes, you verify", name the signal and the response you would use.
Step 1 of 2
What the agent is genuinely good atthe heavy lifting worth delegating
Tagging area / severity / sentiment at volume
Clustering and dedup by semantic similarity
Drafting a candidate repro from a vague ticket
Summarizing a cluster's common thread for review
Confirming the repro actually reproduces
Final severity call on anything near S0
The number that goes in front of leadership
Deciding what to escalate and when
Human-in-the-loop is the same judgment bar the interview grades. An agent that guesses severity is a triage assistant; an agent whose guesses you accept blindly is a liability that will mislabel an S0 as an S2 and bury it. You own the output the agent drafts.
The same rule guards the repro the agent drafts for you. An agent will happily generate a check that asserts nothing and passes green, and a green check that proves nothing is worse than no check. Determine the golden state - the input and the expected output - and feed that to the agent as the anchor, then read the generated code before you trust it.
Never trust an automated test that you didn't see fail.
Two failure shapes recur when an agent writes its own checks: it logs a failure instead of actually failing the run, or it wraps the assertion in a try-catch that swallows the error. The fix is engineering discipline. Keep the agent's context lean and pointed at the one task - a long, cluttered conversation can overwrite your instructions as the window fills - and run a validator step that executes the check, reads the output and reruns to confirm a real failure surfaces before you rely on it.