1 min lesson
The metric taxonomy
Connect "A strong answer names the family before it names any one metric" to the decision you would make.
Step 1 of 3
Reliability is a feeling until you give it a number. This seat exists to turn that feeling into a metric system an engineer could query tomorrow.
Cursor measures itself in user↔AI interactions at very large scale and the product is non-deterministic. The first thing the metric-design round checks is whether you reach for a single vanity number or a structured family of metrics that hold each other honest. A strong answer names the family before it names any one metric.
Learn more
Advanced table
Name the family, then defend your one north-STAR inside it
- Metric type
- North-star
- What it does
- The single clearest expression of sustained value
- Reliability example at Cursor
- Share of agent runs that complete successfully inside a latency budget
- Metric type
- Guardrail
- What it does
- Must not degrade while you optimize the north-star
- Reliability example at Cursor
- Cost per run, p99 latency, support-ticket rate
- Metric type
- Counter-metric
- What it does
- Catches the cheat that would game the north-star
- Reliability example at Cursor
- False-accept rate of bad edits, silent truncation rate
- Metric type
- Leading
- What it does
- Moves at deploy time, before users feel it
- Reliability example at Cursor
- Canary success rate, post-deploy error-rate delta
- Metric type
- Lagging
- What it does
- Confirms real outcome days or weeks later
- Reliability example at Cursor
- Weekly retained-active, churn linked to a bad run
| Metric type | What it does | Reliability example at Cursor |
|---|---|---|
| North-star | The single clearest expression of sustained value | Share of agent runs that complete successfully inside a latency budget |
| Guardrail | Must not degrade while you optimize the north-star | Cost per run, p99 latency, support-ticket rate |
| Counter-metric | Catches the cheat that would game the north-star | False-accept rate of bad edits, silent truncation rate |
| Leading | Moves at deploy time, before users feel it | Canary success rate, post-deploy error-rate delta |
| Lagging | Confirms real outcome days or weeks later | Weekly retained-active, churn linked to a bad run |
Name the family, then defend your one north-star inside it.
Picking a defensible north-starone number, heavily qualified
A good reliability north-star bundles three things into one definition: an outcome users care about, a quality bar and a time bound. "Agent success rate" alone is too loose. Share of agent runs that reach an accepted edit within the latency budget, measured over a rolling 7-day window is something a query can return.
Notice what that definition refuses to do. It does not reward a run that finishes fast but produces a rejected edit and it does not reward a correct edit that arrived after the user gave up. The bar and the time bound are both load-bearing.