1 min lesson
Segment-aware alerting
Recall the main items in "Segment-aware alerting", then connect each one to the work.
Step 1 of 2
Segment-aware alerting
The global line is a liar. A regression that doubles latency for one model in one region can be completely invisible in the aggregate, because that slice is a small share of total volume. At Cursor's scale the unit of analysis is the user-AI interaction across billions per period and the regressions that matter most often hide in a single dimension.
- Cut every reliability metric by model, region, client version and surface (Tab, Agent, chat) before you trust the headline number.
- A flat global p95 with a spiking per-model p95 is the classic Simpson's-paradox trap - the mix moved, not the experience.
- Run detection per segment, not only on the rollup or you'll ship the regression that only hurts the new client build.
Learn more
Full explanation
Make it actionable, then auto-triage
Make it actionable, then auto-triage
A detected regression that says "latency is up" wastes the on-call's first hour. Stamp every interaction with a release/commit/model-version dimension at log time so a change-point can name its likely cause: this regression starts at deploy abc123, on model X. That single join turns an observation into an assignment.
The hardest, most Cursor-specific triage is separating a model regression from a harness regression. The model got worse versus your agent loop, tool-calls or context assembly got worse. Route them to different owners and route fast.
- Symptom pattern
- Quality drop tracks a model-version flip, harness unchanged
- Likely culprit
- Model regression
- Route to
- Model/eval team
- Symptom pattern
- Tool-call failure or timeout rate jumps after a deploy
- Likely culprit
- Harness regression
- Route to
- Agent-harness eng
- Symptom pattern
- Latency up only on one provider/region
- Likely culprit
- Upstream/infra
- Route to
- Infra / provider on-call
- Symptom pattern
- Success down across all models at once
- Likely culprit
- Shared harness or context layer
- Route to
- Harness eng
| Symptom pattern | Likely culprit | Route to |
|---|---|---|
| Quality drop tracks a model-version flip, harness unchanged | Model regression | Model/eval team |
| Tool-call failure or timeout rate jumps after a deploy | Harness regression | Agent-harness eng |
| Latency up only on one provider/region | Upstream/infra | Infra / provider on-call |
| Success down across all models at once | Shared harness or context layer | Harness eng |
Auto-triage is a routing table over the dimensions you logged - design those dimensions on purpose.
When they hand you "latency went up, how do you investigate," do not start guessing causes. Say you'd first confirm the signal is real (is the A/A clean, is it one segment or all), then localize it on the dimensions you log (deploy, model, region, client), then classify model-vs-harness, then quantify blast radiusHow much breaks if a change goes wrong; the scope of potential damage. Press Enter for the full definition. before paging anyone. Naming that order is what reads as someone who has owned detection, not just plotted a metric.
Don't promise a detector with zero false positives - that just means it's deaf to real regressions. The honest framing is that you pick a point on the sensitivity curve deliberately, justify it with the false-positive cost and the error budget and revisit it as the org's tolerance changes.
QYour global p95 latency is flat after a deploy, but support reports slowness. What's the most likely measurement failure and what do you check first?