Bug Triage & Prioritization at Scale

At 1M+ DAU the reports never stop. The job isn't to feel busy clearing them, it's to make sure the right one gets worked first - and to be able to defend that ordering to an engineer who'd rather you left them alone.

A Product Quality Engineer on User Operations is the person who decides what's worth an engineer's afternoon. That decision can't be a gut call you redo every morning. It has to be a rubric: a severity axis (how bad is the failure) crossed with an impact axis (how much of the world it touches), producing a number you can put in a brief and stand behind.

Severity: how bad is the failure itselfone report, judged on its worst outcome

Severity describes the damage independent of how many people hit it. A single user losing uncommitted work is a serious event even if they're the only one. Rank failure classes from the one you'd interrupt someone over down to the one that can wait a sprint.

Impact: how much of the world it touchesthe multiplier on severity

Impact is the reach. The same S1 bug is a different priority if it hits 0.1% of free users versus every seat at a paying enterprise. Four dimensions carry most of the signal.

Combining the axes into a scoreso loud-but-rare can't beat quiet-but-catastrophic

Keep severity and impact as separate axes, then collapse them into a priority band. The point of the multiply is that volume alone never wins: a flood of S3 cosmetic complaints stays below a single S0 data-loss report with one reporter.

priority = severity_weight × impact_score

severity_weight:  S0=100  S1=30  S2=8  S3=2
impact_score   =  (pct_DAU_affected × 10)
               +  (arr_affected_usd / 10_000)
               +  frequency_factor        // 1 rare … 5 every-keystroke
               +  (is_regression ? 15 : 0)

# S0 with one reporter:  100 × (0.001 + 0 + 1 + 0)  ≈ 100
# S3 with 400 reporters:   2 × (4   + 0 + 3 + 0)    ≈  14
# the catastrophe wins and the math shows why

Timebox investigation, escalate by ruleremove the per-ticket judgment tax

Two disciplines keep the rubric fast under load. First, timebox: give yourself a fixed window to reproduce and isolate before you either escalate or downgrade, so one gnarly bug doesn't eat the day the other forty needed. Second, write down the escalation line in advance so paging an engineer is a rule, not a fresh agony each time.

Three hundred reports rarely mean three hundred bugs. They usually mean a dozen real problems, each described thirty different ways by people who don't share your vocabulary.

Clustering is the step that makes everything downstream honest. If you prioritize a raw inbox, you reward whichever bug happens to be described in the most tickets, not the one doing the most harm. The skill is collapsing reports onto the underlying defect, then counting the cluster - that count is what makes a priority number defensible.

Cluster on signals, not on phrasingusers describe the same bug a dozen ways

• Symptom shape - “Tab stopped suggesting,” “autocomplete is dead,” and “no ghost text” are one cluster.
• Error signature - the same stack trace, error code or log line is the strongest possible join key.
• Version - pin every report to the Cursor build and, separately, the model version in use.
• Feature area - Tab, Agent, ⌘+K, chat, @-context, indexing, the model picker, privacy mode.

Canonical issue plus linked duplicatescount once, count right

Pick one canonical issue per cluster and link every duplicate to it. Resist the urge to just close dupes - each linked report is a unit of volume and a different repro environment and the live count is what you'll cite to engineering. A cluster of 1 and a cluster of 600 need the same canonical record but produce very different priorities.

1. 1Open or find the canonical. One issue per real defect, named by symptom and area, not by the loudest ticket.
2. 2Attach evidence. Roll the strongest repro, logs, versions and affected accounts up to the canonical.
3. 3Link, don't delete. Point duplicates at the canonical so volume and ARR stay countable.
4. 4Recount on every new report. Volume, ARR affected and trend direction update live - that's the prioritization input.

New bug or regression? Correlate with the timelineCursor ships fast, so regressions are constant

The single most useful question about a spike is “what changed?” Overlay the cluster's first-seen timestamps on two timelines: app releases and model-version swaps. A cluster that appears the hour a new build rolls out is a regression and jumps the queue. A cluster smeared evenly across weeks is a long-standing issue you're only now seeing clearly.

The JD doesn't ask you to triage by hand faster. It asks you to drive bug prioritization at scale using agent-assisted tools - meaning you point Cursor and LLM agents at the firehose and supervise, rather than reading every ticket yourself.

This is the line that separates a strong candidate from a generic support hire. Cursor is an AI-native company building an AI-native product and they expect the quality function to be AI-native too. The interview reads your judgment with agents: do you wield them aggressively and do you reject bad output instead of pasting it into a brief?

The pipeline you should be able to describeingest → classify → cluster → rank → brief

1. 1Ingest. Pull raw signal from support tickets, the forum, Discord, social and in-product feedback into one store.
2. 2Classify. An LLM tags each report with feature area, candidate severity and sentiment - the boring labeling that used to eat hours.
3. 3Cluster. Embed reports and group by similarity, joining on error signatures where present, to collapse dupes automatically.
4. 4Rank. Apply the severity × impact score per cluster, with volume and ARR joined from account data.
5. 5Brief. The agent drafts the weekly quality brief: top clusters, trend arrows, suspected regressions - you edit and own it.

What the agent is genuinely good atthe heavy lifting worth delegating

Two failure shapes recur when an agent writes its own checks: it logs a failure instead of actually failing the run, or it wraps the assertion in a try-catch that swallows the error. The fix is engineering discipline. Keep the agent's context lean and pointed at the one task - a long, cluttered conversation can overwrite your instructions as the window fills - and run a validator step that executes the check, reads the output and reruns to confirm a real failure surfaces before you rely on it.

Build lightweight tooling, don't wait for a platformscripts and small dashboards beat a roadmap ticket

A Product Quality Engineer at a ~300-person hypergrowth company won't get a polished internal platform handed to them. The expectation is that you build the scrappy thing yourself - a script that pulls tickets, calls a model to classify and cluster and writes a ranked Markdown brief you can paste into the channel.

import { tickets } from "./ingest";        // tickets from support, forum, Discord
import { classify, cluster, draftRepro } from "./agent";

// 1. agent classifies each report - you set the schema, it fills it in
const tagged = await classify(tickets, {
  area: ["tab", "agent", "cmd-k", "chat", "context", "indexing", "model-picker"],
  severity: ["S0", "S1", "S2", "S3"],
  sentiment: ["angry", "confused", "neutral"],
});

// 2. group by semantic similarity, join on error signature when present
const clusters = await cluster(tagged, { joinOn: "errorSignature" });

// 3. score per cluster - the human-owned formula, not the model's guess
const ranked = clusters
  .map((c) => ({
    ...c,
    priority: sevWeight(c.severity) * impactScore(c),  // your rubric
    repro: draftRepro(c.canonical),                     // agent drafts, you verify
  }))
  .sort((a, b) => b.priority - a.priority);

// 4. you review ranked[0..n], confirm the repros, then ship the brief

Mine the backlog, rank it, then file the ticketsone plan-mode prompt does the triage and the paperwork

The same agent loop works against your own codebase, not just the inbox. Point Cursor's plan mode at the FIXME and TODO comments rotting in a service and let it produce the ranked test plan you'd otherwise spend a meeting arguing about. Plan modeA mode that makes no edits: it researches the codebase and produces an editable plan you review before any code changes. runs search-and-retrieval first, then writes a Markdown plan you refine before any code touches the repo - so the ranking is reviewable, not a black box.

Find all my FIXME and TODO comments in the alerting module.
Determine the top five most critical. Create a detailed testing
plan to make sure behavior stays consistent, then migrate to the
new functionality while making sure nothing else breaks.

Plan modeA mode that makes no edits: it researches the codebase and produces an editable plan you review before any code changes. came back with the items already banded P0-P2, each carrying a problem statement and a per-item unit-test plan - the kind of line that reads like triage you'd defend: “notification policy reset silently swallows errors, so users think the reset succeeded when it failed.” That is severity reasoning, written for you, on real backlog debt.

A generic QA person triages crashes and 500s. Cursor's product breaks in ways that don't throw an exception at all - the diff is just wrong, the context was stale, the model quietly got worse. Naming these classes on sight is what marks you as a power user who debugs like an engineer.

The product round and the debugging screen both probe whether you actually understand how an AI-native desktop app fails. Memorizing a taxonomy isn't enough; you should be able to map a vague report onto a failure class fast, because that class points at the right logs and the right team.

The five failure classesand the signal that names each one

The fifth class is the AI-native onemodel regression

The class a non-AI QA background will miss entirely: a model regression. A model upgrade or a backend routing change silently shifts behavior, diff quality or latency while the app version is unchanged. Nothing crashes. Users just feel that “it got dumber,” and your only tell is a behavior change correlated with a model swap rather than a release.

The behavioral and hiring-manager rounds want to know you'd build the system, not just work the tickets. That means having a clear answer to “how would you know if Cursor's quality is improving?” - with metrics, SLAs tied to severity and trends rather than snapshots.

A quality program is judged on two layers of metrics. Operational metrics tell you whether the response machine is working. Quality metrics tell you whether the product is actually getting better. Confuse them and you can have a beautifully fast support team shipping users a steadily worse product.

Operational metrics: is the machine working?how well we respond

Quality metrics: is the product getting better?what the program exists for

• Top issue themes by volume and ARR - the ranked clusters and which money sits behind them.
• Regression count per release - are we shipping new breakage as fast as we fix old?
• Repeat-contact rate - users coming back about the same theme means it isn't really resolved.
• Theme trend arrows - is each top cluster shrinking or growing week over week?

Tie SLAs to severity, not to a flat clockthe worst issues get the fastest clocks

A flat “respond within 24h” SLA is malpractice for a quality program - it gives a cosmetic typo the same clock as a data-loss bug. Anchor each SLA to the severity tier so urgency and the rubric stay aligned.