Skip to lesson
Exit
Bug Triage & Prioritization at Scale1 / 2

1 min lesson

Build lightweight tooling, don't wait for a platform

Describe what "The expectation is that you build the scrappy thing yourself" changes in practice.

Step 1 of 2

Build lightweight tooling, don't wait for a platformscripts and small dashboards beat a roadmap ticket

A Product Quality Engineer at a ~300-person hypergrowth company won't get a polished internal platform handed to them. The expectation is that you build the scrappy thing yourself - a script that pulls tickets, calls a model to classify and cluster and writes a ranked Markdown brief you can paste into the channel.

Plan with the big model, build with the fast one

Split the model the way you'd split the work when you write that script. Reach for a larger reasoning model - the current frontier Claude Opus (Opus 5 replaced Opus 4.8) or a GPT-5.x - to draft the high-level plan, then drop to a leaner, faster in-house model like Composer 2.5The current Composer release, better at long-running tasks and at judging when a job needs a light touch versus deep work. Press Enter for the full definition. to do the building. After plan mode you pick your execute model and start building. You pay for heavy intelligence where the thinking is and let fast intelligence churn out the rest.

Learn more

Full explanation

A realistic, scrappy triage pass - the kind you'd build week one

A realistic, scrappy triage pass - the kind you'd build week onets
import { tickets } from "./ingest";        // tickets from support, forum, Discord
import { classify, cluster, draftRepro } from "./agent";

// 1. agent classifies each report - you set the schema, it fills it in
const tagged = await classify(tickets, {
  area: ["tab", "agent", "cmd-k", "chat", "context", "indexing", "model-picker"],
  severity: ["S0", "S1", "S2", "S3"],
  sentiment: ["angry", "confused", "neutral"],
});

// 2. group by semantic similarity, join on error signature when present
const clusters = await cluster(tagged, { joinOn: "errorSignature" });

// 3. score per cluster - the human-owned formula, not the model's guess
const ranked = clusters
  .map((c) => ({
    ...c,
    priority: sevWeight(c.severity) * impactScore(c),  // your rubric
    repro: draftRepro(c.canonical),                     // agent drafts, you verify
  }))
  .sort((a, b) => b.priority - a.priority);

// 4. you review ranked[0..n], confirm the repros, then ship the brief
Watch out

Don't let the agent own the score. The classify step can suggest a severity, but the priority formula and the final S0/S1 call stay human. An LLM that quietly downgrades a data-loss report to S2 because the user was polite about it is exactly the failure the loop is checking you'd catch.