AI & Agent Security Threat Model

An autonomous coding agent reads your repo, writes files and runs commands on your machine. The clean line every classic security model draws - between data the system processes and code the system trusts - is gone. The data IS the instructions now.

This is the round that separates a Cursor security interview from any other. The work spans web vulns, cloud IAM and crypto, but the part interviewers will probe hardest is the one no off-the-shelf playbook covers: the threat model of an agent that acts on the user's system. Cursor builds exactly that product and it secures its own codebase the same way, so vague answers get caught fast.

Start by naming the shift cleanly. A traditional app has a fixed set of code paths an attacker tries to subvert with crafted input. An agent has an open-ended action space driven by a model and the model treats almost everything it reads as potential instruction.

The lethal trifectathe condition that turns a quirk into an exfiltration

The cleanest mental model for when an agent becomes dangerous is the combination of three capabilities. Any one alone is survivable. Together they let untrusted text steal data.

Your job as a security engineer is to break the trifecta. You usually can't drop the first two without killing the product, so most paved-road defenses attack the third and constrain the second: cut the exfil channels and quarantine or label untrusted content so the model is less likely to obey it.

Untrusted input is everywherethe surface is wider than candidates expect

• Repo contents - a comment, README or test fixture can carry an injected instruction the agent reads while working.
• Web pages and search results - the moment the agent browses, it ingests attacker-controlled HTML.
• Tool and command outputs - a linter, a curl response or a build log the agent reads can be poisoned.
• MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. servers - every connected server returns content the model treats as context and you may not control that server.
• Prior agent turns - output the agent generated earlier can re-enter context and reinforce a hijack.

Prompt injection is SQL injection for the model era, except there is no parser you can parameterize. The instruction and the data share one channel - natural language - and the model has no reliable way to tell them apart.

Expect the agent-security round to hand you a scenario and ask where the injection lives and what it can do. The high-signal distinction to lead with is direct versus indirect.

There's a one-line root cause worth saying out loud: the models have no separation of control and data. They don't understand the different trust levels of whoever is feeding them information. A README, a web page and your own instructions all arrive as the same kind of token. Security has made this exact mistake before, several times, and the model era makes it again.

Indirect injection is what makes the lethal trifecta lethal. The attacker writes a payload into a file, a web page, an issue or a tool's output, then waits for an agent with private-data access and an exit channel to read it.

Why you cannot prompt your way outthe load-bearing point of this whole topic

Candidates lose this round by claiming a stronger system prompt fixes it - "just tell the model to never follow instructions in files." That is not a security control. It is a request to a probabilistic system that an attacker gets to add text to.

Mitigations that actually bound impactprivilege separation over persuasion

Since you can't trust the model to disobey, design so that a hijacked agent can't do much. Each control assumes the injection already succeeded and limits the blast radiusHow much breaks if a change goes wrong; the scope of potential damage..

1. 1Privilege separation. The agent runs with the least authority the task needs. Reading code to summarize it should not carry the right to make network calls or write outside the workspace.
2. 2Human-in-the-loop for dangerous actions. Destructive or irreversible operations - deleting files, pushing to remote, running an arbitrary shell command - require explicit user confirmation, not an autonomous yes.
3. 3Output constraints and exfil starvation. Strip or sandbox the channels an injection would use: auto-rendered links and images, outbound requests to non-allowlisted hosts, writes to paths outside the project.
4. 4Allowlisting tools and commands. The agent can call a vetted set of tools and run a vetted set of commands. Anything off the list is denied or escalated to the user, so a hijack can't reach for curl or rm -rf.
5. 5Detection and clamping. Flag content that looks like injection (imperative text aimed at the model, base64 blobs, instructions to ignore prior context) and reduce what the agent may do for the rest of that turn.

const SAFE_TOOLS = new Set(["read_file", "search", "list_dir"]);
const DANGEROUS = new Set(["run_shell", "write_file", "http_request"]);

async function dispatch(call: ToolCall, ctx: AgentContext) {
  if (SAFE_TOOLS.has(call.name)) return run(call);
  if (DANGEROUS.has(call.name)) {
    // Confirmation is enforced in code, regardless of what the model 'decided'.
    const ok = await ctx.confirmWithUser(describe(call));
    if (!ok) return deny(call, "user declined");
    return run(call);
  }
  return deny(call, "tool not allowlisted");
}

Every tool you hand the agent is a capability an attacker can borrow. A web-fetch tool is an exfil channel. A shell tool is arbitrary code execution. File-write is persistence. The security work is deciding what to grant, how narrowly and under what gate.

The JD calls this out directly: build the framework for securely handling agent manipulation of user systems - tool use, MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs., command execution, file writes. This section is where that responsibility lives and an onsite build round may ask you to sketch it.

Treat the tool set as your attack surfaceminimize and scope before you harden

The first move is reduction. Fewer tools, each with the narrowest contract, beats a broad toolbox with elaborate runtime checks. You can't abuse a capability that was never granted.

Capability-based designgrant what the current task needs, nothing standing

The same least-privilege and just-in-time thinking you'd apply to cloud IAM applies inside the agent. The agent shouldn't hold a standing grant to every tool for the whole session. It should hold a capability scoped to the task in front of it and that capability should expire.

1. 1Default deny. No tool, command or path is reachable unless explicitly granted for this task.
2. 2Scope to the task. A "summarize this file" task gets read-only file access and nothing else; a "run the tests" task gets the test command and the workspace, not the network.
3. 3Time-box and revoke. Capabilities are just-in-time - granted when the task starts, dropped when it ends, so a hijack mid-session can't reuse an earlier grant.
4. 4Gate the irreversible. Network egress, destructive commands and writes outside the workspace always escalate to a human, independent of what the model proposed.
5. 5Log every call. Each tool invocation, its arguments and its decision (run / denied / confirmed) is recorded for audit and incident response.

type Capability = {
  tools: Set<string>;       // exact tools allowed this task
  fsMode: "read" | "write"; // write requires a gate per call
  workspaceRoot: string;    // all paths must canonicalize under this
  networkEgress: false;     // explicit; summarize/read tasks get none
  expiresAt: number;        // JIT: capability dies with the task
};

MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. - the Model Context Protocol - lets an agent connect to external tools and data sources through a standard interface. Each server you connect is a new mouth feeding the model context and a new hand the model can use. Each one is a trust boundary.

MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. is genuinely useful and genuinely dangerous, which is why it earns its own round-worthy discussion. A connected server returns content the model treats as context and exposes tools the model can call. If you don't control that server, you've extended the lethal trifecta to a third party.

Two distinct risks from one connectioningress instructions and egress data

Vetting third-party MCP integrationstreat every server like an untrusted dependency

The right analogy is supply-chain security. An MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. server is a dependency that runs with your agent's reach, so apply the same rigor you'd apply to a package you're about to import.

• Vet and pin the server - known publisher, reviewed behavior, pinned version, no silent updates that change what its tools do.
• Sandbox and least-privilege it - the server's tools get only the capabilities that integration needs and its outputs are treated as untrusted content, not trusted instruction.
• Bound the egress - what the model is allowed to pass into a server's tool call is constrained, so it can't quietly hand over secrets or whole files.
• Label provenance - content that came from an external MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. server is marked as untrusted in context so downstream controls can clamp on it.

Model context is itself sensitive datawhat goes in is what can leak out

Whatever lands in the model's context - code, secrets that slipped into a file, internal API responses - can leave through any tool call or any provider request. Minimizing context isn't just a cost lever, it's a data-protection control. The less you put in, the less there is to exfiltrate.

The code and prompts an agent sends to a model are some of the most sensitive data a company has. Once they cross the boundary to a provider, the questions are: does the provider train on them, how long are they retained and could one customer's data ever surface in another's session.

This section closes the loop on the trifecta's exit channel and adds the data-protection layer the JD calls out: logging and retention safeguards that prevent inappropriate retention or leakage of customer and code data. Be ready to reason about both the obvious channel - the provider request - and the sneaky ones an injection drives.

Provider controls: contractual and technicalpolicy alone isn't a control

Controlling what leaves the boundary takes two layers working together. A contract that says "we won't train on your data" is necessary but not sufficient; you want technical enforcement behind it.

Cross-tenant leakageone customer's context must never reach another's

The failure that ends a B2B security review is one tenant's code or context bleeding into another tenant's session. Caches keyed without a tenant dimension, shared embeddings and logs that mix customers are the usual culprits. Tenant isolation has to be a property of the system, enforced in code, not a convention people remember to follow.

Exfiltration channels an injection will usethe trifecta's exit, concretely

When an indirect injection succeeds, it needs a way out. These are the channels it reaches for and each maps to a control from the earlier sections.

The same agent that's a threat surface is also the best security teammate you've had. The whole skill is driving it skeptically: it's fast at search and great for ramping into an unfamiliar system, and it will also confidently report findings that aren't real. You hold it to ground truth.

This is how a security FDE at Cursor actually works the tool day to day. It maps cleanly onto the interview: the threat model says assume the model can be fooled, and this section is the same suspicion turned into a working loop you run on a real codebase.

Four tenets for security work with an agentthe mindset before the workflow

• Models need context, not prompt engineering. Treat the agent like a delegate: be clear about what the work is and what good looks like. You no longer have to spend hours crafting the perfect prompt.
• Assume the model might be lying. Build every workflow around the idea that any single fact could be wrong. Security people are uniquely good at this kind of suspicion.
• Ask for proof. When in doubt, push: show me the code, show me the snippet, make me an architecture diagram. Don't accept a claim you can't see evidence for.
• No dumb questions. Agents are great for ramping into systems you don't know and extremely fast at search, so use them to learn the terrain before you judge it.

The "I don't believe you" validation loophow you kill false positives

When the agent reports a finding - say it flags a P0 - don't take it. Copy the finding back to the agent and tell it, in effect, that you don't believe it. Ask for the attack chain in depth, the relevant code snippets, exactly how it would be exploited and a proof that the exploit actually works. Push it down to docs and code until model and human reach the same ground truth.

Fan out sub-agents per vulnerability classparallel coverage, then validate each finding

For a thorough pass, kick off one agent that spawns sub-agents grouped by vulnerability class - infra, AppSec, prompt injection, XSS and so on. Each sub-agentA child agent a main agent spawns to work in parallel with its own context window, handing results back so the parent's context stays clean. produces a table of findings, and for every finding it has to show the full attack chain with code snippets plus a simulated proof-of-concept. Sub-agents keep distinct vuln classes in separate context, and they're the right tool for the validation loop. Run that loop and the sub-agents often realize a bunch of their own findings were wrong.

1. 1Kick off a lead agent and have it spawn sub-agents, one per vulnerability class you care about (infra, AppSec, prompt injection, XSS, and so on).
2. 2Make each sub-agentA child agent a main agent spawns to work in parallel with its own context window, handing results back so the parent's context stays clean. produce a table of findings - not prose, a table you can scan.
3. 3For every finding, require the full attack chain with code snippets and a simulated proof-of-concept before it counts.
4. 4Run the validation loop on each one; expect the sub-agents to retract a chunk of their own findings.
5. 5For the classes you most care about, run parallel passes so a single missed sweep doesn't become a false negative.

Which model, and how to prompt ita daily driver, told to self-validate

On model choice, be a daily driver rather than a constant switcher. Staying on one model for a while - Opus 4.5 has been a solid daily driver for security work - builds an intuition for how it approaches problems and where it gets stuck. Evaluate new models when they ship. Opus is very good at thinking about security if you tell it to, and it does an even better job if you give it the validation loop in the prompt. Some GPT models (GPT-5.2 among them) are very good at hard problems but slower, so reach for those when you want in-depth work. This is one practitioner's read, not the Cursor opinion.

Everything so far has been principle: assume the injection succeeds, bound the blast radiusHow much breaks if a change goes wrong; the scope of potential damage. in code, starve the exit channels. This section names the actual Cursor enterprise controls that implement those principles, so when an interviewer asks "how would you enforce that?" you answer with a real mechanism, not a wish.

Four controls do most of the work: hooks, the model and MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. allowlist, Sandbox ModeAn isolated agent execution mode that contains the blast radius: file access scoped to the workspace, network off by default and git restricted to read-only, enforceable org-wide. and three-layer egress. Each one is a hard control - it lives in the code or the org config around the agent, where an attacker who only controls text can't reach.

Hooks: deterministic checkpoints in the agent loopyour code intercepts the loop at named lifecycle events

Hooks are scripts that fire at fixed points in the agent loop. They run your code, not the model's, so the decision they make is deterministic. Where a system prompt asks the model to behave, a hook enforces behavior regardless of what the model decided.

Read each checkpoint as a place to attach one of the earlier controls. before-submit-prompt is where you audit or scan a prompt for sensitive content before it crosses the provider boundary. before-file-read enforces a file-type or path policy before untrusted content enters context. before-shell-command is the one that matters most: chain a logger and an allowlist so any command outside the approved set is blocked. after-agent-response is your telemetry tap for what happened in the loop.

const ALLOWED = new Set(["npm test", "npm run build", "git status", "git diff"]);

// Fired by Cursor before any shell command the agent proposes.
export function beforeShellCommand(cmd: string) {
  audit.log("shell.proposed", { cmd });          // logger runs first
  if (!ALLOWED.has(cmd.trim())) {
    return { decision: "block", reason: "command not on allowlist" };
  }
  return { decision: "allow" };
}

Model & MCP allowlist: per-tool auto-run vs manual approvaldefault-deny applied to models and servers

The same default-deny capability thinking from the tool-use section applies to which models and which MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. servers the org permits. An admin sets an allowlist and a denylist, and - crucially - decides per tool whether it auto-runs or waits for manual approval. That second axis is what lets you keep a useful server connected without trusting it to act unattended.

Map this back to the trifecta. A third-party server that can post to prod is an exit channel you don't control, so the default-deny posture keeps it off until someone vets it. An internal CI server you do control can auto-run its safe tools while its one dangerous tool - the deploy - stays behind manual approval. The allowlist isn't a static wall; it's per-tool privilege separation.

Sandbox Mode: network off, git read-only, org-enforcedthe agent runs with no ambient reach by default

Sandbox ModeAn isolated agent execution mode that contains the blast radius: file access scoped to the workspace, network off by default and git restricted to read-only, enforceable org-wide. gives command execution the constrained environment the tool-use section called for, as a product default. Network access is off unless explicitly granted, git is read-only so the agent can inspect history without rewriting or pushing it, and an admin can enforce the sandbox org-wide so individual users can't quietly disable it.

Three-layer egress: allowlist, denylist, default-denyDNS filtering plus an HTTP proxy enforce it

When the agent does need network access, egress is controlled in three layers. An allowlist names the hosts traffic may reach, a denylist names hosts it never may, and anything matching neither hits a default-deny. Two enforcement points back the policy: DNS filtering resolves only permitted names, and an HTTP proxy inspects and gates the actual requests.

This is the tool-layer egress bound from the exfiltration section, made concrete. A markdown image or a hijacked curl that aims at attacker.example resolves to nothing under DNS filtering and is rejected at the proxy, because the host is on neither the allowlist nor anywhere it could be. Fail-closed is the property that matters - an unknown destination is denied, not allowed by omission.