Deep Dive: Foundations & Training Systems

Cursor's Research Scientist loop opens with fundamentals interviewers expect cold. If you stumble deriving attention or explaining why the KV cache eats inference memory, the rest of the conversation never reaches the interesting RL questions.

The bar here is not reciting the Attention Is All You Need abstract. It is reconstructing a transformer block from the matmuls up, knowing which design choices the ComposerCursor's own fast coding model, tuned for the editor and priced well below frontier models; the recommended day-to-day model for executing a plan. model family actually depends on and being able to reason about where the compute and the memory go when you serve it to millions of developers.

Self-attention from the matmuls upQ, K, V and why we scale

Each token's embedding is projected three ways: a query (what am I looking for), a key (what do I offer) and a value (what I pass along if attended to). The attention weight from token i to token j is the dot product of query i and key j, softmaxed across all keys, then used to mix the values.

import torch, torch.nn.functional as F

def attention(x, Wq, Wk, Wv, mask=None):
    Q, K, V = x @ Wq, x @ Wk, x @ Wv      # each (B, T, d_k)
    d_k = Q.size(-1)
    scores = (Q @ K.transpose(-2, -1)) / d_k ** 0.5   # (B, T, T)
    if mask is not None:                  # causal: upper triangle = -inf
        scores = scores.masked_fill(mask, float('-inf'))
    w = F.softmax(scores, dim=-1)         # rows sum to 1
    return w @ V                          # (B, T, d_k)

The / sqrt(d_k) is the part interviewers probe. Without it, dot products grow with dimension, so logits land far out on the softmax where gradients vanish and one key dominates. Dividing by sqrt(d_k) keeps the variance of the scores near 1 regardless of head width, which keeps the softmax in a regime where gradients actually flow.

From one head to a blockmulti-head, residuals, norm placement, FFN

Multi-head attention runs several of these in parallel on lower-dimensional slices, then concatenates and projects back. Splitting the representation lets different heads specialize, one tracking syntax, another long-range references, without growing the total compute.

The block is: x = x + attn(norm(x)) then x = x + ffn(norm(x)). Post-norm wraps the norm around the residual sum instead, which was the original design and is harder to train deep without careful warmup. If you say "pre-norm" and can explain why (gradient stability at depth), that reads as someone who has actually trained these.

Positional informationRoPE vs learned/absolute

Attention is permutation-invariant on its own, so position has to be injected. Learned absolute embeddings add a per-position vector and break the moment you exceed the trained length. RoPE (rotary position embeddings) instead rotates the query and key vectors by an angle proportional to position, so attention depends on the relative offset between tokens.

Decoding and the KV cachewhere inference memory actually goes

Generation is autoregressive: one token at a time, each conditioned on all prior tokens through a causal mask. Recomputing every prior key and value at each step would be quadratic, so you cache them. The KV cache stores the keys and values for every past token across every layer and head.

"Train with less compute" is in the job description and scaling laws are the language for spending a fixed FLOP budget well. Expect to be asked, on a whiteboard, how you'd allocate a budget between a bigger model and more data.

The headline result is Chinchilla: for a fixed training-compute budget, loss is minimized by scaling parameters and training tokens together, roughly in proportion, rather than pouring everything into a giant under-trained model. The earlier GPT-3-era models were large and badly under-trained on data.

The power law, in wordsloss vs compute

Test loss falls as a power law in compute: each doubling of FLOPs buys a steady, diminishing fractional improvement and on a log-log plot the loss-vs-compute curve is close to a straight line. The practical consequence is that you can fit the trend on small runs and predict where a large run will land before you spend the budget.

Compute-optimal is not the same as deployment-optimalthe inference twist

Chinchilla minimizes training loss per training FLOP. Cursor serves ComposerCursor's own fast coding model, tuned for the editor and priced well below frontier models; the recommended day-to-day model for executing a plan. to millions of sessions, so the bill that dominates is inference. That flips the calculus: a smaller model trained on far more tokens than Chinchilla-optimal can match a bigger one's quality while costing much less to serve per token.

A state-of-the-art coding agent isn't one training run. It's a pipeline where each stage constrains the next and Cursor research owns decisions across all of them - including which base model to even start from.

Cursor has published that Composer 2Cursor's in-house agentic coding model: frontier-level coding quality at high speed and low cost, built as a software-engineering specialist rather than a general-purpose model. continued pretraining on a code-heavy mix before any preference work, which is a real, defensible choice you can speak to. Knowing the order of stages and what each one is good at, lets you reason about where a problem you're handed actually lives.

1. 1Pretraining. Learn language and code from a broad corpus. Almost never something a single researcher reruns; it's the foundation everything else inherits.
2. 2Continued / mid-training. Composer 2Cursor's in-house agentic coding model: frontier-level coding quality at high speed and low cost, built as a software-engineering specialist rather than a general-purpose model. continued pretraining on a code-heavy mix to shift the base distribution toward real software before any alignment. This is where the model's coding priors are set.
3. 3SFT (supervised fine-tuning). Imitate curated demonstrations to install format, tool-call syntax and basic agent behavior. Cheap, stable, but capped by the quality of the demos.
4. 4RL (reinforcement learning). Optimize toward a reward beyond what any demonstration showed - passing tests, good multi-file edits, graded quality. This is the core charter of the role.
5. 5Deployment. Ship into real Cursor sessions, validate on real user data and feed signal (including online RL behind Tab) back into the next iteration.

SFT vs RL: what each is actually forthe distinction interviewers test

The crisp framing: SFT teaches the model how to respond in the right shape, RL teaches it to respond in a way that scores well. SFT is behavior cloning bounded by your demos; RL pushes past them toward whatever the reward rewards, which is also why a bad reward is dangerous.

Base-model selection as a research decisionComposer 2 chose Kimi K2.5

Cursor has stated Composer 2Cursor's in-house agentic coding model: frontier-level coding quality at high speed and low cost, built as a software-engineering specialist rather than a general-purpose model. was built on Kimi K2.5The open base model Cursor continued-trained into Composer 2 (1T parameters, 32B active, 256K context), chosen mainly for how well it fit Cursor's serving infrastructure., chosen on concrete internal criteria rather than a leaderboard number. That's the texture to bring: base selection is an experiment, not a vibe.

Kimi K2.5The open base model Cursor continued-trained into Composer 2 (1T parameters, 32B active, 256K context), chosen mainly for how well it fit Cursor's serving infrastructure. was also picked because its architecture fit Cursor's infrastructure. The shapes that matter at training and serving time - the MoE layout, the attention scheme, the native context window - were already close to what the ComposerCursor's own fast coding model, tuned for the editor and priced well below frontier models; the recommended day-to-day model for executing a plan. stack runs, so adopting it meant less infra surgery before the interesting work started.

Picking the base is only the start. Cursor continued pretraining on top of K2.5 to raise its code-domain knowledge before any preference work, and the order of that continued run is itself a design choice.

1. 1Short-context continued pretraining. Keep the context window modest at first and pour code-heavy data through it to lift the base's coding knowledge cheaply, before paying the cost of long sequences.
2. 2Long-context extension to 256K. Extend the window out to the native 256K (RoPE base/NTK scaling territory) so the model holds a real repo, tool outputs and many turns of history in one session.
3. 3SFT on agent-like data. Fine-tune on demonstrations shaped like real Cursor agent traces - tool calls, multi-file edits, test runs - so the model arrives at RL already behaving like an agent, not a chat model.

Cursor frames its researchers as people who build the infra their experiments need, not just write loss functions. ComposerCursor's own fast coding model, tuned for the editor and priced well below frontier models; the recommended day-to-day model for executing a plan.'s training stack - MoE, custom low-precision kernels on Blackwell, async multi-region RL, sandboxed rollouts - is fair game in the interview.

You don't need to have written a CUDA kernel to pass, but systems literacy is part of the bar. You should know why MoE buys capacity, where low-precision risks numerical blow-ups and which parallelism axis bottlenecks an RL rollout-plus-train loop.

Mixture-of-Expertscapacity per FLOP

A dense model runs every parameter on every token. An MoE replaces the FFN with many expert MLPs and a router that sends each token to only a few of them. Cursor has described ComposerCursor's own fast coding model, tuned for the editor and priced well below frontier models; the recommended day-to-day model for executing a plan. as a large MoE with a small fraction of parameters active per token, so total capacity is huge while the FLOPs per token stay affordable.

Low-precision kernelsthroughput vs numerical stability

Cursor has said it wrote custom low-precision kernels to train MoE models on NVIDIA's Blackwell hardware. Lower precision (FP8 and below) roughly doubles throughput and halves memory traffic, but the numerics get sharp: small values underflow, large ones overflow and a single bad reduction can wreck a step.

Parallelism and the RL loop bottleneckdata / tensor / pipeline / expert

In RL the loop is generate rollouts, score them, then train on them - and the rollout (inference) phase often dominates wall-clock, because generation is sequential and the policy keeps changing. The async trick is to decouple actors from learners: rollout fleets generate against a slightly stale policy while learners update, instead of letting expensive GPUs idle between phases.

Cursor screens hard for truth-seeking: caring more about what's true than about being right. Experimental rigor is where that trait becomes operational and it's the easiest place to lose a research deep-dive if your conclusions don't hold up under questioning.

The deep-dive round is you presenting prior work and defending the decisions under pressure. The paid practical onsite is you running a real experiment and reasoning about noisy metrics live. Both reward the same habits: isolate variables, quantify uncertainty and predeclare what would prove you wrong.

Isolate one variableno confounds

The most common self-inflicted wound is changing the method and the data and the eval at once, then crediting the win to the method. If two things move together, you've learned nothing about either. Change one thing, hold a matched baseline fixed and keep the eval identical across arms.

• Run a matched baseline under the exact same data, eval and compute - not last quarter's number from a different setup.
• Change exactly one factor per comparison; if you must change two, run the 2x2 so you can attribute the effect.
• Freeze the eval set and harness across arms; an eval that drifts between runs invalidates the comparison.

Quantify uncertaintyone run is not a result

RL training is high-variance: seed, data shuffle and rollout sampling can swing a SWE-bench-style number by more than the effect you're chasing. A single lucky run is a story, not a finding. Report across seeds with a spread and ask whether your claimed delta clears the noise floor.

Leakage and reward hackingthe two ways a coding eval lies

Predeclare the kill criteriontruth-seeking, made concrete

Before the run, write down the result that would make you abandon the idea. Deciding the bar in advance defends you against moving the goalposts when you're emotionally invested, which is the whole point of truth-seeking as a discipline rather than a slogan.

1. 1State the hypothesis. A specific, falsifiable claim: 'grader G raises live edit-accept rate by >=2pts over the baseline.'
2. 2Define the kill criterion. The result that ends the idea: 'if the seed-averaged delta is under 1pt or live impact is flat, I stop.'
3. 3Pre-register the eval. Fix the eval set, seeds and metric before looking, so you can't retrofit a win.
4. 4Run, then report honestly. Show the spread, name confounds you couldn't rule out and state what you'd do next - including killing it.