Study Design & Method Fit

The research-challenge round rarely fails on method. It fails when a candidate accepts the question as handed to them and never reframes it.

A PM at Cursor walks up and says “should we build a chat-history panel?” That is not a research question. It is a business question wearing a feature as a costume. Your first job, before you name a single method, is to find the decision underneath it and the unknown that would actually move that decision.

Senior researchers separate three things that beginners blur together. Keep them on three different lines in your head, because the panel will watch whether you can.

Hunt the riskiest unknownresearch the thing that would change the decision

Every ask carries hidden assumptions. List them out loud, then rank them by two factors: how uncertain you are and how badly the project breaks if the assumption is wrong. The cell that is both high-uncertainty and high-consequence is your study. Everything else can wait or be assumed.

1. 1Surface the assumptions. Write the implicit beliefs as falsifiable claims: “developers want to revisit old chats,” “they can't find them today,” “a panel is the right surface.”
2. 2Rate uncertainty x consequence. A belief you are confident in needs no study. A belief that is shaky and load-bearing is where you spend the budget.
3. 3Pick the one unknown. Name the single question whose answer would most change what the team builds and design for that. Scope is a choice, so make it visibly.
4. 4Tie it to a decision date. If no decision rides on the answer, it is curiosity, not research. Park it.

Method fit is a two-question test: are you trying to learn what to build or check something you've already built? And do you need what people say or what they do?

Get the first question wrong and every downstream choice is off. Generative research expands the space of what you might build. Evaluative research narrows or validates something already on the table. Bringing a usability test to a generative question gives you a polished answer to a question nobody needed.

The two axes that place every methodwhat people say vs what they do; depth vs scale

Lay methods on a grid. The vertical axis is attitudinal (what people say, believe, prefer) versus behavioral (what they actually do). The horizontal axis is qualitative depth versus quantitative scale. A method's position tells you instantly what it can and can't see.

The Cursor tilt: lightweight and behavioral firstkeeping pace with a fast AI-tool team

Cursor ships fast and instruments heavily, so the cheapest signal is often already in the logs. Before you recruit anyone, you can see retry rates, accept rates on Tab completions and where agent runs stall. Lean on behavioral analytics and small evaluative studies to keep cadence and reserve heavyweight generative work for the genuinely open questions where the logs go quiet.

Cursor is a small, flat research function on a product that ships weekly. The bar is not the most rigorous study. It is the smallest study that can still change the decision, run before the decision is made.

“Scrappy” gets misread as “sloppy.” It isn't. Scrappy means you cut the parts of the study that don't change the answer and keep the parts that do. The skill the panel is grading is whether you know the difference.

Default to the smallest decisive studyright-size n to the question, not to a habit

Trade timeline, sample and depth out loudevery fast study gives something up

There is no free speed. When you compress a study you sacrifice one of three things and a senior researcher names which one. The panel wants to hear the tradeoff, not pretend it doesn't exist.

Use existing data before you recruitthe fastest study is the one you don't run

• Logs and product analytics. Accept rates, revert rates, where agent runs stall, funnel drop-off. Often answers the “how widespread” question for free.
• Support tickets and forum/Discord threads. A pre-tagged corpus of real friction in users' own words; great for generating interview hypotheses.
• Prior studies and the research repo. Someone may have already answered part of this. Reusing it is speed, not laziness.
• Sales and CS notes. For enterprise questions, the field team has heard the objection a hundred times before you frame the study.

In dev-tools research, the study design is easy and recruiting is the bottleneck. Treat recruiting as a first-class design problem or your beautiful plan dies waiting for participants.

Cursor's users are professional developers: time-poor, skeptical of being “researched,” and immune to a $25 gift card. They don't answer panel screeners and they can smell a generic study. Your sourcing and screening choices shape the validity of everything downstream, so they belong in the plan, not in an afterthought.

Where the right developers come fromeach source carries its own bias

Screen for the segment that answers the questionspecificity beats volume

• Language / stack. A Rust systems engineer and a frontend React developer experience the AI differently; decide which one your question is about.
• Team size and setup. Solo dev, small startup and large-org with review gates have different needs from the same feature.
• AI-tool adoption level. Daily Cursor power user, occasional user, evaluator and skeptic each tell a different and necessary story.
• Role. IC, tech lead and platform/DevX owner buy and use the tool for different reasons.

Incentives and consent for a professional audiencerespect their time and their NDAs

A senior engineer's hour is worth more than a consumer's, so a token gift card can insult more than it attracts. Right-size the incentive (meaningful honorarium, donation option or early-access perk) and, for enterprise developers, plan around their employer's confidentiality. Get clear consent for recording and for any code or screen they share, because they will be sharing proprietary work.

The research-challenge round ends with you defending a plan to a room of other researchers. They are not grading the plan in isolation. They are grading how you think when they push on it.

Give the panel a spine they can follow without taking notes. A predictable structure frees their attention for the reasoning instead of the bookkeeping and it signals you've presented research to busy stakeholders before.

1. 1Question. State the decision and the one unknown you're resolving. Thirty seconds, no preamble.
2. 2Method + why. Name the method and justify it against the question, not against your comfort zone.
3. 3Participants / sampling. Who, how many, recruited from where and the bias you accepted.
4. 4Timeline. The calendar, including the fast version if asked.
5. 5Analysis. How raw data becomes a finding: your thematic approach, your stats, your triangulation.
6. 6Expected outcomes + risks. What you expect to learn, what could go wrong and what you'd do about it.

Lead with the tradeoff, not around itname what you optimized for and what you sacrificed

Every defensible plan optimized for one thing at the cost of another. Say it before they find it. “I optimized for speed and depth on the mechanism and I gave up prevalence: this won't tell us how common the problem is, only that it's real and why.” Owning the limitation removes the panel's best line of attack and turns a weakness into evidence of judgment.

Show the decision treeevery finding maps to an action

The strongest signal you can send is connecting findings to decisions before you've gathered the data. Walk the panel through what each plausible result would mean for the team. This proves the study is designed to drive a decision, not to generate a report nobody reads.