2 min lesson
Where the bias hides
Answer this: "You launch an in-app survey via a banner and 2,500 developers respond. A colleague wants to report the satisfaction average as representing all Cursor users. What's the main risk?"
Step 1 of 3
Where the bias hideserrors that survive a clean-looking dataset
Earlier questions prime later answers
Asking about bugs first depresses later satisfaction
Randomize item and option order where you can
People tend to agree with statements
Inflates every “do you agree…” item
Mix positively and negatively keyed items
Who skips the survey isn't random
Frustrated power users may opt out entirely
Check response rate by segment, not just overall
Volunteers differ from the population
An in-app prompt over-samples active, happy users
Weight or caveat; don't generalize to churned users
Learn more
Full explanation
Sampling and how confident you get
Sampling and how confident you getwho, how many, how sure
Sample size is a precision dial, not a magic threshold. Bigger samples narrow the margin of error with diminishing returns and they never fix a biased frame. Decide who you need first, then how many.
- Who
- Define the target population and the frame you'll actually reach (in-app vs. emailed vs. all seats).
- How many
- For a single proportion, ~400 responses gives roughly a ±5% margin at 95% confidence.
- Segments
- You need that n per subgroup you want to compare, not in total.
- Bias beats size
- A representative 300 beats a skewed 3,000; coverage error doesn't shrink with n.
These figures are standard survey rules of thumb, not Cursor-specific numbers.