1 min lesson
Use offline and online evidence
Combine repeatable offline checks with production outcomes and explicit rollout guardrails.
Step 1 of 3
Cursor's current product-engineering role lists experiments and A/B tests on millions of users as sample work. Cursor also documents an offline suite, CursorBenchCursor's internal benchmark that scores models on both task performance and token efficiency, not accuracy alone. Press Enter for the full definition., and controlled online experiments for agent and retrieval changes.
No single metric captures agent quality. Use offline correctness and behavior checks for fast comparison, then confirm changes with online outcome signals and latency, cost and reliability guardrails.