2 min lesson
A worked skeleton
Make the call in this situation: "You've built a cost model under time pressure. What's the single most valuable thing to do before presenting the headline number?" Explain what supports it.
Step 1 of 2
A worked skeletonthe shape of a defensible model
# --- INPUTS (known | estimated) --- gpu_hourly_cost = 2.50 # known: reserved instance rate gpu_throughput_tok_s = 3000 # known: from serving benchmarks util_target = 0.55 # estimated: sustained, post-batching active_users = 1_000_000 # scenario: assumed scale, not a Cursor fact reqs_per_user_day = 40 # estimated: blended tab + chat tokens_per_req = 350 # estimated: blended in+out days = 30 # --- DERIVE cost per token --- eff_tok_per_hour = gpu_throughput_tok_s * 3600 * util_target cost_per_token = gpu_hourly_cost / eff_tok_per_hour # ~ $4.2e-7 # --- ROLL UP to monthly COGS --- tokens_month = active_users * reqs_per_user_day * tokens_per_req * days monthly_cogs = tokens_month * cost_per_token # the headline # --- SENSITIVITY: the 2-3 that dominate --- # util_target 0.45 -> 0.65 : COGS swings ~30% # model_mix (share routed to large model) : each +10pt ~ +X% COGS # reqs_per_user_day +/- 20% : linear pass-through to COGS
Notice utilization sits inside cost per token, so it's the most impactful input in the whole model. A model running at 55% vs. 35% utilization changes the headline by more than most feature decisions ever will. That's the variable to sensitize first and to interrogate hardest.
Take the headline COGS and divide by active users - does cost per active user land in a sane range for a flat-priced seat? If the model says $200/user/month against a $20 price, you have a bug or a business that doesn't exist. The back-of-envelope cross-check is a 30-second habit that catches the embarrassing 10x errors.
Narrate the structure as you build: "I'll anchor on cost per token, label inputs as known or estimated, sensitize on utilization and model mix, then land one recommendation." Saying the skeleton first signals you've shipped models before. Then end with a decision - "reserve 60% of baseline, burst the rest" - and state what would change your mind. Interviewers remember the candidate who concluded.