Skip to lesson
Exit
Capstone: Mock Loop & Self-Exam1 / 2

1 min lesson

The prompt, stated the way they'll state it

Walk through each part of "The prompt, stated the way they'll state it", then explain what each one does.

Step 1 of 2

The prompt, stated the way they'll state itVague on purpose - scoping is part of the test

You're handed a month of usage: tab completions, chat turns and agent runs, each with a model tier and rough token counts. Inference is the dominant COGS line. Leadership wants 15% out of it next quarter.

What a strong submission contains
Cost driver tree
COGS decomposed to cost = volume × tokens-per-call × $/token, per feature and model tier - so every dollar has a named cause
Stated assumptions
$/token by tier, blended utilization, what you held fixed and where your numbers came from versus where you guessed
Sensitivity table
How the 15% target moves if token-per-call or tier-mix is off by ±20% - proves the recommendation survives being wrong
One-page recommendation
The two or three levers you'd pull, the expected savings, the product risk and what you would measure to confirm
Learn more

Full explanation

A 4-hour timebox that ends with both artifacts

A 4-hour timebox that ends with both artifactsPrioritize ruthlessly - finished beats elegant

  1. 10:00–0:30 · Frame and assume. Write the cost equation and list every assumption on one tab. State your unit: cost per active user is the metric leadership thinks in, so anchor there. Freeze scope - one month, three features, two model tiers - and push everything else to what I'd do with more time.
  2. 20:30–2:00 · Build the model. Attribute spend bottom-up: per feature, per tier, volume × tokens × $/token. Get the totals to reconcile against the stated COGS number. If they don't reconcile, find the gap before you go further; an unreconciled model is an untrustworthy model.
  3. 32:00–3:00 · Find the 15% and stress it. Identify where the cost concentrates, almost always a few high-volume or large-model paths. Build the sensitivity table. Pick the levers that survive the stress: model routing, prompt/context trimming, caching, tier downgrades for simple calls.
  4. 43:00–3:45 · Write the one-pager. Recommendation up top in two sentences, then the savings math, the product-risk line and the metric you'd watch. Write it for a CFO and an ML lead in the same page - neither should need a glossary.
  5. 53:45–4:00 · Reconcile and ship. Re-check that the model totals match, the assumptions tab is complete and the recommendation names a number. Submit the honest version, not the impressive-looking one.
The attribution core - a defensible cost-per-active-user build you can walk line by line
# Bottom-up COGS attribution. Every term traces to an assumption on the
# assumptions tab - that traceability is what makes the model trusted.

feature_cost = sum(
    calls[f, tier]                 # volume, from usage logs
    * tokens_per_call[f, tier]     # ASSUMPTION: measured median, not mean
    * price_per_token[tier]        # ASSUMPTION: blended in+out $/1M tokens
    for f in features for tier in tiers
)

cost_per_active_user = feature_cost / monthly_active_users

# The 15% lever, modeled explicitly so the saving is auditable:
# route "simple" tab/chat calls from the large tier to a small model.
routed_share        = 0.40        # ASSUMPTION: share of calls that are "simple"
small_tier_discount = 0.85        # small model ~15% the $/token of large
savings = (calls_large * routed_share
           * tokens_per_call_large
           * price_per_token_large * small_tier_discount)