Skip to lesson
Exit
Capstone: Mock Loop & Self-Exam1 / 2

1 min lesson

The prompt, stated the way they'll state it

Walk through each part of "The prompt, stated the way they'll state it", then explain what each one does.

Step 1 of 2

The prompt, stated the way they'll state itVague on purpose, because scoping is part of the test

You're handed a month of usage: tab completions, chat turns and agent runs, each with a model tier and rough token counts. Inference is the dominant COGS line. Leadership wants 15% out of it next quarter.

What a strong submission contains
Cost driver tree
COGS decomposed to cost = volume × tokens-per-call × $/token, per feature and model tier, so every dollar has a named cause
Stated assumptions
$/token by tier, blended utilization, what you held fixed and where your numbers came from versus where you guessed
Sensitivity table
How the 15% target moves if tokens-per-call or tier mix is off by ±20%. It proves the recommendation survives being wrong
One-page recommendation
The two or three levers you'd pull, the expected savings, the product risk and what you would measure to confirm
Learn more

Full explanation

A 4-hour timebox that ends with both artifacts

A 4-hour timebox that ends with both artifactsPrioritize ruthlessly: finished beats elegant

  1. 10:00 to 0:30 · Frame and assume. Write the cost equation and list every assumption on one tab. State your unit: cost per active user is the metric leadership thinks in, so anchor there. Freeze scope to one month, three features and two model tiers, and push everything else to what I'd do with more time.
  2. 20:30 to 2:00 · Build the model. Attribute spend bottom-up: per feature, per tier, volume × tokens × $/token. Get the totals to reconcile against the stated COGS number. If they don't reconcile, find the gap before you go further; an unreconciled model is an untrustworthy model.
  3. 32:00 to 3:00 · Find the 15% and stress it. Identify where the cost concentrates, almost always a few high-volume or large-model paths. Build the sensitivity table. Pick the levers that survive the stress: model routing, prompt and context trimming, caching, tier downgrades for simple calls.
  4. 43:00 to 3:45 · Write the one-pager. Recommendation up top in two sentences, then the savings math, the product-risk line and the metric you'd watch. Write it for a CFO and an ML lead on the same page. Neither should need a glossary.
  5. 53:45 to 4:00 · Reconcile and ship. Re-check that the model totals match, the assumptions tab is complete and the recommendation names a number. Submit the honest version, not the impressive-looking one.
The attribution core - a defensible cost-per-active-user build you can walk line by line
# Bottom-up COGS attribution. Every term traces to an assumption on the
# assumptions tab - that traceability is what makes the model trusted.

feature_cost = sum(
    calls[f, tier]                 # volume, from usage logs
    * tokens_per_call[f, tier]     # ASSUMPTION: measured median, not mean
    * price_per_token[tier]        # ASSUMPTION: blended in+out $/token (list $/1M ÷ 1e6)
    for f in features for tier in tiers
)

cost_per_active_user = feature_cost / monthly_active_users

# The 15% lever, modeled explicitly so the saving is auditable:
# route "simple" tab/chat calls from the large tier to a small model.
routed_share        = 0.40        # ASSUMPTION: share of calls that are "simple"
small_tier_discount = 0.85        # small model ~15% the $/token of large
savings = (calls_large * routed_share
           * tokens_per_call_large
           * price_per_token_large   # $/token, same unit as price_per_token above
           * small_tier_discount)