1 min lesson

Systems-literacy & domain rounds

Connect "Two technical rounds with no algorithms" to the decision you would make.

Two technical rounds with no algorithms. The systems round checks that you can reason about GPUs, latency, batching and utilization. The domain round centers on FinOps-for-AI: you'll define the right metrics and design an attribution model live, with a senior leader pushing back.

Vocabulary is table stakes. The actual signal is judgment - what to measure, what to cut and which tradeoff you'd take with imperfect data.

Learn more

Advanced table

You won't be asked to implement any of this

Concept: Utilization
Why it drives cost: GPUs often run at 15-30%; idle silicon is pure cost
The judgment they're listening for: You target a realistic sustained number, not 100% and know the reliability tradeoff of pushing it up.

Concept: Latency SLA
Why it drives cost: Tab prediction needs tens-of-ms latency, which limits batching
The judgment they're listening for: You see that tight latency caps throughput and raises cost per request and you'd route by need.

Concept: Batching / throughput
Why it drives cost: Bigger batches raise GPU efficiency but add latency
The judgment they're listening for: You'd batch aggressively for background agent work, lightly or not at all for tab completion.

Concept: Model routing
Why it drives cost: Small models for simple queries, large for complex
The judgment they're listening for: You'd send easy completions to a cheap model and reserve the expensive one for hard requests.

Concept: Reserved vs. on-demand
Why it drives cost: Reserved is cheap at high sustained use; burst is pricey
The judgment they're listening for: You size a reserved baseline near the 60-80% sustained-usage breakeven and burst on-demand above it.

Concept	Why it drives cost	The judgment they're listening for
Utilization	GPUs often run at 15-30%; idle silicon is pure cost	You target a realistic sustained number, not 100% and know the reliability tradeoff of pushing it up.
Latency SLA	Tab prediction needs tens-of-ms latency, which limits batching	You see that tight latency caps throughput and raises cost per request and you'd route by need.
Batching / throughput	Bigger batches raise GPU efficiency but add latency	You'd batch aggressively for background agent work, lightly or not at all for tab completion.
Model routing	Small models for simple queries, large for complex	You'd send easy completions to a cheap model and reserve the expensive one for hard requests.
Reserved vs. on-demand	Reserved is cheap at high sustained use; burst is pricey	You size a reserved baseline near the 60-80% sustained-usage breakeven and burst on-demand above it.

You won't be asked to implement any of this. You will be asked which lever you'd pull first and why.

Systems-literacy & domain rounds

Step 1 of 3

You won't be asked to implement any of this