1 min lesson

Adding a model is a config change

Explain the practical point behind "A declarative entry naming the provider, model id, limits and pricing."

Adding a model is a config change

This is the line from the JD, so make it concrete. New models that share an existing provider's API should require zero code: a declarative entry naming the provider, model id, limits and pricing.

Learn more

Full explanation

Declarative model config - onboarding a model with no code change

models:
  claude-sonnet-tab:
    provider: anthropic        # reuses the existing adapter
    upstream_id: claude-sonnet-5
    surfaces: [tab]
    limits: { rpm: 4000, tpm: 2_000_000 }
    timeout_ms: 800            # tight: Tab is sub-100ms-budget critical
    price_per_mtok: { in: 3.00, out: 15.00 }
    fallback: claude-haiku-tab

Interview move

When they ask how you'd onboard a new provider, separate the two cases out loud: a new model on an existing provider is pure config; a new provider with a new API is one new adapter implementing a fixed interface plus its config. Naming that boundary is the whole answer - it proves you've found the right abstraction line instead of hand-waving "we'd abstract it."

Watch out

A leaky abstraction is worse than no abstraction. If a provider's tool-calling semantics or stop-sequence behavior differ in ways callers must handle, surface that in the typed contract rather than silently papering over it. Pretending two genuinely different behaviors are identical produces bugs that only show up in production, on the hot path, for every user.

Learn more

Optional practice

Practice: Adding a model is a config change

QWhere should retry logic live in the gateway - in each provider adapter or in the shared core - and why?

Adding a model is a config change

Step 1 of 3