1 min lesson
Self-hosted vs provider-hosted
Use "Host your own models, or call a provider's API" to describe the practical rule.
Step 1 of 3
Self-hosted vs provider-hostedthe tradeoff that frames the whole role
This is the strategic fork the inference platform lives on: host your own models, or call a provider's API. A mature platform does both and routes between them.
Learn more
Advanced table
Reference table
- Dimension
- Cost at scale
- Self-hosted
- Lowest if utilization is high
- Provider-hosted
- Per-token margin baked in
- Dimension
- Control / tuning
- Self-hosted
- Full (batching, KV paging, quantization)
- Provider-hosted
- Limited to API knobs
- Dimension
- Operational burden
- Self-hosted
- You run serving + on-call + capacity
- Provider-hosted
- Provider runs it
- Dimension
- Flexibility / new models
- Self-hosted
- You must deploy each one
- Provider-hosted
- New models appear behind one API
- Dimension
- Failure exposure
- Self-hosted
- Your fleet, your blast radiusHow much breaks if a change goes wrong; the scope of potential damage. Press Enter for the full definition.
- Provider-hosted
- Their outage degrades you unless you fail over
| Dimension | Self-hosted | Provider-hosted |
|---|---|---|
| Cost at scale | Lowest if utilization is high | Per-token margin baked in |
| Control / tuning | Full (batching, KV paging, quantization) | Limited to API knobs |
| Operational burden | You run serving + on-call + capacity | Provider runs it |
| Flexibility / new models | You must deploy each one | New models appear behind one API |
| Failure exposure | Your fleet, your blast radiusHow much breaks if a change goes wrong; the scope of potential damage. Press Enter for the full definition. | Their outage degrades you unless you fail over |
The honest interview answer starts with "it depends on volume, latency budget and which models." Then reason it through.