1 min lesson
The three flagship projects from the JD
Use the lesson to respond to this: "The JD says onboarding a new model should be 'a config change, not a code change.' What design property does that imply for the gateway?" Keep the answer plain.
Step 1 of 2
The JD names three projects and they are the spine of the role. Each is a different facet of one goal: keep the path fast, reliable and cheap while the real world throws chaos at it.
"A single abstraction over every provider's API."
Onboarding a new model becomes a config change, not a code change.
Normalizes request/response, streaming, timeouts, idempotency, observability.
"No single provider outage causes user-visible degradation."
Health-based routing, circuit breakers, retries, hedging, fallback ladders.
Correctness under retry: idempotency so you don't double-charge or double-stream.
"Traffic spikes don't cascade into providers."
Queueing, load shedding, token buckets, concurrency limits, QoS classes.
Protect the fleet and the providers instead of melting both down.
The gateway is the foundation the other two stand on. If every provider hides behind one normalized interface, then failover is just "pick a different backend behind the same contract" and backpressure is just "refuse or delay requests at one well-defined choke point." Build the abstraction badly and both of those become a mess of special cases.
Learn more
Full explanation
One Request Through the Path
Interactive diagram. Step through it with the Next and Previous controls below, or Tab to a region to read its detail.
Admission control is the gate - it decides admit, queue or shed before a request ever reaches a provider.