1 min lesson
Don't treat failover and backpressure as the same thing
Connect "Name the gateway first as the abstraction boundary, then layer failover and backpressure on top" to the decision you would make.
Step 1 of 2
These three are the most likely seeds for both the take-home and the onsite design discussion. Internalize how the JD phrases each one, because echoing "onboarding a new model should be a config change" back in the design round tells the interviewer you read the role and think in their terms.
If a design prompt is open-ended ("design our inference layer"), structure your answer around these three pillars explicitly. Name the gateway first as the abstraction boundary, then layer failover and backpressure on top. That structure shows you've decomposed the problem the way the team has, instead of free-associating about LLMs.
Don't treat failover and backpressure as the same thing. Failover handles a provider that is down or unhealthy. Backpressure handles too much load even when everything is healthy. A common mistake is retrying harder into an overloaded provider - that's a failover reflex making a backpressure problem worse and it cascades.