Data/ML/AIBackendDevOps/CloudEmerging
LLM Gateway / Model Router
Also written as AI Gateway, LLM Gateway, LiteLLM, OpenRouter, Model Routing
A layer between an application and the AI model providers that handles switching between models, fallbacks when a provider is down, rate limits, cost tracking and logging. A router can also send easy requests to a cheap model and hard ones to a stronger one.
Think of it like
A company travel desk that books whichever airline suits each trip and rebooks you when a flight is cancelled.
Junior or senior?
Shows platform thinking about AI beyond one feature.
Senior sounds like
Can talk about cost savings, handling a provider outage, or how routing decisions were checked for quality.
Ask them
“Why did you put a gateway in front of the models, and what did it save you or protect you from?”