1 post with tag #gateway

Architecture diagram: LiteLLM API gateway in front of LLM inference services

vllm llm kubernetes ovh litellm gateway inference

LLM Inference on OVH MKS: LiteLLM API Gateway

LiteLLM gateway on top of vLLM: per-user API keys, budget limits, and automatic fallback to commercial APIs when the local GPU node is cold. Part 6 of 6.

2026-06-028 min

LLM Inference on OVH MKS: LiteLLM API Gateway