1 post with tag gateway
LLM Inference on OVH MKS: LiteLLM API Gateway
LiteLLM gateway on top of vLLM: per-user API keys, budget limits, and automatic fallback to commercial APIs when the local GPU node is cold. Part 6 of 6.
· 8 minutes reading time