Alek's Blog - vllm

LLM Inference on OVH MKS: Connect IDEs and Web UIs

2026-06-02T00:00:00+00:00

The first four parts of this series deployed a vLLM inference endpoint at https://llm.YOUR_DOMAIN/v1</code>, protected by a Bearer token, running on an OVH RTX5000-28 GPU node. This part shows how to connect coding assistants, web UIs, and other OpenAI-compatible clients to that endpoint.</p>



LLM Inference on OVH MKS: Terraform, Ansible, and Deployment
2026-06-02T00:00:00+00:00
Part 1</a> covered the architecture and use cases. This post walks through the complete Terraform and Ansible setup and a first deployment.</p>


LLM Inference on OVH MKS: LiteLLM API Gateway
2026-06-02T00:00:00+00:00
Parts 1–5 deployed a functional self-hosted LLM endpoint. This part adds LiteLLM</a> — an open-source proxy that exposes a unified OpenAI-compatible API across multiple LLM backends — in front of vLLM. The gateway layer brings per-user API keys, budget enforcement, and automatic fallback to commercial APIs when the local model is unavailable.</p>


LLM Inference on OVH MKS: The Complete Guide
2026-06-02T00:00:00+00:00
This is the index and reading guide for a six-part series on self-hosting LLM inference on a
GPU-enabled Kubernetes node pool, using OVH Managed Kubernetes Service (MKS) as the concrete
platform throughout. The series runs end to end: the decision of whether to self-host at all,
provisioning the GPU infrastructure, serving a model behind an OpenAI-compatible API, wiring up
observability and autoscaling, connecting real client tools, and finally putting a multi-user
gateway in front of the whole thing.</p>


LLM Inference on OVH MKS: Introduction
2026-06-02T00:00:00+00:00
This post covers the decision context for self-hosting LLM inference on OVH MKS: when it makes sense, what the stack looks like, and what the costs are. Part 2</a> walks through the Terraform and Ansible setup.</p>


LLM Inference on OVH MKS: Prometheus, Grafana, and KEDA
2026-06-02T00:00:00+00:00
Part 1</a> covered the architecture and use cases.
Part 2</a> walked through Terraform and Ansible.
Part 3</a> covered models and the OpenAI API.
This part adds observability (Prometheus + Grafana) and scale-to-zero autoscaling via KEDA.</p>


LLM Inference on OVH MKS: Models, AWQ, and OpenAI API
2026-06-02T00:00:00+00:00
Part 1</a> covered the architecture and use cases.
Part 2</a> walked through the Terraform and Ansible setup.
This post covers which models fit on the RTX5000-28’s 16 GB GPU VRAM, why AWQ quantization is required for 7B+ models,
and how to use the OpenAI-compatible endpoint from your own code.</p>