https://blog.none.at Production notes on Kubernetes, OpenShift, and OVHcloud: observability, log archiving, service mesh, LLM inference, and digital sovereignty. Zola en Sat, 27 Jun 2026 00:00:00 +0000 Tue, 02 Jun 2026 00:00:00 +0000 aleks https://blog.none.at/blog/2026/2026-06-02-llm-inference-on-ovh-serving/ https://blog.none.at/blog/2026/2026-06-02-llm-inference-on-ovh-serving/ <p><a href="https://blog.none.at/blog/2026/2026-06-02-llm-inference-on-ovh-introduction/">Part 1</a> covered the architecture and use cases. <a href="https://blog.none.at/blog/2026/2026-06-02-llm-inference-on-ovh-deployment/">Part 2</a> walked through the Terraform and Ansible setup. This post covers which models fit on the RTX5000-28’s 16 GB GPU VRAM, why AWQ quantization is required for 7B+ models, and how to use the OpenAI-compatible endpoint from your own code.</p>