Production notes on Kubernetes, OpenShift, and OVHcloud: observability, log archiving, service mesh, LLM inference, and digital sovereignty. Zola 2026-06-27T00:00:00+00:00 https://blog.none.at/tags/awq/atom.xml 2026-06-02T00:00:00+00:00 2026-06-27T00:00:00+00:00 aleks https://blog.none.at/blog/2026/2026-06-02-llm-inference-on-ovh-serving/

<p><a href="https://blog.none.at/blog/2026/2026-06-02-llm-inference-on-ovh-introduction/">Part 1</a> covered the architecture and use cases. <a href="https://blog.none.at/blog/2026/2026-06-02-llm-inference-on-ovh-deployment/">Part 2</a> walked through the Terraform and Ansible setup. This post covers which models fit on the RTX5000-28’s 16 GB GPU VRAM, why AWQ quantization is required for 7B+ models, and how to use the OpenAI-compatible endpoint from your own code.</p>