Opleiding: Deploying and fine-tuning open source LLMs

Deploy and fine-tune open source LLMs like Llama and Mistral on your own infrastructure with vLLM, Ollama, and LoRA.

Not every AI project can or should run on commercial APIs. Regulation, latency requirements, cost, or data sovereignty can be reasons to self-host open source models. In this course you learn to run models like Llama and Mistral with the inference frameworks vLLM and Ollama, and how to fine-tune them with LoRA on your own data. You finish with a deployment on your own infrastructure.
This course is intended for software, ML, and platform engineers who want to run and fine-tune open source LLMs on their own or cloud infrastructure. Working in a Microsoft/.NET environment and want to build AI agents in C#? Have a look at our related courses.
- Operate inference frameworks (vLLM, Ollama) to deploy open source LLMs on your own infrastructure [Apply]
- Customize a base LLM for domain-specific tasks using LoRA and QLoRA fine-tuning [Apply]
- Compare model formats and quantization strategies (FP16, GPTQ, AWQ, GGUF) for inference [Understand]
- Distinguish when to use fine-tuning, RAG, or prompt engineering for a given use case [Understand]
- Evaluate which open source model and deployment strategy best fits the quality, latency, cost, and compliance requirements of you…