Running Polish 🇵🇱 LLM Bielik on Azure VM 🚀 Cheap and Fast AI in the Cloud

This video (in Polish 🇵🇱) shows how to run the Polish open-source LLM Bielik in the cloud using an Azure Spot VM - a cost-effective option for those without a local GPU.

The guide covers creating a free Azure trial account, installing Terraform, Azure CLI, and WSL2 on Windows, and automatically provisioning a GPU virtual machine with Terraform. Once the VM is running, it walks through installing NVIDIA CUDA, Docker, and the NVIDIA Container Toolkit, then launching the Bielik model (Q4/Q8 quantization) via mistral.rs. Finally, it demonstrates testing the model over SSH and through a Python API, and securing the endpoint with a Caddy reverse proxy.

Resources:

• Azure VM presentation
• Terraform for Azure
• AWS VM presentation
• Terraform for AWS