← All notes
Abstract dark blue artwork of container and orchestration logos
Note

Building a Proxmox lab that actually holds up

Install, bridge networking, LXC containers and backups: the choices I'd make again, and the ones I regret.

I built and tore down my lab three times before landing on something stable. This note is the short version of what should have been done on the first install.

The hardware

Nothing exotic: a mini-PC with 64 GB of RAM, an NVMe SSD for the system and the virtual disks, and a second SATA SSD dedicated to backups. The second disk is the part that matters: backing up to the same disk that hosts your VMs protects you from exactly one failure mode, and it isn’t the interesting one.

Networking first

Network configuration is what breaks most often, and always at the worst possible moment: while you’re on SSH to the box. The bridge is declared in /etc/network/interfaces:

auto vmbr0
iface vmbr0 inet static
    address 192.168.1.10/24
    gateway 192.168.1.1
    bridge-ports enp1s0
    bridge-stp off
    bridge-fd 0

Two rules I now apply every time:

  1. Static IP on the host, never DHCP. One expired lease during a power cut and the hypervisor is unreachable.
  2. Test before you commit. ifreload -a applies the config without a reboot; scheduling shutdown -r +5 beforehand brings the old config back if the SSH session drops.

LXC over VMs, whenever possible

For anything that is just a Linux service (reverse proxy, internal DNS, monitoring), an LXC container uses a fraction of the resources of a full VM and boots in about a second. I keep VMs for workloads that need their own kernel: Windows, Kubernetes nodes, or anything touching kernel modules.

Workload Choice Why
Reverse proxy, DNS LXC Lightweight, near-instant boot
Database LXC Direct disk access, one less layer
Kubernetes cluster VM Kernel isolation is required
Kernel testing VM Obviously

Backups, or nothing

One scheduled backup job, Sundays at 03:00, snapshot mode, zstd compression, three copies retained. Snapshot mode doesn’t stop the guests, which matters when the lab hosts the house DNS.

More importantly: a restore you have actually tested. A backup that has never been restored is a hypothesis, not a guarantee. I restore one random guest every quarter into a throwaway container, purely to check.

What I regret

Waiting six months before setting up monitoring. For six months I discovered problems by noticing a service had stopped responding, instead of being told the disk had crossed 85%.