libvirt

How to setup and run KVM/QEMU virtual machines

Install packages

pacman -Syu libvirt qemu edk2-ovmf virt-manager

If using the default NAT/DHCP networking instead of a bridge

pacman -Syu iptables-nft dnsmasq

Access permissions qemu:///system

usermod -aG libvirt-qemu USER

Enable service

systemctl enable libvirtd.service -f

Create a network bridge

Warning

Check /etc/systemd/network/ for conflicting files

nmcli connection add type bridge ifname br0 stp no

nmcli connection add type bridge-slave ifname enp5s0 master br0

nmcli connection down enp5s0

nmcli connection up bridge-br0

nmcli connection up bridge-slave-enp5s0

nmcli connection edit br0

set bridge.mac-address

save persistent

quit

/etc/systemd/network/mybridge.netdev

[NetDev]
Name=br0
Kind=bridge
MACAddress=a8:5e:45:a7:09:99

/etc/systemd/network/bind.network

[Match]
Name=en*

[Network]
Bridge=br0

/etc/systemd/network/mybridge.network

[Match]
Name=br0

[Network]
DHCP=ipv4

systemctl enable systemd-networkd -f

ps aux | grep -i dnsmasq

virsh list --all

ip a s

virsh net-dhcp-leases default

route

virsh net-destroy default

virsh net-list --all

ip link set enp4s0 down

ip addr del 192.168.0.100/24 dev enp4s0

systemctl disable NetworkManager

systemctl disable dhcpcd.service

systemctl stop systemd-networkd

ip link add name br0 type bridge

ip link set enp4s0 master br0

ip addr add 192.168.0.100/24 dev br0 brd 192.168.255.255

ip link set up enp4s0

ip link set up br0

From another device

arping 192.168.0.100 -I enp4s0

route add default gw 192.168.0.1

Add network bridge to virt-manager

bridged-network.xml

<network>
    <name>bridged-network</name>
    <forward mode="bridge" />
    <bridge name="br0" />
</network>

virsh net-define bridged-network.xml

virsh net-start bridged-network

virsh net-autostart bridged-network

virsh net-list

Create a dynamic virtual disk from scratch

qemu-img create -f qcow2 -o preallocation=off diskname.qcow2 1T

Disable virtual disk preallocation

qemu-img convert -f qcow2 -O qcow2 -o preallocation=off /home/user/directory/diskname-old.qcow2 /home/user/directory/diskname.qcow2

Resize virtual disk

qemu-img info diskname.qcow2

qemu-img resize diskname.qcow2 +180G

Shrink virtual disk

Noop conversion (qcow2-to-qcow2) removes sparse space.
Shrink your disk without compression (better performance, larger disk size).

qemu-img convert -O qcow2 diskname.qcow2_backup diskname.qcow2

Shrink your disk with compression (smaller disk size, takes longer to shrink, performance impact on slower systems)

qemu-img convert -O qcow2 -c diskname.qcow2_backup diskname.qcow2

Enable huge pages

/etc/fstab

hugetlbfs /dev/hugepages hugetlbfs mode=01770,gid=kvm 0 0

Dynamic huge pages

/etc/sysctl.d/40-hugepage.conf

vm.nr_hugepages = 0
vm.nr_overcommit_hugepages = 6144

Static huge pages

/etc/sysctl.d/40-hugepage.conf

vm.nr_hugepages = 6144

Determine the number of hugepages needed. Huge pages will be automatically allocated, and freed after VM stops.

Check the size of the hugepages

grep Hugepagesize /proc/meminfo

Tip

It is hardly recommended to drop caches, compact memory and wait couple of seconds before starting VM, as there could be not enough free contiguous memory for required huge pages blocks. Especially after some uptime of the host system.

echo 3 > /proc/sys/vm/drop_caches

echo 1 > /proc/sys/vm/compact_memory

Virt-manager

Make sure that Chipset: Q35 is selected. Under Firmware, select the 64 bit UEFI firmware UEFI x86_64: OVMF_CODE.fd

<memoryBacking>
  <hugepages/>
</memoryBacking>

<features>
  <acpi/>
  <apic/>
  <hyperv>
    <relaxed state="on"/>
    <vapic state="on"/>
    <spinlocks state="on" retries="8191"/>
    <vpindex state='on'/>
    <runtime state="on"/>        
    <synic state='on'/>
    <stimer state="on">
      <direct state="on"/>
    </stimer>
    <reset state="on"/>
    <vendor_id state="on" value="0123456789ab"/>
    <frequencies state="on"/>
    <reenlightenment state="on"/>
    <tlbflush state="on"/>
    <ipi state="on"/>
    <evmcs state="off"/>
  </hyperv>
  <kvm>
    <hidden state='on'/>
  </kvm>
  <vmport state="off"/>
</features>
<cpu mode="host-passthrough" check="none" migratable="on">
  <topology sockets="1" dies="1" cores="4" threads="2"/>
  <cache mode="passthrough"/>
  <feature policy="require" name="topoext"/>
</cpu>
<clock offset="utc">
  <timer name="rtc" tickpolicy="catchup"/>
  <timer name="pit" tickpolicy="delay"/>
  <timer name="hpet" present="no"/>
  <timer name="hypervclock" present="yes"/>
  <timer name="tsc" present="yes" mode="native"/>
</clock>

<memballoon model="none"/>

<cpu mode="host-passthrough" check="none" migratable="on">
  <numa>
    <cell memory='2097152' unit='KiB' memAccess='shared'/>
  </numa>
</cpu>

<filesystem type='mount' accessmode='passthrough'>
  <driver type='virtiofs'/>
  <source dir='/mnt/sdc1'/>
  <target dir='sdc1'/>
</filesystem>

It should now be possible to mount the folder in the shared machine

mount -t virtiofs sdc1 /mnt/sdc1

Add the following fstab entry to mount the folder automatically at boot

/etc/fstab

sdc1 /mnt/sdc1 virtiofs rw,noatime,_netdev 0 0