OpenBSD + VMD + autoinstall

OpenBSD comes with a very simple, very nice hypervisor called vmd(8).  This hypervisor is not  a full featured as others, but it does a reasonable job of running VMs and makes it really convenient to  spin-up a virtual environment for testing configurations.  There are other articles on setting up a VM environment on OpenBSD.  Its even covered in the FAQ which is a must read.  And while this article and the others all show you how to start a VM, with this one I'll add the addition of how to get an OpenBSD guest to auto install.

Prep work

Host Firewall

To start with we need to setup the host system.  First thing we need to do is update PF.  While this isn't strictly necessary, I have wasted may moments troubleshooting a VM issue, when the problem was on the Host.  To make networking on the VM easy, I SNAT all traffic from the VM environment out the physical interface.

match out on external tagged TVMS nat-to (external:0)
pass on VMS
pass in on VMS from (VMS:network) to !(VMS) tag TVMS rtable 0

Two things to note: 1) I utilize a new interface group VMS so that these rules will match any new interface that get created; 2) we will setup the VM's in a separate rdomain(4) so any traffic that is not remaining local will be tagged and moved to rdomain 0 so it can be SNAT'd.

Routing

Since we will be forwarding traffic from the VM environment to the outside world and visa versa we need to enable forwarding.

sysctl net.inet.ip.forwarding=1
sysctl net.inet6.ip6.forwarding=1

Interfaces

Next we need to create the interfaces we will use.   A new rdomain and bridge interface will be created.  NOTE: the lo5 interface gets created when the first interface with the new rdomain gets created.

ifconfig bridge0 rdomain 5 group VMS up
ifconfig vether0 rdomain 5 group VMS
ifconfig vether0 inet 100.64.0.1/24 up
ifconfig bridge0 add vether0
ifconfig lo5 inet 127.0.0.1/8

DHCP

Since we want to have our VMs automatically get an IP address when they start we need to setup dhcpd(8).  This dhcpd.conf file will handout addresses out of the 100.64.0.0/24 netwok and deliver some dhcp options that we need to tell our VM's to autoinstall.  We have also specified the MAC address of the VM.   This allows dhcp server to send a hostname to our VM when it's installing allowing us to set the name.  (This could also be done in the install.conf file but then we would need to have a separate install.conf file for every VM.  This way we can keep one file and just update the name in dhcpd.conf

#
option  domain-name "example.com";
default-lease-time 120;
max-lease-time 120;

subnet 100.64.0.0 netmask 255.255.255.0 {
        range 100.64.0.20 100.64.0.254;
        option routers 100.64.0.1;
        #option domain-name-servers x.x.x.x;
        filename "auto_install";
        next-server 100.64.0.1;

        host vm1 {
                hardware ethernet fe:e1:ba:00:00:01;
                option host-name "vm1";
        }
}

Now start the dhcp server in rdomain 5.

rcctl enable dhcpd
rcctl set dhcpd rtable 5
rcctl set dhcpd flags vether0
rcctl start dhcpd

HTTPD

We will also need to serve some files from the httpd(8) server.  Using the simple config below, setup the httpd server to start on rdomain 5.

# httpd.conf
server "default" {
	listen on 100.64.0.1 port 80
	root "/htdocs"
}

rcctl enable httpd
rcctl set httpd rtable 5
rcctl start httpd

Now copy the install.conf below to the web root /var/www/htdocs/.

# hostname is set by dhcpd server
System hostname =
Which network interface do you wish to configure = vio0
IPv4 address for vio0 = autoconf
IPv6 address for vio0 = none
Which network interface do you wish to configure = done
Start sshd(8) by default = yes
Change the default console to com0 = yes
Which speed should com0 use = 115200
Password for root account = letmein
Setup a user = no
Allow root ssh login = yes
Which disk is the root disk = sd0
Use (W)hole disk MBR, whole disk (G)PT, (O)penBSD area or (E)dit = whole
Use (A)uto layout, (E)dit auto layout, or create (C)ustom layout = a
Location of sets = http
HTTP proxy URL = none
HTTP Server = 100.64.0.1
Server directory = pub/OpenBSD/7.1/amd64
Set name(s) = done
Directory does not contain SHA256.sig. Continue without verification = yes
Location of sets = done
What timezone are you in = UTC
Exit to (S)hell, (H)alt or (R)eboot = r

Now we need to add the OpenBSD install files to the web directory.

mkdir -p /var/www/htdocs/pub/OpenBSD/7.1/amd64
for i in `echo "SHA256 bsd bsd.mp bsd.rd base71.tgz comp71.tgz game71.tgz man71.tgz xbase71.tgz xfonts71.tgz xserv71.tgz xshare71.tgz index.txt"`; do
ftp -o /var/www/htdocs/pub/OpenBSD/7.1/amd64/$i https://cdn.openbsd.org/pub/OpenBSD/7.1/amd64/$i
done

Optional:  If you want to further customize the install you can add a site71.tgz or hostname-site71.tgz file to the web directory.  If you do, you'll need to remake the index.txt file.  This is easily accomplished below.

ls -lT > index.txt

Finally we need the vm.conf configuration file for vmd(8).  Notice that there are 2 VM definitions.  The build-vm1 definition is only used when building the new VM. Also notice that the MAC address is the same one that was specificed up in the dhcpd.conf file.  The vm1 definition is the one that will run the VM after it's been created.

PATH=/home/vms
RD_PATH=/var/www/htdocs/pub/OpenBSD/7.1/amd64
switch "build" {
        interface bridge0
        rdomain 5
        group VMS
        enable
}
vm "build-vm1" {
        disable
        memory 512M
        boot $RD_PATH/bsd.rd
        interfaces 1
        interface { switch "build" lladdr fe:e1:ba:00:00:01 }
}
vm "vm1" {
        disable
        memory 512M
        disk $PATH "/vm1.qcow2" format qcow2
        interfaces 1
        interface {
                group VMS
                switch "build"
        }
}

Now enable vmd(8) and start it.

rcctl enable vmd
rcctl start vmd

To make sure the configuration loaded correctly use vmctl(8) to show the VM status.

vmctl status

Build VM

Now we can finally build the VMs.

Create disk

First step is to create the disk.  An assumption is being made the the VM disk images will be stored in the /home/vms directory.

vmctl create -s 5G /home/vms/vm1.qcow2

Build the VM

Now start the build process.

vmctl start -c -B net -d /home/vms/vm1.qcow2 -t build-vm1 vm1

This will start the VM build process.  The -c option attached the current terminal to the the console of the building VM.  If all goes well the only key that will need to be pressed is the one at the end of the build process to exit the VM.

Run the VM

After the VM is built, to start it just run the vmctl(8) command again.

vmctl start vm1

This will start the VM but not attach the console to the existing terminal.  To attach to the console you can use the vmctl command below.  To exit the console use ~. escape sequence.

vmctl console vm1

Or you can ssh to the VM.  Check the /var/log/daemon log file to see what IP was assigned to the VM and then ssh to the VM.  Note that the VM is running in a separate rdomain so to ssh from that rdomain use the route(8) command.

route -T 5 exec ssh root@100.64.0.XXX

Wrap Up

This is just a simple tutorial on how to get a VM up and autoinstalled on OpenBSD hypervsor. Pleas let me know if there are anything I might have missed.