Setting up a Bridge Network for VMs

Introduction

This is a guide for building a network that will allow virtual machines running on Linux to communicate with each other. This method should differ between Linux distributions only in the firewall configuration tool. I use Fedora, so firewall-cmd will be used.

Building the Bridge

Enable IPv4 Packet Forwarding

The host machine will need to act as a router between the virtual machines, which can be enabled by setting the kernel parameter net.ipv4.ip_forward. When forwarding is disabled, the kernel will drop any packets that arrive on one interface but are destined for another interface. Without this step, packets sent between virtual machines will never be delivered, as they must all pass through the host’s network stack to reach their destination.

Check the parameter status

sysctl net.ipv4.ip_forward
# or
cat /proc/sys/net/ipv4/ip_forward

Set the parameter

sudo sysctl -w net.ipv4.ip_forward=1

Note that this setting will not be persistent, which is intentional. See Basic Kernel Management for information on how to make it persistent.

Add Bridge Device

A bridge device is a virtual network switch that forwards packets between the interfaces connected to it. This will allow virtual machines to communicate as if they were on the same physical network. You can use any name you like, but they are commonly named using a br0, br1, ..., brN format. Since my bridge is specifically for virtual machines, I chose to use vmbr0.

sudo ip link add name vmbr0 type bridge

Add Gateway Address

This will be the host’s IP address on the bridge device and the gateway address inside the virtual machines. Feel free to use whatever subnet you want as long as it differs from your local network.

sudo ip addr add 10.0.10.1/24 dev vmbr0

Set Bridge Up

sudo ip link set vmbr0 up

Add TAP Devices

A TAP (Terminal Access Point) device is a virtual network interface that acts like a physical network interface. Add a tap interface for each of your virtual machines and connect them to the bridge.

sudo ip tuntap add dev tap1 mode tap
sudo ip tuntap add dev tap2 mode tap
sudo ip link set tap1 master vmbr0
sudo ip link set tap2 master vmbr0

Configure a DHCP Server

The setup at this point will probably work, but you’d need to manually configure the network inside each virtual machine. Instead of doing that, I would recommend running a DHCP server on the host. There are several good options out there, but I chose to use dnsmasq. It’s lightweight, simple to use, and it just works.

Most of the options here are pretty standard and explained very well in the man pages. I’ve highlighted the important parts with variables, all of which are self-explanatory.

bridge=vmbr0
subnet=10.0.10.1
vm1MAC='52:54:00:12:34:52'
vm2MAC='52:54:00:12:34:53'
leaseFile=/tmp/dnsmasq.leases
logFile=/tmp/dnsmasq.log
sudo dnsmasq \
  --interface=$bridge \
  --bind-dynamic \
  --dhcp-range=${subnet%.*}.2,${subnet%.*}.254,255.255.255.0 \
  --dhcp-option=3,$subnet \
  --dhcp-option=6,$subnet \
  --server=8.8.8.8 \
  --server=8.8.4.4 \
  --dhcp-authoritative \
  --dhcp-host=${vm1MAC},${subnet%.*}.2 \
  --dhcp-host=${vm2MAC},${subnet%.*}.3 \
  --dhcp-leasefile=${leaseFile} \
  --no-daemon \
  &> ${logFile} &

Configure Firewall

sudo firewall-cmd --zone=public --add-interface=vmbr0
sudo firewall-cmd --zone=public --add-masquerade
sudo firewall-cmd --zone=public --add-forward
sudo firewall-cmd --zone=public --add-service=dhcp
sudo firewall-cmd --zone=public --add-service=dns

Notice that I did not add the --permanent flag to these commands, which is intentional. If you would like this firewall configuration to be persistent, you will need to do so.

Connect Virtual Machines

That’s it. All these steps are a bit much to do every time you spin up a set of virtual machines, so I’d recommend writing script(s) to automate the process for your needs.

Here’s an example qemu command that would launch one virtual machine connected to the bridge network. Obviously, the important parts here are the -device and -netdev options.

sessionName=example
mac='52:54:00:12:34:52'
memory='2G'
vcpu='2'
cpuMode='host'
image='my-image.qcow2'
format='qcow2'
tap='tap1'
qemu-system-x86_64 \
  -name $sessionName \
  -boot order=d  \
  -enable-kvm \
  -m $memory \
  -smp $vcpu \
  -cpu $cpuMode \
  -vga virtio \
  -drive file=$image,format=$format \
  -device virtio-net-pci,netdev=net0,mac=$mac \
  -netdev tap,id=net0,ifname=$tap,script=no,downscript=no

- Clint Jordan