Setting up a Bridge Network for VMs
Introduction
This is a guide for building a network that will allow virtual machines running
on Linux to communicate with each other. This method should differ between
Linux distributions only in the firewall configuration tool. I use Fedora, so
firewall-cmd will be used.
Building the Bridge
Enable IPv4 Packet Forwarding
The host machine will need to act as a router between the virtual machines,
which can be enabled by setting the kernel parameter net.ipv4.ip_forward.
When forwarding is disabled, the kernel will drop any packets that arrive on one
interface but are destined for another interface. Without this step, packets
sent between virtual machines will never be delivered, as they must all pass
through the host’s network stack to reach their destination.
Check the parameter status
sysctl net.ipv4.ip_forward
# or
cat /proc/sys/net/ipv4/ip_forward
Set the parameter
sudo sysctl -w net.ipv4.ip_forward=1
Note that this setting will not be persistent, which is intentional. See Basic Kernel Management for information on how to make it persistent.
Add Bridge Device
A bridge device is a virtual network switch that forwards packets between the
interfaces connected to it. This will allow virtual machines to communicate as
if they were on the same physical network. You can use any name you like, but
they are commonly named using a br0, br1, ..., brN format. Since my bridge is
specifically for virtual machines, I chose to use vmbr0.
sudo ip link add name vmbr0 type bridge
Add Gateway Address
This will be the host’s IP address on the bridge device and the gateway address inside the virtual machines. Feel free to use whatever subnet you want as long as it differs from your local network.
sudo ip addr add 10.0.10.1/24 dev vmbr0
Set Bridge Up
sudo ip link set vmbr0 up
Add TAP Devices
A TAP (Terminal Access Point) device is a virtual network interface that acts like a physical network interface. Add a tap interface for each of your virtual machines and connect them to the bridge.
sudo ip tuntap add dev tap1 mode tap
sudo ip tuntap add dev tap2 mode tap
sudo ip link set tap1 master vmbr0
sudo ip link set tap2 master vmbr0
Configure a DHCP Server
The setup at this point will probably work, but you’d need to manually
configure the network inside each virtual machine. Instead of doing that, I
would recommend running a DHCP server on the host. There are several good
options out there, but I chose to use dnsmasq. It’s lightweight, simple to
use, and it just works.
Most of the options here are pretty standard and explained very well in the man pages. I’ve highlighted the important parts with variables, all of which are self-explanatory.
bridge=vmbr0
subnet=10.0.10.1
vm1MAC='52:54:00:12:34:52'
vm2MAC='52:54:00:12:34:53'
leaseFile=/tmp/dnsmasq.leases
logFile=/tmp/dnsmasq.log
sudo dnsmasq \
--interface=$bridge \
--bind-dynamic \
--dhcp-range=${subnet%.*}.2,${subnet%.*}.254,255.255.255.0 \
--dhcp-option=3,$subnet \
--dhcp-option=6,$subnet \
--server=8.8.8.8 \
--server=8.8.4.4 \
--dhcp-authoritative \
--dhcp-host=${vm1MAC},${subnet%.*}.2 \
--dhcp-host=${vm2MAC},${subnet%.*}.3 \
--dhcp-leasefile=${leaseFile} \
--no-daemon \
&> ${logFile} &
Configure Firewall
sudo firewall-cmd --zone=public --add-interface=vmbr0
sudo firewall-cmd --zone=public --add-masquerade
sudo firewall-cmd --zone=public --add-forward
sudo firewall-cmd --zone=public --add-service=dhcp
sudo firewall-cmd --zone=public --add-service=dns
Notice that I did not add the --permanent flag to these commands, which is
intentional. If you would like this firewall configuration to be persistent, you
will need to do so.
Connect Virtual Machines
That’s it. All these steps are a bit much to do every time you spin up a set of virtual machines, so I’d recommend writing script(s) to automate the process for your needs.
Here’s an example qemu command that would launch one virtual machine connected
to the bridge network. Obviously, the important parts here are the -device and
-netdev options.
sessionName=example
mac='52:54:00:12:34:52'
memory='2G'
vcpu='2'
cpuMode='host'
image='my-image.qcow2'
format='qcow2'
tap='tap1'
qemu-system-x86_64 \
-name $sessionName \
-boot order=d \
-enable-kvm \
-m $memory \
-smp $vcpu \
-cpu $cpuMode \
-vga virtio \
-drive file=$image,format=$format \
-device virtio-net-pci,netdev=net0,mac=$mac \
-netdev tap,id=net0,ifname=$tap,script=no,downscript=no - Clint Jordan