View source for VPP/Use VPP to connect VMs Using Vhost-User Interface


= VPP setup =

We will use VPP to create a L2 bridge between two VMs, connected using vhost-user interface.  With this in mind, first create the vhost-user interfaces, and set them up:
<pre>
$ sudo vppctl create vhost socket /var/run/vpp/sock1.sock server
VirtualEthernet0/0/0
$ sudo vppctl create vhost socket /var/run/vpp/sock2.sock server
VirtualEthernet0/0/1
$ sudo vppctl set interface state VirtualEthernet0/0/0 up
$ sudo vppctl set interface state VirtualEthernet0/0/1 up
</pre>

Next, connect the interfaces to the same bridge domain.  Below we are creating and attaching the vhost-user interfaces to bridge domain `1`:
<pre>
$ vppctl set interface l2 bridge VirtualEthernet0/0/0 1
$ vppctl set interface l2 bridge VirtualEthernet0/0/1 1
</pre>

You can see the setup of the bridge domain as follows:
<pre>
$ sudo vppctl show bridge-domain 1 detail
  ID   Index   Learning   U-Forwrd   UU-Flood   Flooding   ARP-Term     BVI-Intf   
  1      1        on         on         on         on         off          N/A     

           Interface           Index  SHG  BVI  TxFlood        VLAN-Tag-Rewrite       
     VirtualEthernet0/0/0        1     0    -      *                 none             
     VirtualEthernet0/0/1        2     0    -      *                 none 
</pre>

= Boot virtual machines, making use of vhost-user network interface =

Now that VPP infrastructure is setup, we are ready to boot our VM.  For this example, pull down a Clear Linux KVM image from https://download.clearlinux.org/image/ , as well as the BIOS image, OVMF.fd and the sample startup script, start_qemu.sh.

Use start_qemu.sh to boot the KVM image, and install necessary packages to test connectivity, and then shutdown:
<pre>
$ bash start_qemu.sh clear-14200-kvm.img
...
~ # swupd update
~ # swupd bundle-add web-server-basic network-basic
~ # shutdown now
</pre>
Copy the clear.\*kvm.img to a second unique .img (so that you can boot two of them at a time)

Since we are making use of vhost-user, its required that the VM be launched with prealloced numa-mode memory, backed by hugepages.  Since hugepages are also being used by VPP (only 1024 by default), it is necessary to allocate more (how much will vary depending on the VM, and should be selected pending your system's available RAM):
<pre>
sudo sysctl -w vm.nr_hugepages=4096
</pre>

Launch the first VM:

<pre>
qemu-system-x86_64 \
    -enable-kvm -m 1024 \
    -bios OVMF.fd \
    -smp 4 -cpu host \
    -vga none -nographic \
    -drive file="1-clear-14200-kvm.img",if=virtio,aio=threads \
    -chardev socket,id=char1,path=/var/run/vpp/sock1.sock \
    -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \
    -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1 \
    -object memory-backend-file,id=mem,size=1024M,mem-path=/dev/hugepages,share=on \
    -numa node,memdev=mem -mem-prealloc \
    -debugcon file:debug.log -global isa-debugcon.iobase=0x402
</pre>

Launch the second VM similar as above, only update the image, the mac address and the vhost-user socket:
<pre>
qemu-system-x86_64 \
    -enable-kvm -m 1024 \
    -bios OVMF.fd \
    -smp 4 -cpu host \
    -vga none -nographic \
    -drive file="2-clear-14200-kvm.img",if=virtio,aio=threads \
    -chardev socket,id=char1,path=/var/run/vpp/sock2.sock \
    -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \
    -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet1 \
    -object memory-backend-file,id=mem,size=1024M,mem-path=/dev/hugepages,share=on \
    -numa node,memdev=mem -mem-prealloc \
    -debugcon file:debug.log -global isa-debugcon.iobase=0x40
</pre>

In VM #1, setup your IP address as follows:

<pre>
$ ip addr add dev enp0s2 192.168.0.1/24
</pre>

In VM #2, setup your IP address as follows:

<pre>
$ ip addr add dev enp0s2 192.168.0.2/24
</pre>

You can now test basic connectivity from VM#1 to VM#2:
<pre>
 # ping 192.168.0.1 -c1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=0.242 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.242/0.242/0.242/0.000 ms
</pre>