Revision as of 15:21, 2 January 2018

High Speed container communications using VPP Communications Library (VCL)

Using the VPP Communications Library and an LD_PRELOAD library for standard POSIX sockets, VPP can demonstrate a 2.75x speed improvement over traditional Linux bridge.

HowTo

Prequisites

• docker (Version: 17.05)
• iperf3 (3.0.11)
• VPP: git clone https://gerrit.fd.io/r/vpp
• VPPSB: git clone https://gerrit.fd.io/r/vppsb

Environment Setup and Building

• Clone VPP and the VPPSB projects.

• Edit vppsb/vcl-ldpreload/env.sh
  • Set VCL_BASE to directory above VPP and VPPSB source. Leave all other env vars in this file alone.

 source ./env.sh

• Change to VPP source directory and build

 cd $VPP_DIR

• Modify uri.am to enable socket_test program

 perl -pi -e 's/noinst_PROGRAMS/bin_PROGRAMS/g' $VPP_DIR/src/uri.am

• Build VPP release

 make install-dep wipe-release bootstrap dpdk-install-dev build-release

• Build LD_PRELOAD library against VPP build above.

This does not install the LD_PRELOAD library in your system. Instead it will be referenced from the build directory set in VCL_LDPRELOAD_LIB

 cd $LDP_DIR/vcl-ldpreload/src
 autoreconf -i -f
 ./configure VPP_DIR=$VPP_DIR
 make

Running the demo

• Run test script without parameters to see help menu:

 cd $VPP_DIR/test/scripts
 ./socket_test.sh

Single Host Tests

These demos launch xterms (or xfce4-terminals), and have been tested on Ubuntu16.04.

Connecting two Docker Containers

• Docker iPerf using default Linux Bridge

 ./socket_test.sh -bi docker-kernel

• Docker iPerf using VCL-LDPRELOAD and VPP

 ./socket_test.sh -bi docker-preload

To quit, close the xterms and run following docker kill command

WARNING: This will kill all docker containers on your system. The command below is deliberately commented out to protect the innocent, and copypasta-happy.

 #docker kill $(docker ps -q)

Multi-host Tests

VCL-Client/VCL-Server on Vagrant VMs

Create two vagrant VMs, build VPP in each, and run the VCL-Client sock_test_client on one and connect to the VCL-Server sock_test_server on the other.

 cd $WS_ROOT  # VPP workspace root directory
 cd extras/vagrant
 cp Vagrantfile.vcl_test Vagrantfile
 vagrant up

Enjoy a cup of coffee, tea, water, donut, bagel or scooby snack while the VM's are created and build VPP.

• VCL Socket Test Client/Server on default Linux Bridge

Start the VCL-Server sock_test_server application on the vcl-server VM:

 vagrant ssh vcl-server
 cd $WS_ROOT
 $WS_ROOT/test/scripts/socket_test.sh -bm s native-kernel
 ifconfig enp0s8

In another terminal, run the VCL-Client sock_test_client application on the vcl-client VM:

 cd $WS_ROOT/extras/vagrant
 vagrant ssh vcl-client
 cd $WS_ROOT
 $WS_ROOT/test/scripts/socket_test.sh -bm c -S <server ip address> native-kernel

• VCL Socket Test Client/Server over VPP

Start the VCL-Server vcl_test_server application on the vcl-server VM:

 vagrant ssh vcl-server
 cd $WS_ROOT
 $WS_ROOT/test/scripts/socket_test.sh -tbm s native-vcl

Note: Vagrant configures the NICs to acquire an ip address using DHCP. socket_test.sh configures VPP with the address of enp0s8 or another interface specified by the '-n' option or a default address (e.g. 10.10.10.10). It prints the address that VPP uses to the terminal for use on the VCL-Client VM.

In another terminal, run the VCL-Client vcl_test_client application on the vcl-client VM:

 cd $WS_ROOT/extras/vagrant
 vagrant ssh vcl-client
 cd $WS_ROOT
 $WS_ROOT/test/scripts/socket_test.sh -tbm c -S <server ip address> native-vcl

Note: If VPP failed to exit cleanly, then enp0s8 may not have been successfully re-bound to the kernel driver. In order to fix this issue, run the following dpdk python script:

 sudo /usr/share/dpdk/usertools/dpdk-devbind.py -b e1000 0000:00:08.0

• Post testing cleanup.

After completing testing, issue the following command to clean up the vagrant VM's:

 cd $WS_ROOT/extras/vagrant
 vagrant destroy -f
 git checkout -- Vagrantfile

Note: The vagrant VM infrastructure is stored in $WS_ROOT/extras/vagrant/.vagrant, therefore beware of the use of 'git clean -fdX' in the VPP workspace when running vagrant VM's as it will unceremoniously delete it (as you asked it to!). However, don't panic if you forget and run git clean -- to recover, execute virtualbox to shut down and remove the remnants of the VMs. Then go enjoy some more coffee, tea, water, donuts or scooby snacks :-)