VPP/HostStack/LDP/iperf
Contents
iperf3 with LD_PRELOAD
Example of how to run iperf3 via ldp and vcl on top of vpp's host stack. This was last tested with iperf 3.1.3 on Ubuntu 18.04.1.
To run the test two hosts with networking connectivity are needed for the client and server instances of iperf.
VPP configuration
In addition to the typical startup config parameters, the host stack requires the ones lower for enabling the use of memfds segments for vpp message queues and to configure the binary api socket transport. For a list of startup parameters see here. This is needed for both vpp instances.
session { evt_qs_memfd_seg }
socksvr { socket-name /tmp/vpp-api.sock}
Run iperf
First start the two vpp instances and ensure that the network between them is functional. The simplest option would be to use the vpp builtin ping utility. Then, on both hosts, define the following variables, with the appropriate paths.
VCL_CFG=/path/to/vcl.conf LDP_PATH=/path/to/vpp/build-root/build-vpp-native/vpp/lib/libvcl_ldpreload.so
To start the server:
sudo LD_PRELOAD=$LDP_PATH VCL_CONFIG=$VCL_CFG taskset --cpu-list <core-list> iperf3 -4 -s
To start the client:
sudo LD_PRELOAD=$LDP_PATH VCL_CONFIG=$VCL_CFG taskset --cpu-list <core-list> iperf3 -c
Make sure that the core-list is such selected that it doesn't overlap vpp's workers but it maintains the same numa.
Recommended half-duplex throughput testing configuration
- 16k mbufs
- 1 thread (main thread) since connection oriented transport protocols like TCP have sessions pinned to a core
- 256 rx/tx-descriptors
- 1 rx-queue, 1 tx-queue
To ensure that main thread runs on the same numa as the nic, first find the numa for the nic with sh hardware and then in startup.conf make sure main-core under cpu is set to a core on the same numa as the nic. To find out what numa a core pertains to use lscpu.
