VPP/Using mTCP user mode TCP stack with VPP

This example shows how to configure and run sample client/server applications using user mode mTCP in 2 linux namespaces (or containers) which communicate through VPP via netmap virtual interfaces.

In this setup we use 2 different namespaces called vpp1 and vpp2 and two sample applications epserver and epwget available with mTCP.

Setup

NETMAP

Download the sources from the upstream repository using following command:

  git clone git@github.com:vpp-dev/netmap.git       OR      https://github.com/vpp-dev/netmap/archive/master.zip

Enter LINUX directory and configure netmap. To compile only NETMAP/VALE (using unmodified drivers):

./configure --no-drivers

make

make apps

sudo insmod netmap.ko

To verify that netmap module is loaded, Use the following command which should show (Module-name , Size and Used by):

lsmod | grep netmap

VPP

We assume that you are already running vpp. If it is not the case, please follow the following link to build, install and test VPP:

  https://wiki.fd.io/view/VPP/Build,_install,_and_test_images

mTCP

Download the sources using following command:

  git clone git@github.com:vpp-dev/mtcp.git     OR    https://github.com/vpp-dev/mtcp/archive/master.zip

Enter mtcp root directory and configure mtcp. To compile for netmap module:

./configure --enable-netmap

make

Namespaces

Create namespaces, using the following commands:

sudo ip netns add vpp1
sudo ip netns add vpp2

sudo ip netns show
vpp1
vpp2

Configure Interfaces

VPP

Run VPP/VPP-lite and create netmap interfaces using the VPP debug Command-line Interface (CLI):

        create netmap name vale00:vpp1 hw-addr 02:FE:3F:34:15:9B pipe master
        create netmap name vale00:vpp2 hw-addr 02:FE:75:C5:43:66 pipe master

        set int state netmap-vale00:vpp2 up
        set int state netmap-vale00:vpp1 up

        set int l2 xcon netmap-vale00:vpp1 netmap-vale00:vpp2
        set int l2 xcon netmap-vale00:vpp2 netmap-vale00:vpp1

To verify that interfaces have been created and up, use the following command:

vpp# show int
              Name               Idx       State          Counter          Count     
local0                            0        down      
netmap-vale00:vpp1                5         up       
netmap-vale00:vpp2                6         up       
pg/stream-0                       1        down      
pg/stream-1                       2        down      
pg/stream-2                       3        down      
pg/stream-3                       4        down    

Modify Config Files

mTCP

In <mTCP-ROOT>/apps/example/, you can change the epserver.conf file and epwget.conf.

example epserver.conf file:

# module
io = netmap

# Port
port vale00:vpp1}0

# Hw addr of port
hw_addr = 02:fe:3f:34:15:9b

# Ip addr of port
ip_addr = 10.0.42.3

# Netmask of port
netmask = 255.255.255.0

# Maximum concurrency per core
max_concurrency = 10000

# Maximum number of socket buffers per core
max_num_buffers = 10000

# Receive buffer size of sockets
rcvbuf = 16384

# Send buffer size of sockets
sndbuf = 16384

# TCP timeout seconds
tcp_timeout = 30

# TCP timewait seconds
tcp_timewait = 0

# Interface to print stats
stat_print = vale00:vpp1}0

example epwget.conf file

# module
io = netmap

# Port
port vale00:vpp2}0

# Hw addr of port
hw_addr = 02:fe:75:c5:43:66

# Ip addr of port
ip_addr = 10.0.42.2

# Netmask of port
netmask = 255.255.255.0

# Maximum concurrency per core
max_concurrency = 10000

# Maximum number of socket buffers per core
max_num_buffers = 10000

# Receive buffer size of sockets
rcvbuf = 16384

# Send buffer size of sockets
sndbuf = 8192

# TCP timeout seconds
tcp_timeout = 30

# TCP timewait seconds
tcp_timewait = 0

# Interface to print stats
stat_print = vale00:vpp2}0

Test

Enter to <mtcp root>/apps/example/, and create a new directory using the following command:

mkdir www
cd www/
nano index.html

Write something in the file, save and close it.

Enter to <mtcp root>/apps/example/ and use the following command to start the http server:

sudo ip netns exec vpp1 ./epserver -p www/ -f epserver.conf -c 1 -N 1

On another terminal, Enter to <mtcp root>/apps/example/ and use the following command to start the epwget client:

sudo ip netns exec vpp2 ./epwget 10.0.42.3/index.html 1 -N 1 -s 2 -o output.txt

In VPP CLI, use the command show int:

vpp# show int
              Name               Idx       State          Counter          Count     
local0                            0        down      
netmap-vale00:vpp1                5         up       rx packets                   501
                                                     rx bytes                   48460
                                                     tx packets                   601
                                                     tx bytes                   51860
netmap-vale00:vpp2                6         up       rx packets                   601
                                                     rx bytes                   51860
                                                     tx packets                   501
                                                     tx bytes                   48460
pg/stream-0                       1        down      
pg/stream-1                       2        down      
pg/stream-2                       3        down      
pg/stream-3                       4        down

You can also use the command cat to see the http response output:

cat output.txt.0
HTTP/1.1 200 OK