Difference between revisions of "VPP/Build, install, and test images"

Latest revision as of 17:50, 22 December 2018

This page explains how to build, install, and smoke-test a VPP package.

These procedures assume that you have a working development environment available. If you are not sure, see Pulling, Building, Hacking, and Pushing VPP Code.

Running Vagrant

For more information about using Vagrant on a command-line interface (CLI), see: https://docs.vagrantup.com/v2/cli/index.html

cd to the vagrant directory

In the command-line interface, navigate to the directory that has the pre-configured Vagrantfile. (In the following sample command, <install_dir> is the directory where you unzipped or cloned the VPP software.)

cd <install_dir>/build-root/vagrant/

NOTE: The .../build-root directory contains the files that make up most of the build system. It contains all of the generic targets, including: xxx-build, xxx-rebuild, xxx-install, xxx-clean, xxx-wipe, xxx-configure, and xxx-find-source.

Adding network interfaces to Vagrant

By default, Virtual Machine will have access to only private networks, specifically local0 and pg interfaces. But they won't be useful for tutorial. To make Gigabit*(public network) interfaces appear, find comment in the Vagrant file.

# Define some physical ports for your VMs to be used by DPDK

Set nics to the number of Gigabit interfaces needed and config.vm.network to "private_network".
Code should look like following :

  nics = 1
  if ENV.key?('VPP_VAGRANT_NICS')
    nics = ENV['VPP_VAGRANT_NICS'].to_i(10)
  end
  for i in 1..nics
    config.vm.network "private_network", type: "dhcp"
  end

Build the VM with Vagrant

By default this will build an Ubuntu 14.0.4 VM.

If you wish instead to build a Centos7 VM instead:

(export VPP_VAGRANT_DISTRO=centos7;vagrant up)

If you wish to use VMWareFusion as your provider, use:

(export VAGRANT_DEFAULT_PROVIDER=vmware_fusion;vagrant up)

Note: If you use vmware please see Vagrant VMWare Provider

When you first start Vagrant, it is normal for it to run for several minutes, building the VM, building VPP, and then a README will be displayed telling you how to run VPP.

Use the Vagrant up command to cause Vagrant to start. Vagrant uses the Vagrantfile in the current working directory.

vagrant up

Access the shell

Use the Vagrant SSH command to access the running Vagrant machine and give you access to a shell.

vagrant ssh

If you wish to forward X-windows server requests, use this variation:

vagrant ssh -- -X

Build A VPP Package

The following steps explain how to build a package. Steps 1 through 3 show how to build from the build-root directory.

For an alternative method, use the top level make command. Steps 1A through 3A show this method.

Step 1: Navigate to the build-root directory

After connecting vagrant through ssh

cd /vpp/build-root

Step 2: (Optional) Update the tree and clean up the build directories

You can execute a git pull command to obtain the latest updates from the repository. It's also a good idea to execute a "make distclean" command after you do this. Execute the bootstrap script to make sure that build paths and build tools are in a clean state.

git pull
make distclean
./bootstrap.sh
git clean {-x|-X}

Step 3: Build Debian or RPM Packages

Execute the make command to start the build process. You can review the makefile (located in the build-root directory) to examine the build parameters.

To build a Debian package:

make V=0 PLATFORM=vpp TAG=vpp install-deb

To build an RPM package:

make V=0 PLATFORM=vpp TAG=vpp install-rpm

On Ubuntu 15.10 I needed to execute the line above using 'sudo' as the dh_install failed with errors.

Step 1A: Navigate to the top level directory.

After connecting vagrant through ssh

cd /vpp

For list of commands, execute

make

Step 2A: (Optional) Clean up build directories and execute fresh release build.

You can execute a git pull command to obtain the latest updates from the repository. It's also a good idea to execute a "make distclean" command after you do this. Execute the bootstrap script to make sure that build paths and build tools are in a clean state.

git pull
make wipe-release
make install-dep
make build-release

Make also allows for multiple targets.

git pull
make wipe-release install-dep build-release

Step 3A: Build Debian or RPM Packages

Execute the make command to start the build process. You can review the makefile (located in the build-root directory) to examine the build parameters.

To build a Debian package:

make pkg-deb

To build an RPM package:

make pkg-rpm

Install A VPP Package

After you have successfully built the VPP packages, you need to install them in order to run.

Step 1: Install the packages

Install the packages. You can install packages with standard package installation tools (dpkg -i on debian/ubuntu, rpm on RedHat/CentOS).

NOTE: You will need root privileges.

To install the package(s) on a Debian operating system:

1. Login with root privileges
2. Type the dpkg installation command:
   

sudo dpkg -i /vpp/build-root/*.deb

To install the package(s) using Redhat Package Manager (RPM):

1. Login with root privileges
2. Type the RPM installation command:
   

sudo rpm -i /vpp/build-root/*.rpm

NOTE: On Ubuntu/Debian and RedHat/CentOS systems, the vpp engine consists of multiple installation packages:

• vpp - main VPP process
• vpp-lib - dynamically linked libraries
• vpp-dev - development support files and examples
• vpp-dpdk-dkms - DKMS based DPDK kernel module package (only on Debian/Ubuntu)

Step 2: Examine the startup configuration

After successful installation, vpp installs a startup config file named startup.conf in the /etc/vpp directory. You can modify that file if appropriate.

Here is a sample:

unix {
  nodaemon
  log /var/log/vpp/vpp.log
  cli-listen localhost:5002
  full-coredump
}
api-trace {
  on
}

dpdk {
  socket-mem 1024
}

Note: In VPP 18.04, the default log file location was moved from '/tmp/vpp.log' to '/var/log/vpp/vpp.log' . The VPP code is indifferent to the file location. However, if SELinux is enabled, then the new location is required for the file to be properly labelled. Check your local startup.conf file for the log file location on your system.

Step 3: Start vpp

To start the data plane:

1. Login with root privileges
2. Type the start vpp command:
   

(Debian / Ubuntu)

sudo start vpp

On Ubuntu 15.10 you need to have Upstart installed as they switched to systemd. Use 'sudo apt-get install upstart-sysv' then reboot.

(Centos)

sudo service vpp start

3. Wait for a few seconds
4. Check the interface list.
   It's a good idea to check the interface list to see which interfaces vpp has been discovered.
   To show the interface list, execute the show command:
   

sudo vppctl show interface  # The interface set varies depending on the system configuration.
              Name               Idx       State          Counter          Count
GigabitEthernet2/2/0              5        down
GigabitEthernet2/3/0              6        down
GigabitEthernet2/4/0              7        down
GigabitEthernet2/5/0              8        down
GigabitEthernet2/6/0              9        down
GigabitEthernet2/7/0              10       down
local0                            0        down
pg/stream-0                       1        down
pg/stream-1                       2        down
pg/stream-2                       3        down
pg/stream-3                       4        down

Step 4: Change blacklist behavior

If vpp seems not to have discovered an interface that you expected to see, it's likely that the data-plane blacklisted it. The VPP process blacklists interfaces whose corresponding Linux interfaces are up.

You can change this blacklist behavior. For example, if you want vpp to take over an interface eth1 which has been blacklisted, execute the following commands.

To remove eth1 from the running VPP blacklist:

sudo ifconfig  
# Save eth1 IP
sudo ifconfig eth1 down
sudo ip addr flush dev ethX
sudo stop vpp
sudo start vpp

Test The VPP Package

It's a good idea to perform a few basic smoke-tests. In other words, it's a good idea to perform some basic tasks to make sure that vpp is running as expected.

Step 1: Configure and enable an interface

Use the set int command to configure an interface. For this test, configure an ipv4 address on an interface, and enable the interface. Use the sudo command in combination with the vpp engine's vppctl command to execute the command with root privilege:

Replace X.X.X.X with saved IP.

sudo vppctl set int ip address GigabitEthernet2/2/0 X.X.X.X/24
sudo vppctl set int state GigabitEthernet2/2/0 up

You can also use the vppctl command in combination with native Linux commands such as grep:

sudo vppctl show int | grep State

Use the ip probe command to probe an adjacent system, by sending an ipv4 icmp echo-request:

sudo vppctl ip probe 192.168.1.2 GigabitEthernet2/2/0
Resolved 192.168.1.2

Use the sh ip command to check the ARP and ipv4 FIB tables:

sudo vppctl  sh ip arp
 Time      FIB        IP4      Stat      Ethernet              Interface     
2782.7392   0     192.168.1.2        00:50:56:b7:05:bb    GigabitEthernet2/2/0

sudo vppctl sh ip fib
Table 0, fib_index 0, flow hash: src dst sport dport proto
     Destination         Packets          Bytes         Adjacency
192.168.1.0/24                     0               0 weight 1, index 5
                                                     arp GigabitEthernet2/2/0 192.168.1.1/24
192.168.1.1/32                     0               0 weight 1, index 4
                                                     local 192.168.1.1/24
192.168.1.2/32                     0               0 weight 1, index 3
                                                     GigabitEthernet2/2/0
                                                     IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb

Step 2: Check the error counters

After pinging the vpp engine from 192.168.1.2, use the show error command to check the error counters.

In this case, all of the counters represent normal events:

sudo vppctl show error
         Count                     Node               Reason
             1                 ip4-arp                ARP requests sent
          9810             ip4-icmp-input             echo replies sent
             2                arp-input               ARP replies receive

Step 3: Reset the statistics

Use the clear run command to reset the per-node runtime statistics:

sudo vppctl clear run

Step 4: Check for new statistics

Send a small number of ipv4 icmp echo replies from the adjacent Linux system. Note that the vpp data plane is capable of answering several MPPS worth of ipv4 icmp echo replies; as the vector size increases, the clocks/pkt statistics will improve drastically.

Use the show run command to see the new statistics.

sudo vppctl show run
Time 18.1, average vectors/node 1.00, last 128 main loops 0.00 per node 0.00
  vector rates in 2.2068e-1, out 2.2068e-1, drop 0.0000e0, punt 0.0000e0
             Name                 State         Calls          Vectors        Suspends         Clocks       Vectors/Call
GigabitEthernet2/2/0-output      active                  4               4               0          1.71e3            1.00
GigabitEthernet2/2/0-tx          active                  4               4               0          7.49e3            1.00
api-rx-from-ring                any wait                 0               0               1          9.90e3            0.00
cnat-db-scanner                 any wait                 0               0            1813          1.17e3            0.00
dpdk-input                       polling          60666120               4               0          4.04e9            0.00
dpdk-process                    any wait                 0               0               4          1.29e7            0.00
ethernet-input                   active                  4               4               0          5.51e3            1.00
gmon-process                    time wait                0               0               3          5.59e3            0.00
ip4-icmp-echo-request            active                  4               4               0          1.85e3            1.00
ip4-icmp-input                   active                  4               4               0          1.51e3            1.00
ip4-input                        active                  4               4               0          3.49e3            1.00
ip4-local                        active                  4               4               0          3.23e3            1.00
ip4-lookup                       active                  4               4               0          4.49e3            1.00
ip4-rewrite-local                active                  4               4               0          3.72e3            1.00
ip6-icmp-neighbor-discovery-ev  any wait                 0               0              18          6.74e3            0.00
unix-cli-127.0.0.1:48387         active                  0               0               9          7.15e4            0.00
unix-epoll-input                 polling          60666120               0               0          5.15e2            0.00
vpe-oam-process                 any wait                 0               0               9          6.45e3            0.00

Step 5: Start a packet trace

Use the trace add command to start a packet-tracer capture:

sudo vppctl trace add dpdk-input 10

Step 6: Verify test packets

Ping 192.168.1.1 from 192.168.1.2, AKA send ipv4 icmp echo request packets. If the ping succeeds, you should be all set.

Use the show trace command to display the packet trace.

To show trace information:

sudo vppctl show trace

The packet trace will look something like this:

------------------- Start of thread 0 vpp_main -------------------
Packet 1

01:10:14:046893: dpdk-input
 GigabitEthernet2/2/0 rx queue 0
 buffer 0x631e: current data 0, length 102, free-list 0, totlen-nifb 0, trace 0x0
 PKT MBUF: port 0, nb_segs 1, pkt_len 102
   buf_len 2304, data_len 102, ol_flags 0x0,
   packet_type 0x0
 IP4: 00:50:56:b7:05:bb -> 00:50:56:b7:05:bc
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7c1f
   fragment id 0x3b36, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x7cd1
01:10:14:046989: ethernet-input
 IP4: 00:50:56:b7:05:bb -> 00:50:56:b7:05:bc
01:10:14:047010: ip4-input
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7c1f
   fragment id 0x3b36, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x7cd1
01:10:14:047013: ip4-local
 fib: 0 adjacency: local 192.168.1.1/24 flow hash: 0x00000000
01:10:14:047017: ip4-icmp-input
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7c1f
   fragment id 0x3b36, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x7cd1
01:10:14:047019: ip4-icmp-echo-request
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7c1f
   fragment id 0x3b36, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x7cd1
01:10:14:047019: ip4-rewrite-local
 fib: 0 adjacency: GigabitEthernet2/2/0
                   IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb flow hash: 0x00000000
 IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb
 ICMP: 192.168.1.1 -> 192.168.1.2
   tos 0x00, ttl 64, length 84, checksum 0x6383
   fragment id 0x53d2, flags DONT_FRAGMENT
 ICMP echo_reply checksum 0x84d1
01:10:14:047021: GigabitEthernet2/2/0-output
 GigabitEthernet2/2/0
 IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb
 ICMP: 192.168.1.1 -> 192.168.1.2
   tos 0x00, ttl 64, length 84, checksum 0x6383
   fragment id 0x53d2, flags DONT_FRAGMENT
 ICMP echo_reply checksum 0x84d1
01:10:14:047022: GigabitEthernet2/2/0-tx
 GigabitEthernet2/2/0 tx queue 0
 buffer 0x631e: current data 0, length 102, free-list 0, totlen-nifb 0, trace 0x0
 IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb
 ICMP: 192.168.1.1 -> 192.168.1.2
   tos 0x00, ttl 64, length 84, checksum 0x6383
   fragment id 0x53d2, flags DONT_FRAGMENT
 ICMP echo_reply checksum 0x84d1

Packet 2

01:10:15:044487: dpdk-input
 GigabitEthernet2/2/0 rx queue 0
 buffer 0x6345: current data 0, length 102, free-list 0, totlen-nifb 0, trace 0x1
 PKT MBUF: port 0, nb_segs 1, pkt_len 102
   buf_len 2304, data_len 102, ol_flags 0x0,
   packet_type 0x0
 IP4: 00:50:56:b7:05:bb -> 00:50:56:b7:05:bc
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7bc7
   fragment id 0x3b8e, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x64d4
01:10:15:044496: ethernet-input
 IP4: 00:50:56:b7:05:bb -> 00:50:56:b7:05:bc
01:10:15:044500: ip4-input
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7bc7
   fragment id 0x3b8e, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x64d4
01:10:15:044504: ip4-local
 fib: 0 adjacency: local 192.168.1.1/24 flow hash: 0x00000000
01:10:15:044506: ip4-icmp-input
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7bc7
   fragment id 0x3b8e, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x64d4
01:10:15:044507: ip4-icmp-echo-request
 ICMP: 192.168.1.2 -> 192.168.1.1
   tos 0x00, ttl 64, length 84, checksum 0x7bc7
   fragment id 0x3b8e, flags DONT_FRAGMENT
 ICMP echo_request checksum 0x64d4
01:10:15:044507: ip4-rewrite-local
 fib: 0 adjacency: GigabitEthernet2/2/0
                   IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb flow hash: 0x00000000
 IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb
 ICMP: 192.168.1.1 -> 192.168.1.2
   tos 0x00, ttl 64, length 84, checksum 0x82be
   fragment id 0x3497, flags DONT_FRAGMENT
 ICMP echo_reply checksum 0x6cd4
01:10:15:044509: GigabitEthernet2/2/0-output
 GigabitEthernet2/2/0
 IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb
 ICMP: 192.168.1.1 -> 192.168.1.2
   tos 0x00, ttl 64, length 84, checksum 0x82be
   fragment id 0x3497, flags DONT_FRAGMENT
 ICMP echo_reply checksum 0x6cd4
01:10:15:044510: GigabitEthernet2/2/0-tx
 GigabitEthernet2/2/0 tx queue 0
 buffer 0x6345: current data 0, length 102, free-list 0, totlen-nifb 0, trace 0x1
 IP4: 00:50:56:b7:05:bc -> 00:50:56:b7:05:bb
 ICMP: 192.168.1.1 -> 192.168.1.2
   tos 0x00, ttl 64, length 84, checksum 0x82be
   fragment id 0x3497, flags DONT_FRAGMENT
 ICMP echo_reply checksum 0x6cd4

If you see packet information from ICMP: 192.168.1.2, then you know that vpp is successfully processing packets.

Update

If debian packages should be updated. See all VPP packages installed.

dpkg -l | grep vpp

You should get something like

# ii  vpp                                 1.0.0-190~g9f50b0b-dirty            amd64        Vector Packet Processing--executables
# ii  vpp-dbg                             1.0.0-190~g9f50b0b-dirty            amd64        Vector Packet Processing--debug symbols
# ii  vpp-dev                             1.0.0-190~g9f50b0b-dirty            amd64        Vector Packet Processing--development support
# ii  vpp-dpdk-dev                        1.0.0-190~g9f50b0b-dirty            amd64        Vector Packet Processing--development support
# ii  vpp-dpdk-dkms                       1.0.0-190~g9f50b0b-dirty            amd64        DPDK 2.1 igb_uio_driver
# ii  vpp-lib                             1.0.0-190~g9f50b0b-dirty            amd64        Vector Packet Processing--runtime libraries

Remove all of them

sudo dpkg --purge vpp vpp-dbg vpp-dev vpp-dpdk-dev vpp-dpdk-dkms vpp-lib

Build, Install, and Test VPP without using Packages

The following steps explain how to build, install, and test the code without building and installing packages. This is useful for doing development. Assume that <top> is the top directory of the tree that you cloned (e.g., git clone https://gerrit.fd.io/r/vpp) and should be spelled out appropriately.

Step 1: Navigate to the top level directory.

cd <top>

For list of commands, execute

make

Step 2: Clean up build directories and execute fresh release build.

You can execute a git pull command to obtain the latest updates from the repository. It's also possible, though typically not necessary, to execute a "make wipe-release" command after you do this. Execute the bootstrap script during the initial build sequence to make sure that build paths and build tools are in a clean state; this is a no-op on subsequent builds.

git pull
make wipe-release # rarely required
make install-dep  # extremely rarely required after the initial build sequence
make bootstrap    # only for first time build
make build-release

You can substitute "make wipe" and "make "build" for "make wipe-release" and "make build-release" respectively, to build debug versions. If that is done, references to "install-vpp-native" below should be changed to "install-vpp_debug-native".

Step 3: Copy and Examine the startup configuration

I am recommending that configuration files be kept under /etc, and that we do a one time copy of the default versions from our source tree. This way, we can modify configuration without worrying that our modifications are overwritten. Thus, copying the configuration files is a one time operation:

cd <top>/src/vpp/conf
sudo mkdir -p /etc/vpp
sudo cp startup.conf /etc/vpp/
sudo cp 80-vpp.conf /etc/sysctl.d

Here is a sample of startup.conf:

unix {
  nodaemon
  log /var/log/vpp/vpp.log
  cli-listen localhost:5002
  full-coredump
}
api-trace {
  on
}

dpdk {
  socket-mem 1024
}
# Alternate syntax to choose plugin path
 plugin_path <top>/build-root/install-vpp-native/vpp/lib64/vpp_plugins

Note in particular that the plugin_path must be set for this inline development mode.

You need to reboot after copying 80-vpp.conf to /etc/sysctl.d/ to pickup that configuration. Alternatively, you could accomplish the same thing using the sysctl command. I believe that a minimum of 3.5 to 4 GB system memory is required, but I do not know the precise number. You can probably figure it out if you read 80-vpp.conf correctly.

Note: In VPP 18.04, the default log file location was moved from '/tmp/vpp.log' to '/var/log/vpp/vpp.log' . The VPP code is indifferent to the file location. However, if SELinux is enabled, then the new location is required for the file to be properly labelled. Check your local startup.conf file for the log file location on your system.

Step 4: Start vpp

To start the data plane I use this script that I save in ~/vpp.sh and then I run "source ~/vpp.sh":

T=~/vpp/build-root/install-vpp-native/vpp
sudo /bin/rm -f /dev/shm/db /dev/shm/global_vm /dev/shm/vpe-api || return
sudo /sbin/modprobe uio_pci_generic || return
sudo $T/bin/vpp -c /etc/vpp/startup.conf

Next, we need some preparation to use the vppctl command. It calls the vpp_api_test command; both commands are in directory $T/bin. Because vpp_api_test is called with sudo access, the sudo secure_path must be set correctly. Using the command

sudoedit /etc/sudoers 

modify the definition of secure_path to read

Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:<top>/build-root/install-vpp-native/vpp/bin