VPP/Tutorial Routing and Switching
This is a basic tutorial intended for VPP newcomers. It introduces main CLI commands for creating a switched and routed network. For the purpose of this tutorial, a virtual network will be created by the mean of Linux network namespaces, veth, and tap interfaces.
Contents
Prerequisites
For this tutorial, you will need a Linux environment with VPP installed. You can follow this tutorial to setup your development environment.
Running vpp
Start VPP
If you installed VPP using the vagrant tutorial, do vagrant up
and vagrant ssh
in VPP's vagrant directory. VPP should be already be running.
~$ sudo vppctl show version vpp v1.0.0-433~gb53693a-dirty built by vagrant on localhost at Wed May 4 03:03:02 PDT 2016
If it is not running, try:
~$ sudo start vpp
If you played with VPP. It might be a good idea to restart it.
~$ sudo stop vpp ~$ sudo start vpp
If you have installed vpp through other means, you can execute VPP directly.
~$ sudo vpp unix { interactive log /tmp/vpp.log full-coredump } api-trace { on }
This command will start VPP in interactive mode. Which means you will be able to enter VPP CLI commands just like if they were executed using sudo vppctl your command
.
From now on, we will use vppctl
and ommit sudo
, but you can use VPP's interactive mode if you want.
Basic VPP commands
Execute the following commands.
~# vppctl show interface Name Idx State Counter Count GigabitEthernet0/8/0 5 down GigabitEthernet0/9/0 6 down local0 0 down pg/stream-0 1 down pg/stream-1 2 down pg/stream-2 3 down pg/stream-3 4 down
In this example, the VM has two PCI interfaces, owned by DPDK drivers. DPDK runs in polling mode, which means that the single VPP thread currently takes 100% CPU.
~# top PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8845 root 20 0 2123488 26908 9752 R 97.1 0.7 1:01.59 vpp_main
VPP Debug CLI implements a lot of different commands. You can display CLI help with '?'.
~# vppctl ? ... ~# vppctl show ? ...
Virtual Network Setup
VPP supports two non-DPDK drivers for communications with Linux namespaces:
- veth interfaces with vpp host interfaces (based on efficient AF_PACKET shared memory with kernel). Click here for more information about veth interfaces and Linux network namespaces.
- tap interfaces from Linux's tuntap support.
This tutorial is going to use 3 different namespaces: ns0, ns1, and ns2. ns0 and ns1 will be connected to VPP by the mean of veth interfaces, while ns2 will be using a tap interface.
ns0, ns1 and veth interfaces
Let's configure ns0.
~# ip netns add ns0 ~# ip link add vpp0 type veth peer name vethns0 ~# ip link set vethns0 netns ns0 ~# ip netns exec ns0 ip link set lo up ~# ip netns exec ns0 ip link set vethns0 up ~# ip netns exec ns0 ip addr add 2001::1/64 dev vethns0 ~# ip netns exec ns0 ip addr add 10.0.0.1/24 dev vethns0 ~# ip link set vpp0 up
And do the same for ns1.
~# ip netns add ns1 ~# ip link add vpp1 type veth peer name vethns1 ~# ip link set vethns1 netns ns1 ~# ip netns exec ns1 ip link set lo up ~# ip netns exec ns1 ip link set vethns1 up ~# ip netns exec ns1 ip addr add 2001::2/64 dev vethns1 ~# ip netns exec ns1 ip addr add 10.0.0.2/24 dev vethns1 ~# ip link set vpp1 up
Now on VPP side.
Let's create the host (af-packet) interfaces and set them up.
~# vppctl create host-interface name vpp0 ~# vppctl create host-interface name vpp1 ~# vppctl set interface state host-vpp0 up ~# vppctl set interface state host-vpp1 up
Host interfaces are created with names like host-<linux-ifname>.
~# vppctl show interface Name Idx State Counter Count GigabitEthernet0/8/0 5 down GigabitEthernet0/9/0 6 down host-vpp0 7 up host-vpp1 8 up rx packets 2 rx bytes 140 drops 2 local0 0 down pg/stream-0 1 down pg/stream-1 2 down pg/stream-2 3 down pg/stream-3 4 down
~$ vppctl show hardware
Name Idx Link Hardware GigabitEthernet0/8/0 5 down GigabitEthernet0/8/0 Ethernet address 08:00:27:1b:35:da Intel 82540EM (e1000) carrier up full duplex speed 1000 mtu 9216 GigabitEthernet0/9/0 6 down GigabitEthernet0/9/0 Ethernet address 08:00:27:59:74:1a Intel 82540EM (e1000) carrier up full duplex speed 1000 mtu 9216 host-vpp0 7 up host-vpp0 Ethernet address 02:fe:22:32:72:72 Linux PACKET socket interface host-vpp1 8 up host-vpp1 Ethernet address 02:fe:17:f7:19:ae Linux PACKET socket interface [...]
Give ns2 a tap interface
tap connect
is used to create a tap interface. It can also be used to connect to an existing detached tap interface.
~# vppctl tap connect tap0 ~# vppctl show int
[...] tap-0 10 down drops 8 [...]
The tap interface is created in VPP's namespace (default one). We need to move it to ns2 and configure it.
~# ip netns add ns2 ~# ip link set tap0 netns ns2 ~# ip netns exec ns2 ip link set lo up ~# ip netns exec ns2 ip link set tap0 up ~# ip netns exec ns2 ip addr add 10.0.1.1/24 dev tap0 ~# ip netns exec ns2 ip addr add 2001:1::1/64 dev tap0
Now we are good to go to configure VPP.
Routing and Switching
This section will show how to configure our little virtual network with switching and routing.
Switching ns0 and ns1
In this section, we are going to switch ns0, ns1, and VPP within a common bridging domain.
~# vppctl set interface l2 bridge host-vpp0 1 ~# vppctl set interface l2 bridge host-vpp1 1
The two interfaces are now bridged ! Let's try and see packets coming in and out by using VPP's tracing.
~# vppctl trace add af-packet-input 8 ~# ip netns exec ns0 ping6 2001::2 ~# vppctl show trace Packet 1 00:08:21:483138: af-packet-input af_packet: hw_if_index 7 next-index 1 tpacket2_hdr: status 0x20000001 len 86 snaplen 86 mac 66 net 80 sec 0x5729ffe1 nsec 0xee2cbd5 vlan_tci 0 00:08:21:484336: ethernet-input IP6: 3e:ad:9f:23:9f:66 -> 33:33:ff:00:00:02 00:08:21:484350: l2-input l2-input: sw_if_index 7 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66 00:08:21:484353: l2-learn l2-learn: sw_if_index 7 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66 bd_index 1 00:08:21:484748: l2-flood l2-flood: sw_if_index 7 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66 bd_index 1 00:08:21:485086: l2-output l2-output: sw_if_index 8 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66 00:08:21:485105: host-vpp1-output host-vpp1 IP6: 3e:ad:9f:23:9f:66 -> 33:33:ff:00:00:02 ICMP6: 2001::1 -> ff02::1:ff00:2 tos 0x00, flow label 0x0, hop limit 255, payload length 32 ICMP neighbor_solicitation checksum 0xbc60 target address 2001::2 Packet 2 00:08:21:485533: af-packet-input af_packet: hw_if_index 8 next-index 1 tpacket2_hdr: status 0x20000001 len 86 snaplen 86 mac 66 net 80 sec 0x5729ffe1 nsec 0xf07ee19 vlan_tci 0 00:08:21:485536: ethernet-input IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66 00:08:21:485538: l2-input l2-input: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f 00:08:21:485540: l2-learn l2-learn: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1 00:08:21:485542: l2-fwd l2-fwd: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1 00:08:21:485544: l2-output l2-output: sw_if_index 7 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f 00:08:21:485554: host-vpp0-output host-vpp0 IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66 ICMP6: 2001::2 -> 2001::1 tos 0x00, flow label 0x0, hop limit 255, payload length 32 ICMP neighbor_advertisement checksum 0x3101 target address 2001::2 Packet 3 00:08:21:485573: af-packet-input af_packet: hw_if_index 7 next-index 1 tpacket2_hdr: status 0x20000001 len 118 snaplen 118 mac 66 net 80 sec 0x5729ffe1 nsec 0xf08a8c5 vlan_tci 0 00:08:21:485574: ethernet-input IP6: 3e:ad:9f:23:9f:66 -> 9a:90:35:8a:b4:7f 00:08:21:485575: l2-input l2-input: sw_if_index 7 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66 00:08:21:485575: l2-learn l2-learn: sw_if_index 7 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66 bd_index 1 00:08:21:485576: l2-fwd l2-fwd: sw_if_index 7 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66 bd_index 1 00:08:21:485576: l2-output l2-output: sw_if_index 8 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66 00:08:21:485577: host-vpp1-output host-vpp1 IP6: 3e:ad:9f:23:9f:66 -> 9a:90:35:8a:b4:7f ICMP6: 2001::1 -> 2001::2 tos 0x00, flow label 0x0, hop limit 64, payload length 64 ICMP echo_request checksum 0xd538 Packet 4 00:08:21:485589: af-packet-input af_packet: hw_if_index 8 next-index 1 tpacket2_hdr: status 0x20000001 len 118 snaplen 118 mac 66 net 80 sec 0x5729ffe1 nsec 0xf08efa8 vlan_tci 0 00:08:21:485590: ethernet-input IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66 00:08:21:485591: l2-input l2-input: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f 00:08:21:485591: l2-learn l2-learn: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1 00:08:21:485592: l2-fwd l2-fwd: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1 00:08:21:485592: l2-output l2-output: sw_if_index 7 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f 00:08:21:485592: host-vpp0-output host-vpp0 IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66 ICMP6: 2001::2 -> 2001::1 tos 0x00, flow label 0x0, hop limit 64, payload length 64 ICMP echo_reply checksum 0xd438 ~# vppctl clear trace
You should be able to see NDP packets followed by echo requests and responses.
The two namespaces are connected but VPP is not. Let's change that by adding a loopback interface to the bridge domain.
~# vppctl create loopback interface ~# vppctl show interface Name Idx State Counter Count [...] loop0 10 down [...]
The additional bvi option means that this interface is used to send, receive and forward packets for this bridge domain.
~# vppctl set interface l2 bridge loop0 1 bvi ~# vppctl set interface state loop0 up
Now let's take a look at current bridging state.
~# vppctl show bridge-domain ID Index Learning U-Forwrd UU-Flood Flooding ARP-Term BVI-Intf 0 0 off off off off off local0 1 1 on on on on off loop0 ~# vppctl show bridge-domain 1 detail ID Index Learning U-Forwrd UU-Flood Flooding ARP-Term BVI-Intf 1 1 on on on on off loop0
Interface Index SHG BVI VLAN-Tag-Rewrite loop0 10 0 * none host-vpp1 8 0 - none host-vpp0 7 0 - none
And configure IP addresses on the loopback interface.
~# vppctl set interface ip address loop0 2001::ffff/64 ~# vppctl set interface ip address loop0 10.0.0.10/24
VPP is now plugged to the bridge and configured. You should be able to ping VPP.
~# vppctl trace add af-packet-input 15 ~# ip netns exec ns0 ping6 2001::ffff ~# ip netns exec ns0 ping 10.0.0.10 ~# vppctl show trace ~# vppctl clear trace
The Layer 2 fib can also be displayed.
~# show l2fib verbose Mac Address BD Idx Interface Index static filter bvi refresh timestamp 3e:ad:9f:23:9f:66 1 host-vpp0 7 0 0 0 0 0 de:ad:00:00:00:00 1 loop0 10 1 0 1 0 0 9a:90:35:8a:b4:7f 1 host-vpp1 8 0 0 0 0 0
Routing
Now that ns0 and ns1 are switched, let's configure the tap interface such that we can do routing between ns2 and ns0+ns1.
~# vppctl set interface state tap-0 up ~# vppctl set interface ip address tap-0 2001:1::ffff/64 ~# vppctl set interface ip address tap-0 10.0.1.10/24
We can take a look at IP routing tables.
~# vppctl show ip fib
Table 0, fib_index 0, flow hash: src dst sport dport proto Destination Packets Bytes Adjacency 10.0.0.0/24 0 0 weight 1, index 3 arp loop0 10.0.0.10/24 10.0.0.1/32 0 0 weight 1, index 5 loop0 IP4: de:ad:00:00:00:00 -> 3e:ad:9f:23:9f:66 10.0.0.10/32 2 196 weight 1, index 4 local 10.0.0.10/24 10.0.1.0/24 0 0 weight 1, index 6 arp tap-0 10.0.1.10/24 10.0.1.10/32 0 0 weight 1, index 7 local 10.0.1.10/24
~# vppctl show ip6 fib
FIB lookup table: 65536 buckets, 32 MB heap 17 objects, 513k of 516k used, 620 free, 0 reclaimed, 2k overhead, 32764k capacity VRF 0, fib_index 0, flow hash: src dst sport dport proto Destination Packets Bytes Adjacency 2001::/64 0 0 weight 1, index 7 arp loop0 2001::ffff/64 2001::1/128 3 354 weight 1, index 14 loop0 IP6: de:ad:00:00:00:00 -> 3e:ad:9f:23:9f:66 2001::ffff/128 3 312 weight 1, index 8 local 2001::ffff/64 2001:1::/64 0 0 weight 1, index 15 arp tap-0 2001:1::ffff/64 2001:1::ffff/128 0 0 weight 1, index 16 local 2001:1::ffff/64 fe80::/64 0 0 weight 1, index 20 arp tap-0 fe80::2cdb:ceff:fe44:b20d/64 fe80::2cdb:ceff:fe44:b20d/128 0 0 weight 1, index 12 local fe80::2cdb:ceff:fe44:b20d/64 fe80::dcad:ff:fe00:0/128 0 0 weight 1, index 13 local fe80::dcad:ff:fe00:0/64 ff02::1/128 0 0 weight 1, index 5 local ff02::2/128 0 0 weight 1, index 4 local ff02::16/128 0 0 weight 1, index 6 local ff02::1:ff00:0/104 1 72 weight 1, index 3 local
On VPP side, we are good to go. But we just need to setup default routes in every namespaces. Depending on your linux configuration, IPv6 routes may already exist as VPP automatically sends IPv6 router advertisements.
~# ip netns exec ns0 ip route add default via 10.0.0.10 ~# ip netns exec ns0 ip -6 route add default via 2001::ffff ~# ip netns exec ns1 ip route add default via 10.0.0.10 ~# ip netns exec ns1 ip -6 route add default via 2001::ffff ~# ip netns exec ns2 ip route add default via 10.0.1.10 ~# ip netns exec ns2 ip -6 route add default via 2001:1::ffff
And now we can ping through VPP forwarding engine.
~# vppctl trace add af-packet-input 15 ~# ip netns exec ns0 ping6 2001:1::1 ~# ip netns exec ns0 ping 10.0.1.1 ~# vppctl show trace ~# vppctl clear trace
Cleaning up
In order to cleanup this hands-on:
~# ip netns del ns0 ~# ip netns del ns1 ~# ip netns del ns2 ~# ip link del vpp0 ~# ip link del vpp1 ~# ip link del tap0 ~# stop vpp ~# start vpp