Difference between revisions of "VPP/Tutorial Routing and Switching"
(added ethtool turning off checksumming so TCP flows will pass (re: https://lists.fd.io/pipermail/vpp-dev/2016-August/002207.html and similar to https://github.com/weaveworks/weave/issues/2354)) |
m (→Virtual Network Setup) |
||
| Line 67: | Line 67: | ||
VPP supports two non-DPDK drivers for communications with Linux namespaces: | VPP supports two non-DPDK drivers for communications with Linux namespaces: | ||
| − | * '''veth''' interfaces with vpp ''host'' interfaces (based on efficient AF_PACKET shared memory with kernel). Click here for more information about [http://blog.scottlowe.org/2013/09/04/introducing-linux-network-namespaces/ | + | * '''veth''' interfaces with vpp ''host'' interfaces (based on efficient AF_PACKET shared memory with kernel). Click here for more information about [http://blog.scottlowe.org/2013/09/04/introducing-linux-network-namespaces/ veth interfaces and Linux network namespaces]. |
* '''tap''' interfaces from Linux's tuntap support. | * '''tap''' interfaces from Linux's tuntap support. | ||
Revision as of 19:33, 5 January 2017
This is a basic tutorial intended for VPP newcomers. It introduces main CLI commands for creating a switched and routed network. For the purpose of this tutorial, a virtual network will be created by the mean of Linux network namespaces, veth, and tap interfaces.
This tutorial was given as part of the June 2016 Paris summit (video here).
Contents
Prerequisites
For this tutorial, you will need a Linux environment with VPP installed. You can follow this tutorial to setup your development environment.
Running vpp
Start VPP
If you installed VPP using the vagrant tutorial, do vagrant up and vagrant ssh in VPP's vagrant directory. VPP should be already be running.
~$ sudo vppctl show version vpp v1.0.0-433~gb53693a-dirty built by vagrant on localhost at Wed May 4 03:03:02 PDT 2016
If it is not running, try:
~$ sudo start vpp
If you played with VPP. It might be a good idea to restart it.
~$ sudo stop vpp ~$ sudo start vpp
If you have installed vpp through other means, you can execute VPP directly.
~$ sudo vpp unix { interactive log /tmp/vpp.log full-coredump } api-trace { on }
This command will start VPP in interactive mode. Which means you will be able to enter VPP CLI commands just like if they were executed using sudo vppctl your command.
From now on, we will use vppctl and ommit sudo, but you can use VPP's interactive mode if you want.
Basic VPP commands
Execute the following commands.
~# vppctl show interface
Name Idx State Counter Count
GigabitEthernet0/8/0 5 down
GigabitEthernet0/9/0 6 down
local0 0 down
pg/stream-0 1 down
pg/stream-1 2 down
pg/stream-2 3 down
pg/stream-3 4 down
In this example, the VM has two PCI interfaces, owned by DPDK drivers. DPDK runs in polling mode, which means that the single VPP thread currently takes 100% CPU.
~# top PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8845 root 20 0 2123488 26908 9752 R 97.1 0.7 1:01.59 vpp_main
VPP Debug CLI implements a lot of different commands. You can display CLI help with '?'.
~# vppctl ? ... ~# vppctl show ? ...
Virtual Network Setup
VPP supports two non-DPDK drivers for communications with Linux namespaces:
- veth interfaces with vpp host interfaces (based on efficient AF_PACKET shared memory with kernel). Click here for more information about veth interfaces and Linux network namespaces.
- tap interfaces from Linux's tuntap support.
This tutorial is going to use 3 different namespaces: ns0, ns1, and ns2. ns0 and ns1 will be connected to VPP by the mean of veth interfaces, while ns2 will be using a tap interface.
ns0, ns1 and veth interfaces
Let's configure ns0.
~# ip netns add ns0 ~# ip link add vpp0 type veth peer name vethns0 ~# ip link set vethns0 netns ns0 ~# ip netns exec ns0 ip link set lo up ~# ip netns exec ns0 ip link set vethns0 up ~# ip netns exec ns0 ip addr add 2001::1/64 dev vethns0 ~# ip netns exec ns0 ip addr add 10.0.0.1/24 dev vethns0 ~# ip netns exec ns0 ethtool -K vethns0 rx off tx off
~# ip link set vpp0 up
And do the same for ns1.
~# ip netns add ns1 ~# ip link add vpp1 type veth peer name vethns1 ~# ip link set vethns1 netns ns1 ~# ip netns exec ns1 ip link set lo up ~# ip netns exec ns1 ip link set vethns1 up ~# ip netns exec ns1 ip addr add 2001::2/64 dev vethns1 ~# ip netns exec ns1 ip addr add 10.0.0.2/24 dev vethns1 ~# ip netns exec ns1 ethtool -K vethns1 rx off tx off ~# ip link set vpp1 up
Now on VPP side.
Let's create the host (af-packet) interfaces and set them up.
~# vppctl create host-interface name vpp0 ~# vppctl create host-interface name vpp1 ~# vppctl set interface state host-vpp0 up ~# vppctl set interface state host-vpp1 up
Host interfaces are created with names like host-<linux-ifname>.
~# vppctl show interface
Name Idx State Counter Count
GigabitEthernet0/8/0 5 down
GigabitEthernet0/9/0 6 down
host-vpp0 7 up
host-vpp1 8 up rx packets 2
rx bytes 140
drops 2
local0 0 down
pg/stream-0 1 down
pg/stream-1 2 down
pg/stream-2 3 down
pg/stream-3 4 down
~$ vppctl show hardware
Name Idx Link Hardware
GigabitEthernet0/8/0 5 down GigabitEthernet0/8/0
Ethernet address 08:00:27:1b:35:da
Intel 82540EM (e1000)
carrier up full duplex speed 1000 mtu 9216
GigabitEthernet0/9/0 6 down GigabitEthernet0/9/0
Ethernet address 08:00:27:59:74:1a
Intel 82540EM (e1000)
carrier up full duplex speed 1000 mtu 9216
host-vpp0 7 up host-vpp0
Ethernet address 02:fe:22:32:72:72
Linux PACKET socket interface
host-vpp1 8 up host-vpp1
Ethernet address 02:fe:17:f7:19:ae
Linux PACKET socket interface
[...]
Give ns2 a tap interface
tap connect is used to create a tap interface. It can also be used to connect to an existing detached tap interface.
~# vppctl tap connect tap0 ~# vppctl show int
[...] tap-0 10 down drops 8 [...]
The tap interface is created in VPP's namespace (default one). We need to move it to ns2 and configure it.
~# ip netns add ns2 ~# ip link set tap0 netns ns2 ~# ip netns exec ns2 ip link set lo up ~# ip netns exec ns2 ip link set tap0 up ~# ip netns exec ns2 ip addr add 10.0.1.1/24 dev tap0 ~# ip netns exec ns2 ip addr add 2001:1::1/64 dev tap0
Now we are good to go to configure VPP.
Routing and Switching
This section will show how to configure our little virtual network with switching and routing.
Switching ns0 and ns1
In this section, we are going to switch ns0, ns1, and VPP within a common bridging domain.
~# vppctl set interface l2 bridge host-vpp0 1 ~# vppctl set interface l2 bridge host-vpp1 1
The two interfaces are now bridged ! Let's try and see packets coming in and out by using VPP's tracing.
~# vppctl trace add af-packet-input 8
~# ip netns exec ns0 ping6 2001::2
~# vppctl show trace
Packet 1
00:08:21:483138: af-packet-input
af_packet: hw_if_index 7 next-index 1
tpacket2_hdr:
status 0x20000001 len 86 snaplen 86 mac 66 net 80
sec 0x5729ffe1 nsec 0xee2cbd5 vlan_tci 0
00:08:21:484336: ethernet-input
IP6: 3e:ad:9f:23:9f:66 -> 33:33:ff:00:00:02
00:08:21:484350: l2-input
l2-input: sw_if_index 7 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66
00:08:21:484353: l2-learn
l2-learn: sw_if_index 7 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66 bd_index 1
00:08:21:484748: l2-flood
l2-flood: sw_if_index 7 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66 bd_index 1
00:08:21:485086: l2-output
l2-output: sw_if_index 8 dst 33:33:ff:00:00:02 src 3e:ad:9f:23:9f:66
00:08:21:485105: host-vpp1-output
host-vpp1
IP6: 3e:ad:9f:23:9f:66 -> 33:33:ff:00:00:02
ICMP6: 2001::1 -> ff02::1:ff00:2
tos 0x00, flow label 0x0, hop limit 255, payload length 32
ICMP neighbor_solicitation checksum 0xbc60
target address 2001::2
Packet 2
00:08:21:485533: af-packet-input
af_packet: hw_if_index 8 next-index 1
tpacket2_hdr:
status 0x20000001 len 86 snaplen 86 mac 66 net 80
sec 0x5729ffe1 nsec 0xf07ee19 vlan_tci 0
00:08:21:485536: ethernet-input
IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66
00:08:21:485538: l2-input
l2-input: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f
00:08:21:485540: l2-learn
l2-learn: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1
00:08:21:485542: l2-fwd
l2-fwd: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1
00:08:21:485544: l2-output
l2-output: sw_if_index 7 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f
00:08:21:485554: host-vpp0-output
host-vpp0
IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66
ICMP6: 2001::2 -> 2001::1
tos 0x00, flow label 0x0, hop limit 255, payload length 32
ICMP neighbor_advertisement checksum 0x3101
target address 2001::2
Packet 3
00:08:21:485573: af-packet-input
af_packet: hw_if_index 7 next-index 1
tpacket2_hdr:
status 0x20000001 len 118 snaplen 118 mac 66 net 80
sec 0x5729ffe1 nsec 0xf08a8c5 vlan_tci 0
00:08:21:485574: ethernet-input
IP6: 3e:ad:9f:23:9f:66 -> 9a:90:35:8a:b4:7f
00:08:21:485575: l2-input
l2-input: sw_if_index 7 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66
00:08:21:485575: l2-learn
l2-learn: sw_if_index 7 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66 bd_index 1
00:08:21:485576: l2-fwd
l2-fwd: sw_if_index 7 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66 bd_index 1
00:08:21:485576: l2-output
l2-output: sw_if_index 8 dst 9a:90:35:8a:b4:7f src 3e:ad:9f:23:9f:66
00:08:21:485577: host-vpp1-output
host-vpp1
IP6: 3e:ad:9f:23:9f:66 -> 9a:90:35:8a:b4:7f
ICMP6: 2001::1 -> 2001::2
tos 0x00, flow label 0x0, hop limit 64, payload length 64
ICMP echo_request checksum 0xd538
Packet 4
00:08:21:485589: af-packet-input
af_packet: hw_if_index 8 next-index 1
tpacket2_hdr:
status 0x20000001 len 118 snaplen 118 mac 66 net 80
sec 0x5729ffe1 nsec 0xf08efa8 vlan_tci 0
00:08:21:485590: ethernet-input
IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66
00:08:21:485591: l2-input
l2-input: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f
00:08:21:485591: l2-learn
l2-learn: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1
00:08:21:485592: l2-fwd
l2-fwd: sw_if_index 8 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f bd_index 1
00:08:21:485592: l2-output
l2-output: sw_if_index 7 dst 3e:ad:9f:23:9f:66 src 9a:90:35:8a:b4:7f
00:08:21:485592: host-vpp0-output
host-vpp0
IP6: 9a:90:35:8a:b4:7f -> 3e:ad:9f:23:9f:66
ICMP6: 2001::2 -> 2001::1
tos 0x00, flow label 0x0, hop limit 64, payload length 64
ICMP echo_reply checksum 0xd438
~# vppctl clear trace
You should be able to see NDP packets followed by echo requests and responses.
The two namespaces are connected but VPP is not. Let's change that by adding a loopback interface to the bridge domain.
~# vppctl create loopback interface
~# vppctl show interface
Name Idx State Counter Count
[...]
loop0 10 down
[...]
The additional bvi option means that this interface is used to send, receive and forward packets for this bridge domain.
~# vppctl set interface l2 bridge loop0 1 bvi ~# vppctl set interface state loop0 up
Now let's take a look at current bridging state.
~# vppctl show bridge-domain ID Index Learning U-Forwrd UU-Flood Flooding ARP-Term BVI-Intf 0 0 off off off off off local0 1 1 on on on on off loop0 ~# vppctl show bridge-domain 1 detail ID Index Learning U-Forwrd UU-Flood Flooding ARP-Term BVI-Intf 1 1 on on on on off loop0
Interface Index SHG BVI VLAN-Tag-Rewrite
loop0 10 0 * none
host-vpp1 8 0 - none
host-vpp0 7 0 - none
And configure IP addresses on the loopback interface.
~# vppctl set interface ip address loop0 2001::ffff/64 ~# vppctl set interface ip address loop0 10.0.0.10/24
VPP is now plugged to the bridge and configured. You should be able to ping VPP.
~# vppctl trace add af-packet-input 15 ~# ip netns exec ns0 ping6 2001::ffff ~# ip netns exec ns0 ping 10.0.0.10 ~# vppctl show trace ~# vppctl clear trace
The Layer 2 fib can also be displayed.
~# vppctl show l2fib verbose
Mac Address BD Idx Interface Index static filter bvi refresh timestamp
3e:ad:9f:23:9f:66 1 host-vpp0 7 0 0 0 0 0
de:ad:00:00:00:00 1 loop0 10 1 0 1 0 0
9a:90:35:8a:b4:7f 1 host-vpp1 8 0 0 0 0 0
Routing
Now that ns0 and ns1 are switched, let's configure the tap interface such that we can do routing between ns2 and ns0+ns1.
~# vppctl set interface state tap-0 up ~# vppctl set interface ip address tap-0 2001:1::ffff/64 ~# vppctl set interface ip address tap-0 10.0.1.10/24
We can take a look at IP routing tables.
~# vppctl show ip fib
Table 0, fib_index 0, flow hash: src dst sport dport proto
Destination Packets Bytes Adjacency
10.0.0.0/24 0 0 weight 1, index 3
arp loop0 10.0.0.10/24
10.0.0.1/32 0 0 weight 1, index 5
loop0
IP4: de:ad:00:00:00:00 -> 3e:ad:9f:23:9f:66
10.0.0.10/32 2 196 weight 1, index 4
local 10.0.0.10/24
10.0.1.0/24 0 0 weight 1, index 6
arp tap-0 10.0.1.10/24
10.0.1.10/32 0 0 weight 1, index 7
local 10.0.1.10/24
~# vppctl show ip6 fib
FIB lookup table: 65536 buckets, 32 MB heap
17 objects, 513k of 516k used, 620 free, 0 reclaimed, 2k overhead, 32764k capacity
VRF 0, fib_index 0, flow hash: src dst sport dport proto
Destination Packets Bytes Adjacency
2001::/64 0 0 weight 1, index 7
arp loop0 2001::ffff/64
2001::1/128 3 354 weight 1, index 14
loop0
IP6: de:ad:00:00:00:00 -> 3e:ad:9f:23:9f:66
2001::ffff/128 3 312 weight 1, index 8
local 2001::ffff/64
2001:1::/64 0 0 weight 1, index 15
arp tap-0 2001:1::ffff/64
2001:1::ffff/128 0 0 weight 1, index 16
local 2001:1::ffff/64
fe80::/64 0 0 weight 1, index 20
arp tap-0 fe80::2cdb:ceff:fe44:b20d/64
fe80::2cdb:ceff:fe44:b20d/128 0 0 weight 1, index 12
local fe80::2cdb:ceff:fe44:b20d/64
fe80::dcad:ff:fe00:0/128 0 0 weight 1, index 13
local fe80::dcad:ff:fe00:0/64
ff02::1/128 0 0 weight 1, index 5
local
ff02::2/128 0 0 weight 1, index 4
local
ff02::16/128 0 0 weight 1, index 6
local
ff02::1:ff00:0/104 1 72 weight 1, index 3
local
On VPP side, we are good to go. But we just need to setup default routes in every namespaces. Depending on your linux configuration, IPv6 routes may already exist as VPP automatically sends IPv6 router advertisements.
~# ip netns exec ns0 ip route add default via 10.0.0.10 ~# ip netns exec ns0 ip -6 route add default via 2001::ffff ~# ip netns exec ns1 ip route add default via 10.0.0.10 ~# ip netns exec ns1 ip -6 route add default via 2001::ffff ~# ip netns exec ns2 ip route add default via 10.0.1.10 ~# ip netns exec ns2 ip -6 route add default via 2001:1::ffff
And now we can ping through VPP forwarding engine.
~# vppctl trace add af-packet-input 15 ~# ip netns exec ns0 ping6 2001:1::1 ~# ip netns exec ns0 ping 10.0.1.1 ~# vppctl show trace ~# vppctl clear trace
Cleaning up
In order to cleanup this hands-on:
~# ip netns del ns0 ~# ip netns del ns1 ~# ip netns del ns2 ~# ip link del vpp0 ~# ip link del vpp1 ~# ip link del tap0 ~# stop vpp ~# start vpp
