How to deploy VPP in EC2 instance and use it to connect two different VPCs
In this page it is shown how to deploy VPP (18.04 version) in an Amazon instance EC2, set VPP as Gateway towards Internet and connect two VMs belonging in two different VPCs through VPP and SRv6. Remember that to perform all of these operations you need to have an Amazon AWS account. Furthermore, it would be useful to have a prior knowledge of VPCs' concepts and Segment Routing.
Contents
VPP's deployment
First of all, go to the Amazon Management console and create a new instance. If you have not already done it, before creating the instance you need to create a VPC,in which you will put the instance, with its own security group and subnet (for the first one use only in the inbound and outbound rules the "All Traffic" option for IPv4 and IPv6). As Image use Ubuntu server 16.04 and as instance type one of the m5 type. Afterwards, create two new NICs always through the Amazon Console and attach them to the instance ( in total your VM should have 3 virtual NIC, one used to reach it through SSH and the other two for VPP. Before attach them to the VM, select them, go to the "actions" button and change the "source/dest check" option from enable to disable.
Then, access your VM via SSH and execute these commands:
Introduction
To make everything work you need to install some packages:
cd / sudo apt-get update sudo apt-get upgrade sudo apt-get install build-essential sudo apt-get install python-pip sudo apt-get install libnuma-dev sudo apt-get install make
Then you need to install DPDK:
sudo wget https://fast.dpdk.org/rel/dpdk-18.02.1.tar.xz sudo tar -xvf dpdk-18.02.1.tar.xz
Installation of VPP
Now you can install VPP:
cd / sudo mkdir /vpp sudo chmod 777 /vpp cd /vpp git clone https://gerrit.fd.io/r/vpp ./ sudo make install-dep sudo make build cd build-root make V=0 PLATFORM=vpp TAG=vpp install-deb
With the latter command you will install the last version of VPP. However for our purposes we need VPP 18.04 version:
export UBUNTU="xenial" export RELEASE=".stable.1804" sudo rm /etc/apt/sources.list.d/99fd.io.list echo "deb [trusted=yes] https://nexus.fd.io/content/repositories/fd.io$RELEASE.ubuntu.$UBUNTU.main/ ./" | sudo tee -a /etc/apt/sources.list.d/99fd.io.list sudo apt-get update sudo apt-get install vpp vpp-lib sudo apt-get install vpp-plugins vpp-dbg vpp-dev vpp-api-java vpp-api-python vpp-api- lua
Now you need to attach your two virtual NIC to VPP, in order to make all the flow of packets pass through VPP.
cd /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1 sudo modprobe uio sudo make install T=x86_64-native-linuxapp-gcc sudo insmod /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
sudo /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1/usertools/dpdk-devbind.py --bind igb_uio 0000:00:06.0
sudo /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1/usertools/dpdk-devbind.py --bind igb_uio 0000:00:07.0
Then you need necessarly to restart VPP, otherwise the NICs will not appear in the vppctl.
sudo service vpp stop sudo service vpp start
Be careful: if you stop your VM with VPP you will need to attach again the two NICs to VPP. To see if worked you can either go or to the DPDK script in the usertools folder or you just start VPP:
sudo vppctl vpp# show int
Set VPP as Gateway and make two VMs ping each other
In this step we will set VPP as default Gateway. This means that if we create a new instance, it will be able to reach Internet through VPP. Before starting to do this, we need to define our environment:
In our VPC we will have two instances: one with VPP and the other one can be a normal VM. To make all things work properly you need to create 3 subnets ( 1 is already created automatically) inside your VPC: one associated with IPv4 addresses where there is only one VM's NIC, so you can reach your VM through SSH (i.e you can use the default one for this purpouse). The second one, also with IPV4 addresses, where instead there are two NICs, VPP's one and VM's one: Therefore VPP and the VM can communicate each other ( i.e through pinging). Finally you need a third one, with IPv6 address, to connect the other VPP's NIC with the IGW and we will use Ipv6 because we will implement Segment Routing in that NIC. So a very rough draw of our setting would be: IGW ------subnet_IPv6-----VPP-------subnet_IPv4------VM------Subnet IPv4-----SSH ( from your terminal)
Notice that the following example works with two VPCs, where in each of them there are a VM with VPP and a VM. So you will have to execute the same commands also in the other VPC to make the connection between the two VPC possibile.
Now, create a new VM instance (you can use same setting as before (Ubuntu Server 16.04 and m5 type) and attach a NIC. Remember that the two VM's NICs should stay in two different IPv4 Subnet. Afterwards, on the VM's terminal execute these commands:
sudo /sbin/ip -4 addr add 10.1.2.113/24 dev ens6 sudo ifconfig ens6 up sudo /sbin/ip -4 route add 10.2.0.0/16 via 10.1.4.117
Basically you are putting up the interface which you will use to reach VPP and telling that all the traffic belonging to the subnet 10.2.0.0/16, which in our case is the one of the other VPC, should go through VPP's interface. Remember also to do the same thing in the route table menu of Amazon Console
Now go to the terminal of VPP, enter in the vppctl:
sudo vppctl
and execute these commands:
set int state VirtualFunctionEthernet0/6/0 up set int state VirtualFunctionEthernet0/7/0 up
Here you are putting up the two virtual interfaces.
set int ip address VirtualFunctionEthernet0/6/0 10.1.4.117/24 set int ip address VirtualFunctionEthernet0/7/0 2600:1f14:e0e:7f00:f672:1039:4e41:e68/64
Here you are assigning the IP addresses.
set sr encaps source addr 2600:1f14:e0e:7f00:f672:1039:4e41:e68 sr localsid address 2600:1f14:e0e:7f00:8da1:c8fa:5301:1d1f behavior end.dx4 VirtualFunctionEthernet0/6/0 10.1.4.117 sr policy add bsid c:1::999:1 next 2600:1f14:135:cc00:43c1:e860:7ce9:e94a encap sr steer l3 10.2.5.0/24 via bsid c:1::999:1
Here instead you are using Segment Routings's functionalities. Note that for the localsid address we are using a different IPv6 address ( you can generate it through the Amazon console)
set ip6 neighbor VirtualFunctionEthernet0/7/0 fe80::84f:3fff:fe2a:aaf0 0a:4f:3f:2a:aa:f0 ip route add ::/0 via fe80::84f:3fff:fe2a:aaf0 VirtualFunctionEthernet0/7/0
Finally you are setting the ip6 discovery, telling which is the next hop ( the IGW). Notice that the MAC address is the MAC address of the IGW.
Now go in the other instances in the other VPC and do the same commands. First in the VM:
sudo /sbin/ip -4 addr add 10.2.5.190/24 dev ens6 sudo ifconfig ens6 up sudo /sbin/ip -4 route add 10.2.0.0/16 via 10.2.5.21
Then, in VPP:
set int state VirtualFunctionEthernet0/6/0 up set int state VirtualFunctionEthernet0/7/0 up set int ip address VirtualFunctionEthernet0/6/0 10.2.5.21/24 set int ip address VirtualFunctionEthernet0/7/0 2600:1f14:135:cc00:13b9:ff74:348d:7642/64 set sr encaps source addr 2600:1f14:135:cc00:13b9:ff74:348d:7642 sr localsid address 2600:1f14:135:cc00:43c1:e860:7ce9:e94a behavior end.dx4 VirtualFunctionEthernet0/6/0 10.2.5.190 sr policy add bsid c:3::999:1 next 2600:1f14:e0e:7f00:8da1:c8fa:5301:1d1f encap sr steer l3 10.1.4.0/24 via sr policy bsid c:3::999:1 set ip6 neighbor VirtualFunctionEthernet0/7/0 fe80::86a:b7ff:fe5d:73c0 0a:4c:fd:b8:c1:3e ip route add ::/0 via fe80::86a:b7ff:fe5d:73c0 VirtualFunctionEthernet0/7/0
Now if you try a ping from a virtual machine you should be able to reach the other one in the other VPC.
Troubleshooting
- Remember to disable source/dest check on the VPP and VMs interface. You can do it through the Amazon Console.
- The commands work with VPP version 18.04. If you're using a different version, probably the syntax of some VPP commands would be slightly different.
- Be careful: if you stop your VM with VPP you will need to attach again the two NICs to VPP.