How to deploy VPP in EC2 instance and use it to connect two different VPCs

In this page it is shown how to deploy VPP (18.04 version) in an Amazon instance EC2, set VPP as Gateway towards Internet and connect two VMs belonging in two different VPCs through VPP and SRv6. Remember that to perform all of these operations you need to have an Amazon AWS account. Furthermore, it would be useful to have a prior knowledge of VPCs' concepts and Segment Routing.

VPP's deployment

First of all, go to the Amazon Management console and create a new instance. If you have not already done it, before creating the instance you need to create a VPC,in which you will put the instance, with its own security group and subnet (for the first one use only in the inbound and outbound rules the "All Traffic" option for IPv4 and IPv6). As Image use Ubuntu server 16.04 and as instance type one of the m5 type. Afterwards, create two new NICs always through the Amazon Console and attach them to the instance ( in total your VM should have 3 virtual NIC, one used to reach it through SSH and the other two for VPP. Before attach them to the VM, select them, go to the "actions" button and change the "source/dest check" option from enable to disable.

Then, access your VM via SSH and execute these commands:

Introduction

To make everything work you need to install some packages:

cd /
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential
sudo apt-get install python-pip
sudo apt-get install libnuma-dev
sudo apt-get install make

Then you need to install DPDK:

sudo wget https://fast.dpdk.org/rel/dpdk-18.02.1.tar.xz
sudo tar -xvf dpdk-18.02.1.tar.xz 

Installation of VPP

Now you can install VPP:

cd /
sudo mkdir /vpp 
sudo chmod 777 /vpp
cd /vpp
git clone https://gerrit.fd.io/r/vpp ./ 
sudo make install-dep
sudo make build 
cd build-root 
make V=0 PLATFORM=vpp TAG=vpp install-deb

With the latter command you will install the last version of VPP. However for our purposes we need VPP 18.04 version:

export UBUNTU="xenial"
export RELEASE=".stable.1804"
sudo rm /etc/apt/sources.list.d/99fd.io.list
echo "deb [trusted=yes] https://nexus.fd.io/content/repositories/fd.io$RELEASE.ubuntu.$UBUNTU.main/ ./" | 
sudo tee -a /etc/apt/sources.list.d/99fd.io.list
sudo apt-get update
sudo apt-get install vpp vpp-lib
sudo apt-get install vpp-plugins vpp-dbg vpp-dev vpp-api-java vpp-api-python vpp-api- 
lua

Now you need to attach your two virtual NIC to VPP, in order to make all the flow of packets pass through VPP.

cd /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1
sudo modprobe uio 
sudo make install T=x86_64-native-linuxapp-gcc
sudo insmod /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko

sudo /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1/usertools/dpdk-devbind.py --bind igb_uio 0000:00:06.0

sudo /vpp/build-root/build-vpp_debug-native/dpdk/dpdk-stable-18.02.1/usertools/dpdk-devbind.py --bind igb_uio 0000:00:07.0

Then you need necessarly to restart VPP, otherwise the NICs will not appear in the vppctl.

sudo service vpp stop
sudo service vpp start

Be careful: if you stop your VM with VPP you will need to attach again the two NICs to VPP. To see if worked you can either go or to the DPDK script in the usertools folder or you just start VPP:

sudo vppctl
vpp#  show int  

Set VPP as Gateway and make two VMs ping each other

In this step we will set VPP as default Gateway. This means that if we create a new instance, it will be able to reach Internet through VPP. Before starting to do this, we need to define our environment:

In our VPC we will have two instances: one with VPP and the other one can be a normal VM. To make all things work properly you need to create 3 subnets ( 1 is already created automatically) inside your VPC: one associated with IPv4 addresses where there is only one VM's NIC, so you can reach your VM through SSH (i.e you can use the default one for this purpouse). The second one, also with IPV4 addresses, where instead there are two NICs, VPP's one and VM's one: Therefore VPP and the VM can communicate each other ( i.e through pinging). Finally you need a third one, with IPv6 address, to connect the other VPP's NIC with the IGW and we will use Ipv6 because we will implement Segment Routing in that NIC. So a very rough draw of our setting would be: IGW ------subnet_IPv6-----VPP-------subnet_IPv4------VM------Subnet IPv4-----SSH ( from your terminal)

Notice that the following example works with two VPCs, where in each of them there are a VM with VPP and a VM. So you will have to execute the same commands also in the other VPC to make the connection between the two VPC possibile.

Now, create a new VM instance (you can use same setting as before (Ubuntu Server 16.04 and m5 type) and attach a NIC. Remember that the two VM's NICs should stay in two different IPv4 Subnet. Afterwards, on the VM's terminal execute these commands:

sudo /sbin/ip -4 addr add 10.1.2.113/24 dev ens6
sudo ifconfig ens6 up
sudo /sbin/ip -4 route add 10.2.0.0/16 via 10.1.4.117

Basically you are putting up the interface which you will use to reach VPP and telling that all the traffic belonging to the subnet 10.2.0.0/16, which in our case is the one of the other VPC, should go through VPP's interface. Remember also to do the same thing in the route table menu of Amazon Console

Now go to the terminal of VPP, enter in the vppctl:

sudo vppctl 

and execute these commands:

set int state VirtualFunctionEthernet0/6/0 up
set int state VirtualFunctionEthernet0/7/0 up

Here you are putting up the two virtual interfaces.

set int ip address VirtualFunctionEthernet0/6/0 10.1.4.117/24
set int ip address VirtualFunctionEthernet0/7/0 2600:1f14:e0e:7f00:f672:1039:4e41:e68/64

Here you are assigning the IP addresses.

set sr encaps source addr 2600:1f14:e0e:7f00:f672:1039:4e41:e68
sr localsid address 2600:1f14:e0e:7f00:8da1:c8fa:5301:1d1f behavior end.dx4 VirtualFunctionEthernet0/6/0 10.1.4.117
sr policy add bsid c:1::999:1  next 2600:1f14:135:cc00:43c1:e860:7ce9:e94a encap
sr steer l3 10.2.5.0/24 via  bsid c:1::999:1

Here instead you are using Segment Routings's functionalities. Note that for the localsid address we are using a different IPv6 address ( you can generate it through the Amazon console)

set ip6 neighbor VirtualFunctionEthernet0/7/0 fe80::84f:3fff:fe2a:aaf0 0a:4f:3f:2a:aa:f0 
ip route add ::/0 via fe80::84f:3fff:fe2a:aaf0 VirtualFunctionEthernet0/7/0

Finally you are setting the ip6 discovery, telling which is the next hop ( the IGW). Notice that the MAC address is the MAC address of the IGW.

Now go in the other instances in the other VPC and do the same commands. First in the VM:

sudo /sbin/ip -4 addr add 10.2.5.190/24 dev ens6
sudo ifconfig ens6 up
sudo /sbin/ip -4 route add 10.2.0.0/16 via 10.2.5.21

Then, in VPP:

set int state VirtualFunctionEthernet0/6/0 up
set int state VirtualFunctionEthernet0/7/0 up
set int ip address VirtualFunctionEthernet0/6/0 10.2.5.21/24
set int ip address VirtualFunctionEthernet0/7/0 2600:1f14:135:cc00:13b9:ff74:348d:7642/64
set sr encaps source addr 2600:1f14:135:cc00:13b9:ff74:348d:7642
sr localsid address 2600:1f14:135:cc00:43c1:e860:7ce9:e94a behavior end.dx4 VirtualFunctionEthernet0/6/0 10.2.5.190
sr policy add bsid c:3::999:1  next 2600:1f14:e0e:7f00:8da1:c8fa:5301:1d1f encap
sr steer l3 10.1.4.0/24 via sr policy bsid c:3::999:1
set ip6 neighbor VirtualFunctionEthernet0/7/0 fe80::86a:b7ff:fe5d:73c0 0a:4c:fd:b8:c1:3e
ip route add ::/0 via fe80::86a:b7ff:fe5d:73c0 VirtualFunctionEthernet0/7/0

Now if you try a ping from a virtual machine you should be able to reach the other one in the other VPC.

Troubleshooting

• Remember to disable source/dest check on the VPP and VMs interface. You can do it through the Amazon Console.
• The commands work with VPP version 18.04. If you're using a different version, probably the syntax of some VPP commands would be slightly different.
• Be careful: if you stop your VM with VPP you will need to attach again the two NICs to VPP.