Difference between revisions of "VPP/Pure L3 Container Networking"

From fd.io
< VPP
Jump to: navigation, search
(→‎Setup)
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
This example shows how to configure VPP as an IPv4 router interconnecting containers across multiple hosts.
+
This example shows how to configure VPP as an IPv4 router interconnecting containers across two hosts.
  
 
VPP itself runs in the root namespace, with a separate namespace for each container.
 
VPP itself runs in the root namespace, with a separate namespace for each container.
  
 
[[File:Vpp-container.pdf|thumb|test network diagram]]
 
[[File:Vpp-container.pdf|thumb|test network diagram]]
 +
 +
The basic setup is to use static ARPs and to unnumber the host interfaces from a loopback.  From that perspective this is a hybrid of Methods 1 and 2 from [[Pure_L3_Between_Namespaces_with_/32s]]
  
 
= Network Setup =
 
= Network Setup =
  
−
The 2 hosts are interconnected by a router.  The router has /24 routes for the client subnets - pointed at the appropriate vSwitch interfaces.
+
The two hosts are interconnected by a router.  The router has /24 routes for the client subnets - pointed at the appropriate vSwitch interfaces.
−
[[File:container-vpp.pdf]]
+
 
 +
[[File:vpp-container.pdf]]
  
 
= Host Setup =
 
= Host Setup =
Line 14: Line 17:
 
On each host do:
 
On each host do:
  
−
<code>sudo docker create -e MICROSERVICE_LABEL=vpp -it --privileged -v "/tmp/vpp_socket:/tmp" -p 5001:5002 -p 9191:9191 --name vpp --network=host contivvpp/vswitch</code>
+
<code>
 +
sudo docker create -e MICROSERVICE_LABEL=vpp -it --privileged -v "/tmp/vpp_socket:/tmp" -p 5001:5002 -p 9191:9191 --name vpp --network=host contivvpp/vswitch
 +
</code>
 
   
 
   
 
create file vpp.conf as follows:
 
create file vpp.conf as follows:
  
 
<code>
 
<code>
−
unix {
+
unix {
−
:nodaemon
+
    nodaemon
−
:cli-listen 0.0.0.0:5002
+
    cli-listen 0.0.0.0:5002
−
:cli-no-pager
+
    cli-no-pager
−
}
+
}
−
dpdk {
+
dpdk {
−
:dev 0000:09:00.0 # replace this with an Ethernet interface on your host
+
    dev 0000:09:00.0 # replace this with an Ethernet interface on your host
−
:uio-driver igb_uio
+
    uio-driver igb_uio
−
}
+
}
 
</code>
 
</code>
  
 
then:
 
then:
 
<code>
 
<code>
−
<br />sudo docker cp vpp.conf vpp:/etc/vpp/vpp.conf
+
sudo docker cp vpp.conf vpp:/etc/vpp/vpp.conf
−
<br />sudo docker create -it --name client ubuntu  
+
sudo docker create -it --name client ubuntu  
 
</code>
 
</code>
  
Line 43: Line 48:
  
 
<code>
 
<code>
−
<br />sudo docker start vpp client
+
sudo docker start vpp client
−
<br />export pid="$(sudo docker inspect -f '{{.State.Pid}}' "client")"
+
export pid="$(sudo docker inspect -f '{{.State.Pid}}' "client")"
−
<br />sudo ln -sf /proc/$pid/ns/net /var/run/netns/client
+
sudo ln -sf /proc/$pid/ns/net /var/run/netns/client
−
<br />sudo ip link add name veth_client type veth peer name client
+
sudo ip link add name veth_client type veth peer name client
−
<br />sudo ip link set dev client up
+
sudo ip link set dev client up
−
<br />sudo ip link set dev veth_client up netns client
+
sudo ip link set dev veth_client up netns client
−
<br />export mac="$(sudo docker exec client ifconfig veth_client | awk 'NR==1{print $5}')"
+
export mac="$(sudo docker exec client ifconfig veth_client | awk 'NR==1{print $5}')"
−
<br />echo $mac
+
echo $mac
−
<br />export vmac="$(printf '
+
export vmac="$(printf '
−
<br />set int ip address GigabitEthernet1/0/0 192.168.101.1/24\n
+
set int ip address GigabitEthernet1/0/0 192.168.101.1/24\n
−
<br />set int state GigabitEthernet1/0/0 up\n  
+
set int state GigabitEthernet1/0/0 up\n  
−
<br />create loopback interface\n
+
create loopback interface\n
−
<br />set int ip address loop0 192.168.200.1/24\n
+
set int ip address loop0 192.168.200.1/24\n
−
<br />set int state loop0 up\n
+
set int state loop0 up\n
−
<br />create host-interface name client\n
+
create host-interface name client\n
−
<br />set int unnumbered host-client use loop0\n
+
set int unnumbered host-client use loop0\n
−
<br />set ip arp host-client 192.168.200.2 MAC\n
+
set ip arp host-client 192.168.200.2 MAC\n
−
<br />set int state host-client up\n
+
set int state host-client up\n
−
<br />ip route add 192.168.200.2/32 via 192.168.200.2 host-client\n
+
ip route add 192.168.200.2/32 via 192.168.200.2 host-client\n
−
<br />ip route add 192.168.0.0/16 via 192.168.101.254 GigabitEthernet1/0/0\n
+
ip route add 192.168.0.0/16 via 192.168.101.254 GigabitEthernet1/0/0\n
−
<br />show hardware-interfaces host-client\n
+
show hardware-interfaces host-client\n
−
<br />quit' | sed s/MAC/$mac/ | nc 0 5002 | awk 'NR==29{print $3}')"
+
quit' | sed s/MAC/$mac/ | nc 0 5002 | awk 'NR==29{print $3}')"
−
<br />echo $vmac
+
echo $vmac
−
<br />sudo ip netns exec client ip link set dev lo up
+
sudo ip netns exec client ip link set dev lo up
−
<br />sudo ip netns exec client ip addr add 192.168.200.2/32 dev veth_client
+
sudo ip netns exec client ip addr add 192.168.200.2/32 dev veth_client
−
<br />sudo ip netns exec client ip neigh add 192.168.200.1 lladdr $vmac dev veth_client
+
sudo ip netns exec client ip neigh add 192.168.200.1 lladdr $vmac dev veth_client
−
<br />sudo ip netns exec client ip route add 192.168.200.1 dev veth_client scope link
+
sudo ip netns exec client ip route add 192.168.200.1 dev veth_client scope link
−
<br />sudo ip netns exec client ip route add 192.168.0.0/16 via 192.168.200.1 dev veth_client
+
sudo ip netns exec client ip route add 192.168.0.0/16 via 192.168.200.1 dev veth_client
−
<br />sudo ip netns exec client ip route add 1.2.3.4/32 via 192.168.200.1 dev veth_client
+
sudo ip netns exec client ip route add 1.2.3.4/32 via 192.168.200.1 dev veth_client
−
<br />sudo docker exec client ping -c 1 192.168.200.1
+
sudo docker exec client ping -c 1 192.168.200.1
 
</code>
 
</code>
  
Line 82: Line 87:
  
 
<code>
 
<code>
−
<br />sudo docker start vpp server
+
sudo docker start vpp server
−
<br />export pid="$(sudo docker inspect -f '{{.State.Pid}}' "server")"
+
export pid="$(sudo docker inspect -f '{{.State.Pid}}' "server")"
−
<br />sudo ln -sf /proc/$pid/ns/net /var/run/netns/server
+
sudo ln -sf /proc/$pid/ns/net /var/run/netns/server
−
<br />sudo ip link add name veth_server type veth peer name server
+
sudo ip link add name veth_server type veth peer name server
−
<br />sudo ip link set dev server up
+
sudo ip link set dev server up
−
<br />sudo ip link set dev veth_server up netns server
+
sudo ip link set dev veth_server up netns server
−
<br />export mac="$(sudo docker exec server ifconfig veth_server | awk 'NR==1{print $5}')"
+
export mac="$(sudo docker exec server ifconfig veth_server | awk 'NR==1{print $5}')"
−
<br />echo $mac
+
echo $mac
−
<br />export vmac="$(printf '
+
export vmac="$(printf '
−
<br />set int ip address GigabitEthernet1/0/0 192.168.103.1/24\n
+
set int ip address GigabitEthernet1/0/0 192.168.103.1/24\n
−
<br />set int state GigabitEthernet1/0/0 up\n
+
set int state GigabitEthernet1/0/0 up\n
−
<br />create loopback interface\n
+
create loopback interface\n
−
<br />set int ip address loop0 192.168.204.1/24\n
+
set int ip address loop0 192.168.204.1/24\n
−
<br />set int state loop0 up\n
+
set int state loop0 up\n
−
<br />create host-interface name server\n
+
create host-interface name server\n
−
<br />set int unnumbered host-server use loop0\n
+
set int unnumbered host-server use loop0\n
−
<br />set ip arp host-server 192.168.204.2 MAC\n
+
set ip arp host-server 192.168.204.2 MAC\n
−
<br />set int state host-server up\n
+
set int state host-server up\n
−
<br />ip route add 192.168.204.2/32 via 192.168.204.2 host-server\n
+
ip route add 192.168.204.2/32 via 192.168.204.2 host-server\n
−
<br />ip route add 192.168.0.0/16 via 192.168.103.254 GigabitEthernet1/0/0\n
+
ip route add 192.168.0.0/16 via 192.168.103.254 GigabitEthernet1/0/0\n
−
<br />show hardware-interfaces host-server\n
+
show hardware-interfaces host-server\n
−
<br />quit' | sed s/MAC/$mac/ | nc 0 5002 | awk 'NR==29{print $3}')"
+
quit' | sed s/MAC/$mac/ | nc 0 5002 | awk 'NR==29{print $3}')"
−
<br />echo $vmac
+
echo $vmac
−
<br />sudo ip netns exec server ip link set dev lo up
+
sudo ip netns exec server ip link set dev lo up
−
<br />sudo ip netns exec server ip addr add 192.168.204.2/32 dev veth_server
+
sudo ip netns exec server ip addr add 192.168.204.2/32 dev veth_server
−
<br />sudo ip netns exec server ip neigh add 192.168.204.1 lladdr $vmac dev veth_server
+
sudo ip netns exec server ip neigh add 192.168.204.1 lladdr $vmac dev veth_server
−
<br />sudo ip netns exec server ip route add 192.168.204.1 dev veth_server scope link
+
sudo ip netns exec server ip route add 192.168.204.1 dev veth_server scope link
−
<br />sudo ip netns exec server ip route add 192.168.0.0/16 via 192.168.204.1 dev veth_server
+
sudo ip netns exec server ip route add 192.168.0.0/16 via 192.168.204.1 dev veth_server
−
<br />sudo docker exec server ping -c 1 192.168.204.1
+
sudo docker exec server ping -c 1 192.168.204.1
 
</code>
 
</code>
  
 
again the ping should succeed
 
again the ping should succeed

Latest revision as of 15:51, 25 October 2017

This example shows how to configure VPP as an IPv4 router interconnecting containers across two hosts.

VPP itself runs in the root namespace, with a separate namespace for each container.

File:Vpp-container.pdf

The basic setup is to use static ARPs and to unnumber the host interfaces from a loopback. From that perspective this is a hybrid of Methods 1 and 2 from Pure_L3_Between_Namespaces_with_/32s

Network Setup

The two hosts are interconnected by a router. The router has /24 routes for the client subnets - pointed at the appropriate vSwitch interfaces.

File:Vpp-container.pdf

Host Setup

On each host do: 

sudo docker create -e MICROSERVICE_LABEL=vpp -it --privileged -v "/tmp/vpp_socket:/tmp" -p 5001:5002 -p 9191:9191 --name vpp --network=host contivvpp/vswitch

create file vpp.conf as follows: 

unix {
    nodaemon
    cli-listen 0.0.0.0:5002
    cli-no-pager
}
dpdk {
    dev 0000:09:00.0 # replace this with an Ethernet interface on your host
    uio-driver igb_uio
}

then: 

sudo docker cp vpp.conf vpp:/etc/vpp/vpp.conf
sudo docker create -it --name client ubuntu

(for the server host change the name to "server")

Client-Side Setup

from the Linux command line: 

sudo docker start vpp client
export pid="$(sudo docker inspect -f 'Template:.State.Pid' "client")"
sudo ln -sf /proc/$pid/ns/net /var/run/netns/client
sudo ip link add name veth_client type veth peer name client
sudo ip link set dev client up
sudo ip link set dev veth_client up netns client
export mac="$(sudo docker exec client ifconfig veth_client | awk 'NR==1{print $5}')"
echo $mac
export vmac="$(printf '
set int ip address GigabitEthernet1/0/0 192.168.101.1/24\n
set int state GigabitEthernet1/0/0 up\n 
create loopback interface\n
set int ip address loop0 192.168.200.1/24\n
set int state loop0 up\n
create host-interface name client\n
set int unnumbered host-client use loop0\n
set ip arp host-client 192.168.200.2 MAC\n
set int state host-client up\n
ip route add 192.168.200.2/32 via 192.168.200.2 host-client\n
ip route add 192.168.0.0/16 via 192.168.101.254 GigabitEthernet1/0/0\n
show hardware-interfaces host-client\n
quit' | sed s/MAC/$mac/ | nc 0 5002 | awk 'NR==29{print $3}')"
echo $vmac
sudo ip netns exec client ip link set dev lo up
sudo ip netns exec client ip addr add 192.168.200.2/32 dev veth_client
sudo ip netns exec client ip neigh add 192.168.200.1 lladdr $vmac dev veth_client
sudo ip netns exec client ip route add 192.168.200.1 dev veth_client scope link
sudo ip netns exec client ip route add 192.168.0.0/16 via 192.168.200.1 dev veth_client
sudo ip netns exec client ip route add 1.2.3.4/32 via 192.168.200.1 dev veth_client
sudo docker exec client ping -c 1 192.168.200.1

the ping should succeed

Server-Side Setup

Again from the Linux command line: 

sudo docker start vpp server
export pid="$(sudo docker inspect -f 'Template:.State.Pid' "server")"
sudo ln -sf /proc/$pid/ns/net /var/run/netns/server
sudo ip link add name veth_server type veth peer name server
sudo ip link set dev server up
sudo ip link set dev veth_server up netns server
export mac="$(sudo docker exec server ifconfig veth_server | awk 'NR==1{print $5}')"
echo $mac
export vmac="$(printf '
set int ip address GigabitEthernet1/0/0 192.168.103.1/24\n
set int state GigabitEthernet1/0/0 up\n
create loopback interface\n
set int ip address loop0 192.168.204.1/24\n
set int state loop0 up\n
create host-interface name server\n
set int unnumbered host-server use loop0\n
set ip arp host-server 192.168.204.2 MAC\n
set int state host-server up\n
ip route add 192.168.204.2/32 via 192.168.204.2 host-server\n
ip route add 192.168.0.0/16 via 192.168.103.254 GigabitEthernet1/0/0\n
show hardware-interfaces host-server\n
quit' | sed s/MAC/$mac/ | nc 0 5002 | awk 'NR==29{print $3}')"
echo $vmac
sudo ip netns exec server ip link set dev lo up
sudo ip netns exec server ip addr add 192.168.204.2/32 dev veth_server
sudo ip netns exec server ip neigh add 192.168.204.1 lladdr $vmac dev veth_server
sudo ip netns exec server ip route add 192.168.204.1 dev veth_server scope link
sudo ip netns exec server ip route add 192.168.0.0/16 via 192.168.204.1 dev veth_server
sudo docker exec server ping -c 1 192.168.204.1

again the ping should succeed

Retrieved from "https://wiki.fd.io/index.php?title=VPP/Pure_L3_Container_Networking&oldid=6065"