Dumbbell-vicn
Contents
Setup a Dumbbell topology using vICN
This example shows how to create and set up a typical Dumbbell topology using vICN:
+-----+ +-----+ |cons1|---------+ +---------|prod1| +-----+ | | +-----+ +-----+ | | +-----+ |cons2|-------+ | | +-------|prod2| +-----+ | | | | +-----+ +-----+ +--+--+ +--+--+ +-----+ |cons3|------|core1|----------|core2|------|prod3| +-----+ +--+--+ +--+--+ +-----+ +-----+ | | | | +-----+ |cons4|-------+ | | +-------|prod4| +-----+ | | +-----+ +-----+ | | +-----+ |cons5|---------+ +---------|prod5| +-----+ +-----+
- Each node is deployed as an LXC container
- cons1, cons2, cons3, cons4, cons5 run an instance of Metis
- core1 and core2 run the CICN plugin for VPP
- prod1, prod2, prod3, prod4, prod5 run an instance of Metis
Deploying the topology
Requirements
- vICN (Install instructions are here)
- 6 DPDK compatible interfaces
- LXC image with the full CICN suite
How to
To setup the topology:
First (if you have not already done it), install the LXC CICN image
$ wget https://cisco.box.com/shared/static/2vui2mtmk1mp4strcs96w27t467bkady.gz -O ubuntu1604-cicnsuite-rc1.tar.gz --delete-after $ lxc image import ubuntu1604-cicnsuite-rc1.tar.gz ubuntu1604-cicnsuite-rc1
Then download the topology file
From the code tree root $ wget https://cisco.box.com/shared/static/n9qmlpdcfcwwflkmehsm0ll3lkj2ozfg.json -O examples/tutorial/dumbell.json
Update the mac address of the DPDK interfaces in the dumbell.json. The mac address must be the actual mac address of the DPDK interfaces in the server.
You can now run the topology:
$ ./vicn/bin/vicn.py -s examples/tutorial/dumbell.json
Understanding the dumbbell.json file
The actual topology deployed from vICN is slightly different from the one reported above. In particular, it contains two additional bridges that connects consumers and producers to the core nodes. This is because the CICN plugin only supports connectivity through DPDK interfaces.
Using DPDK interfaces to connect all the nodes would require quite a few DPDK interfaces, although a possible approach. A more convenient way for scaling up with the number of nodes is:
- use virtual ethernet interfaces (veths) to connect nodes running Metis
- use DPDK interfaces for the node running the CICN high speed forwarder
- deploy a bridge that connects nodes that use veths with nodes that use DPDK interfaces
As result, the actual Dumbbell topology is the following:
+-----+ +-----+ |cons1|---------+ +---------|prod1| +-----+ | | +-----+ +-----+ | | +-----+ |cons2|-------+ | | +-------|prod2| +-----+ | | | | +-----+ +-----+ +---+---+ +--+--+ +--+--+ +---+---+ +-----+ |cons3|------|bridge1|---|core1|------|core2|---|bridge2|------|prod3| +-----+ +---+---+ +--+--+ +--+--+ +---+---+ +-----+ +-----+ | | | | +-----+ |cons4|-------+ | | +-------|prod4| +-----+ | | +-----+ +-----+ | | +-----+ |cons5|---------+ +---------|prod5| +-----+ +-----+
This topology assumes that the following pair of resources are connected together with a physical connection between two DPDK interfaces:
- bridge1 and core1
- bridge2 and core2
- core1 and core2
Most of the resources reported in the dumbbell.json
file are already explained here. In the following, we walk through the vICN resources that are required to set up the two bridges, the two core nodes and the connectivity among the resources. A detailed explanation of the attributes of each resource is given at the end of the tutorial.
Cores
Each of the two cores, core1
and core2
, is composed of the following resources:
- One
LxcContainer
- The LXC container used to emulate the node.
- Two
DpdkInterface
- These resources describe the two DPDK interfaces that connect the core node to the bridge and to the other core node.
- One
VPP
- This resource describes a VPP forwarder.
- Two
VPPInterface
- These resources, one per DPDK interface, instruct the VPP forwarder about the available DPDK interfaces, and how to set them up properly.
- One
CICNForwarder
- This resource describes the CICN plugin for the VPP forwarder.
The following code shows the list of vICN resources to deploy and set up core1
.
{ "type": "LxcContainer", "node": "server", "name": "core1", "image": "ubuntu1604-cicnsuite-rc1" }, { "type": "VPP", "node": "core1", "name": "core1-vpp" }, { "type": "DpdkDevice", "node": "core1", "device_name": "GigabitEthernet0/9/0", "pci_address": "0000:00:09.0", "ip_address" : "172.17.1.21", "mac_address": "08:00:27:d1:b5:d1", "name": "core1-dpdk1" }, { "type": "VPPInterface", "name": "core1-vppdpdk1", "vpp": "core1-vpp", "node": "core1", "ip_address": "172.17.1.21", "parent": "core1-dpdk1" }, { "type": "DpdkDevice", "node": "core1", "device_name": "GigabitEthernet0/a/0", "pci_address": "0000:00:0a.0", "ip_address" : "172.17.2.21", "mac_address": "08:00:27:d1:b5:c1", "name": "core1-dpdk2" }, { "type": "VPPInterface", "name": "core1-vppdpdk2", "vpp": "core1-vpp", "node": "core1", "ip_address": "172.17.2.21", "parent": "core1-dpdk2" }, { "type": "CICNForwarder", "node": "core1", "name": "core1-fwd" }
Bridges
Each of the two bridges, bridge1
and bridge2
, is made of the following resources:
- One
LxcContainer
- The LXC container used to emulate the node.
- One
DpdkInterface
- The DPDK interface that connects the bridge to the core node.
- One
VPP
- The VPP forwarder that will be run in bridge mode.
- One
VPPBridge
- The bridge that connects the list of consumers with the core node.
The following code shows the list of vICN resources to deploy and set up bridge1
.
{ "type": "LxcContainer", "node": "server", "name": "bridge1", "image": "ubuntu1604-cicnsuite-rc1" }, { "type": "VPP", "node": "bridge1", "name": "bridge1-vpp1" }, { "type": "DpdkDevice", "node": "bridge1", "device_name": "GigabitEthernet0/8/0", "pci_address" : "0000:00:08.0", "ip_address" : "172.17.1.20", "mac_address": "08:00:27:b8:f3:a3", "name": "bridge1-dpdk1" }, { "type": "VPPBridge", "connected_nodes": ["cons1","cons2","cons3","cons4","cons5"], "interfaces": ["core1-dpdk1"], "node": "bridge1" },
Connectivity
To connect the two cores together, it is required to use a PhyLink
resource. Such resource describes the physical link between the two DPDK interfaces belonging to core1
and core2
.
{ "type": "PhyLink", "src": "core1-dpdk2", "dst": "core2-dpdk1" }
Attributes description
-
LxcContainer
- Details for such resource can be found [here]
-
DpdkInterface
- Attributes:
-
node
: the node that controls the DPDK interface. -
device_name
: the name of the DPDK device given by VPP. -
pci_address
: the PCI address of the interface (can be retrieved vialspci
) -
ip_address
: the IP address to assign to the interface. Currently vICN does not support automatic ip assignment for this type of interfaces. -
mac_address
: the mac address assigned to the DPDK device. -
name
: the name of the resource.
-
- Attributes:
-
VPP
- Attributes:
-
node
: the node on which VPP will be installed and run. -
name
: the name of the resource.
-
- Attributes:
-
VPPInterface
- Attributes:
-
name
: the name of the resource. -
vpp
: the VPP instance to which the VPPInterface belong to. -
parent
: the DPDK interface on top of which the VPP Interface is created.
-
- Attributes:
-
CICNForwarder
- Attributes:
-
node
: the node on which the VPP forwarder is installed and run. -
name
: the name of the resource.
-
- Attributes:
-
VPPBridge
- Attributes:
-
connected_nodes
: the list of nodes to connect to the bridge. These nodes will be connected through virtual ethernet devices. To connect a node running VPP on the bridge it is required to include its DPDK interface in theinterfaces
attribute. -
interfaces
: the list of DPDK interfaces belonging to the nodes to connect to the bridge. -
node
: the node on which the VPP forwarder is installed and run.
-
- Attributes:
-
PhyLink
- Attributes:
-
src
: the DPDK interface belonging to the node at one of the two side of the link. -
dst
: the DPDK interface belonging to the node at other side of the link.
-
- Attributes: