Revision as of 13:15, 28 February 2017

Setup a Dumbbell topology using vICN

This example shows how to create and set up a typical Dumbbell topology using vICN:

+-----+                                    +-----+
|cons1|---------+                +---------|prod1|
+-----+         |                |         +-----+
+-----+         |                |         +-----+
|cons2|-------+ |                | +-------|prod2|
+-----+       | |                | |       +-----+
+-----+      +--+--+          +--+--+      +-----+
|cons3|------|core1|----------|core2|------|prod3|
+-----+      +--+--+          +--+--+      +-----+
+-----+       | |                | |       +-----+
|cons4|-------+ |                | +-------|prod4|
+-----+         |                |         +-----+
+-----+         |                |         +-----+
|cons5|---------+                +---------|prod5|
+-----+                                    +-----+

• Each node is deployed as an LXC container
• cons1, cons2, cons3, cons4, cons5 run an instance of Metis
• core1 and core2 run the CICN plugin for VPP
• prod1, prod2, prod3, prod4, prod5 run an instance of Metis

Deploying the topology

Requirements

• vICN (Install instructions are here)
• 6 DPDK compatible interfaces
• LXC image with the full CICN suite

How to

To setup the topology:

First (if you have not already done it), install the LXC CICN image

$ wget https://cisco.box.com/shared/static/o9idhgfcsrwawxp7rtkafjlurhhqblit.gz -O ubuntu1604-cicnsuite-rc1.tar.gz --delete-after
$ lxc image import ubuntu1604-cicnsuite-rc1.tar.gz ubuntu1604-cicnsuite-rc1

Then download the topology file

From the code tree root
$ wget https://cisco.box.com/shared/static/n9qmlpdcfcwwflkmehsm0ll3lkj2ozfg.json -O examples/tutorial/dumbell.json

Update the mac address of the DPDK interfaces in the dumbell.json. The mac address must be the actual mac address of the DPDK interfaces in the server.

You can now run the topology:

$ ./vicn/bin/vicn.py -s examples/tutorial/dumbell.json

Understanding the dumbbell.json file

The actual topology deployed from vICN is slightly different from the one reported above. In particular, it contains two additional bridges that connects consumers and producers to the core nodes. This is because the CICN plugin only supports connectivity through DPDK interfaces.

Using DPDK interfaces to connect all the nodes would require quite a few DPDK interfaces, although a possible approach. A more convenient way for scaling up with the number of nodes is:

• use virtual ethernet interfaces (veths) to connect nodes running Metis
• use DPDK interfaces for the node running the CICN high speed forwarder
• deploy a bridge that connects nodes that use veths with nodes that use DPDK interfaces

As result, the actual Dumbbell topology is the following:

+-----+                                                        +-----+
|cons1|---------+                                    +---------|prod1|
+-----+         |                                    |         +-----+
+-----+         |                                    |         +-----+
|cons2|-------+ |                                    | +-------|prod2|
+-----+       | |                                    | |       +-----+
+-----+      +---+---+   +--+--+      +--+--+   +---+---+      +-----+
|cons3|------|bridge1|---|core1|------|core2|---|bridge2|------|prod3|
+-----+      +---+---+   +--+--+      +--+--+   +---+---+      +-----+
+-----+       | |                                    | |       +-----+
|cons4|-------+ |                                    | +-------|prod4|
+-----+         |                                    |         +-----+
+-----+         |                                    |         +-----+
|cons5|---------+                                    +---------|prod5|
+-----+                                                        +-----+

This topology assumes that the following pair of resources are connected together with a physical connection between two DPDK interfaces:

• bridge1 and core1
• bridge2 and core2
• core1 and core2

Most of the resources reported in the dumbbell.json file are already explained here. In the following, we walk through the vICN resources that are required to set up the two bridges, the two core nodes and the connectivity among the resources. A detailed explanation of the attributes of each resource is given at the end of the tutorial.

Cores

Each of the two cores, core1 and core2, is composed of the following resources:

• One LxcContainer
  • The LXC container used to emulate the node.
• Two DpdkInterface
  • These resources describe the two DPDK interfaces that connect the core node to the bridge and to the other core node.
• One VPP
  • This resource describes a VPP forwarder.
• Two VPPInterface
• These resources, one per DPDK interface, instruct the VPP forwarder about the available DPDK interfaces, and how to set them up properly.
• One CICNForwarder
  • This resource describes the CICN plugin for the VPP forwarder.

The following code shows the list of vICN resources to deploy and set up core1.

 {
   "type": "LxcContainer",
   "node": "server",
   "name": "core1",
   "image": "ubuntu1604-cicnsuite-rc1"
 },
 {
   "type": "VPP",
   "node": "core1",
   "name": "core1-vpp"
 },
 {
   "type": "DpdkDevice",
   "node": "core1",
   "device_name": "GigabitEthernet0/9/0",
   "pci_address": "0000:00:09.0",
   "ip_address" : "172.17.1.21",
   "mac_address": "08:00:27:d1:b5:d1",
   "name": "core1-dpdk1"
 },
 {
   "type": "VPPInterface",
   "name": "core1-vppdpdk1",
   "vpp": "core1-vpp",  
   "node": "core1",
   "ip_address": "172.17.1.21",
   "parent": "core1-dpdk1"
 },
 {
   "type": "DpdkDevice",
   "node": "core1",
   "device_name": "GigabitEthernet0/a/0",
   "pci_address": "0000:00:0a.0",
   "ip_address" : "172.17.2.21",
   "mac_address": "08:00:27:d1:b5:c1",
   "name": "core1-dpdk2"
 },
 {
   "type": "VPPInterface",
   "name": "core1-vppdpdk2",
   "vpp": "core1-vpp",  
   "node": "core1",
   "ip_address": "172.17.2.21",
   "parent": "core1-dpdk2"
 },
 {
   "type": "CICNForwarder",
   "node": "core1",
   "name": "core1-fwd"
 }

Bridges

Each of the two bridges, bridge1 and bridge2, is made of the following resources:

• One LxcContainer
  • The LXC container used to emulate the node.
• One DpdkInterface
  • The DPDK interface that connects the bridge to the core node.
• One VPP
  • The VPP forwarder that will be run in bridge mode.
• One VPPBridge
  • The bridge that connects the list of consumers with the core node.

The following code shows the list of vICN resources to deploy and set up bridge1.

 {
   "type": "LxcContainer",
   "node": "server",
   "name": "bridge1",
   "image": "ubuntu1604-cicnsuite-rc1"
 },
 {
   "type": "VPP",
   "node": "bridge1",
   "name": "bridge1-vpp1"
 },
 {
   "type": "DpdkDevice",
   "node": "bridge1",
   "device_name": "GigabitEthernet0/8/0",
   "pci_address" : "0000:00:08.0",
   "ip_address" : "172.17.1.20",
   "mac_address": "08:00:27:b8:f3:a3",
   "name": "bridge1-dpdk1"
 },
 {
   "type": "VPPBridge",
   "connected_nodes": ["cons1","cons2","cons3","cons4","cons5"],
   "interfaces": ["core1-dpdk1"],
   "node": "bridge1"
  },

Connectivity

To connect the two cores together, it is required to use a PhyLink resource. Such resource describes the physical link between the two DPDK interfaces belonging to core1 and core2.

 {
   "type": "PhyLink",
   "src": "core1-dpdk2",
   "dst": "core2-dpdk1"
 }

Attributes description

• LxcContainer
  • Details for such resource can be found [here]
• DpdkInterface
  • Attributes:
    • node : the node that controls the DPDK interface.
    • device_name : the name of the DPDK device given by VPP.
    • pci_address : the PCI address of the interface (can be retrieved via lspci)
    • ip_address : the IP address to assign to the interface. Currently vICN does not support automatic ip assignment for this type of interfaces.
    • mac_address : the mac address assigned to the DPDK device.
    • name : the name of the resource.
• VPP
  • Attributes:
    • node : the node on which VPP will be installed and run.
    • name : the name of the resource.
• VPPInterface
  • Attributes:
    • name : the name of the resource.
    • vpp : the VPP instance to which the VPPInterface belong to.
    • parent : the DPDK interface on top of which the VPP Interface is created.
• CICNForwarder
  • Attributes:
    • node : the node on which the VPP forwarder is installed and run.
    • name : the name of the resource.
• VPPBridge
  • Attributes:
    • connected_nodes: the list of nodes to connect to the bridge. These nodes will be connected through virtual ethernet devices. To connect a node running VPP on the bridge it is required to include its DPDK interface in the interfaces attribute.
    • interfaces: the list of DPDK interfaces belonging to the nodes to connect to the bridge.
    • node: the node on which the VPP forwarder is installed and run.
• PhyLink
  • Attributes:
    • src: the DPDK interface belonging to the node at one of the two side of the link.
    • dst: the DPDK interface belonging to the node at other side of the link.