Revision as of 14:53, 20 April 2017

Introduction

Topology for the tutorial04-caching.json file

In this tutorial, we will explore multiple characteristics of ICN using various tools of the CICN suite. We will deploy a topology containing a core network linking three local networks (e.g., 3 universities on the Internet2 network). In the studied scenario, two servers in university 3 are producing data that researchers at university 1 and 2 need to access. The links in university 3 are slightly constrained (100Mbps) compared to all the others (200Mbps), thus the need to optimize traffic patterns on the network.

We will use the following tools:

• vICN
• Metis, a socket-based ICN forwarder
• http-server and iget
• The producer-test and consumer-test commands, which are part of the libicnet

You should have been given access to a preconfigured Linux instance. Make sure that you have root access:

$ sudo -v

During this tutorial, we will use the Linux screen command. It is used to have several bash sessions on the same tty (in our case, on the same SSH connection). You can learn more about screen by reading its manual

$ man screen

vICN bootstrap

First, we will use vICN to start a topology. To do so, we will open a new screen called "vicn" and run our topology in it:

$ screen -S vicn
$ cd ~/vicn
$ sudo vicn/bin/vicn.py -s examples/tutorial/tutorial04-caching.json

You will see a lot of debugging appearing on the console, which describes what vICN is currently doing. In this tutorial, we will not get into the meaning of this logs but you are welcome to study it on your own to understanding everything that vICN does. You can detect that vICN has performed all his tasks when the log stops. The last lines should be similar to:

2017-04-18 14:24:49,845 - vicn.core.task - INFO - Scheduling task <Task[apy] partial<_task_resource_update>> for resource <UUID MetisForwarder-BS3XG>
2017-04-18 14:24:49,846 - vicn.core.resource_mgr - INFO - Resource <UUID MetisForwarder-BS3XG> is marked as CLEAN (245/202)

You can now observe the topology by connection to your machine HTTP server (we recommend that you use Google Chrome or Chromium, as Firefox does not always handle Javascript very well). The url should look like: http://ec2-XXX-XXX-XXX-XXX.us-west-2.compute.amazonaws.com

Leave the current screen by pressing CTRL+a and then d

First traffic generation

Now that the topology is deployed, we can create some traffic on the network. We will introduce two ways of doing so: the {consumer,producer}-test application and the http-server.

Using producer-test and consumer-test

Let's start a producer on u3srv1 using the producer-test command. To do so, we open a screen and connect to the node:

$ screen -S u3srv1
$ sudo lxc shell u3srv1

We can now create some a producer for the /u3srv1/test1 prefix:

root@u3srv1:~# producer-test ccnx:/u3srv1/test1
Setting name.. ccnx:/u3srv1/test1
Route set correctly!

Let's exit the screen (CTRL+a, then d) and create a consumer on u1srv1 with consumer-test:

$ screen -S u1srv1
$ sudo lxc shell u1srv1
root@u1srv1:~# consumer-test ccnx:/u3srv1/test1

You should now see some traffic on the path between u1srv1 and u3srv1. Stop the consumer (CTRL+C) and leave the screen (CTRL+a, then d).

Using http-server

http-server is a simple app that sets up a server for downloading files over either ICN or TCP. Let's start by creating some files to download on u3srv1:

$ screen -r u3srv1

Press CTRL+C to stop the previous producer, then create a new directory and a file in that directory:

root@u3srv1:~# mkdir server_files
root@u3srv1:~# echo "This file is transfered over ICN!" > server_files/file.txt

We can now start the http-server:

root@u3srv1:~# http-server -p server_files -l ccnx:/u3srv1

To download that file, we can use the iget command. Let's leave the current screen and log back into u1srv1:

$ screen -r u1srv1
root@u1srv1:~# iget http://u3srv1/file.txt

iget will output the parameters for the congestion control algorithm (RAAQM) as well as some statistics:

Saving to: file.txt 0kB
Elapsed Time: 0.011 seconds -- 0.011[Mbps] -- 0.011[Mbps]

You can then verify that you have correctly downloaded the file:

root@u1srv1:~# cat file.txt
This file is transfered over ICN!

Leave the current screen (CTRL+a, d) and log back to u3srv1:

$ screen -r u3srv1

You should see that the interest has been received in the log:

Received interest name: ccnx:/u3srv1/get/file.txt
Starting new thread

Stop the producer with CTRL+C. You are now able to transfer any file you want using the CICN suite!

Caching experiments

Now that we learned how to create traffic, we will use it to start experimenting with the possibilities offered by ICN. In particular, we will look at how caching impact performances on large file transfer. In our network, only the core nodes (u1core, u2core and u3core) have caches, each of 2M objects. To start with, we will disable the caches on the forwarders using the metis_control command. There are two ways for disabling caching:

• cache serve off, which will prevent Metis from serving content from its cache
• cache store off, which will prevent Metis from storing the content it forwards in its cache.

For our purposes, it is enough to stop prevent Metis from serving content. We can use the lxc exec [container] -- [command] syntax to do it:

$ sudo lxc exec u1core -- metis_control --keystore keystore.pkcs12 --password password cache serve off
$ sudo lxc exec u2core -- metis_control --keystore keystore.pkcs12 --password password cache serve off
$ sudo lxc exec u3core -- metis_control --keystore keystore.pkcs12 --password password cache serve off

Now let's start a new producer on u3srv1, that serves content of size 200Mo. To do so, we will use again the producer-test application with its -s option. While the normal producer test serves the equivalent of an infinite piece of content, the -s option allows to specify the size of the content (in bytes).

$ screen -r u3srv1
root@u3srv1:~# producer-test -s 200000000 ccnx:/u3srv1/get/test2

We will now download it from all consumers at the same time. To do so, we use a script that starts iget almost simultaneously. Before starting the script, make sure that you have the monitoring open and in sight. You might see the effect of aggregation at the beginning, when the accumulated download speed of the three clients is higher than the upload speed of the producer (200Mbps). After a while, the consumers go out of sync and the congestion control protocol ensure fair distribution among the producers.

To start the script, exit the u3srv1 screen and run from the vicn folder:

$ sudo ./scripts/tutorial/tutorial04-iget.sh test2

Please note the flow completion time as given by iget. It should be around 40-60s. We will now try a similar experiment but with caches turned on:

$ sudo lxc exec u1core -- metis_control --keystore keystore.pkcs12 --password password cache serve on
$ sudo lxc exec u2core -- metis_control --keystore keystore.pkcs12 --password password cache serve on
$ sudo lxc exec u3core -- metis_control --keystore keystore.pkcs12 --password password cache serve on
$ screen -r u3srv1 #Then press CTRL+C to stop the producer-test
root@u3srv1:~# producer-test -s 200000000 ccnx:/u3srv1/get/test3

Now leave the screen and start the consumers:

$ sudo ./scripts/tutorial/tutorial04-iget.sh test3

On the monitoring tool, you can see that each consumer is downloading at 100Mbps. This causes the flow completion time to be much slower: less than 20s. This is because only the first interest per chunk to arrive at a node gets forwarded, the others are directly served from the cache.

Finally, let's rerun that experiment again:

$ sudo ./scripts/tutorial/tutorial04-iget.sh test3

This time, all the files have already been cached at the edges of the network. You can thus see the traffic being served directly from u1core and u2core while university 3 stays unburdened.

We can now conclude this part of the tutorial. Enter the u3srv1 screen and stop producer-test with CTRL+C. Then disable the caches.

$ screen -r u3srv1 #Then press CTRL+C to stop the producer-test
root@u3srv1:~# exit
$ sudo lxc exec u1core -- metis_control --keystore keystore.pkcs12 --password password cache serve off
$ sudo lxc exec u2core -- metis_control --keystore keystore.pkcs12 --password password cache serve off
$ sudo lxc exec u3core -- metis_control --keystore keystore.pkcs12 --password password cache serve off

Load balancing experiment

In this section, we try using the ICN-enabled per-packet load-balancing to take advantages of caches and ubiquitous content placement. In this scenario, the researchers at university 2 have already downloaded the data, which is thus cached in u2core. A consumer at university 1 will then be able to leverage the multiple locations of the data by using the load-balancing algorithm described in Template:Citation needed.

Let's first create some background traffic between u1srv2 and u3srv2:

$ sudo lxc exec u3srv2 -- producer-test -D ccnx:/u3srv2
$ sudo lxc exec u1srv2 -- consumer-test -D ccnx:/u3srv2

The -D option makes {producer,consumer}-test run as a daemon. You should see the corresponding traffic on the monitoring GUI.

Let's now create our producer on u3srv1 and user u2srv1 to download the corresponding content:

$ screen -r u3srv1
root@u3srv1:~# producer-test -s 200000000 ccnx:/u3srv1/get/test4
# Exit the screen with CTRL+A, d
$ sudo lxc exec u2srv1 -- iget http://u3srv1/test4

Note that the download time is around 15-20s. Now let's enable the cache on u2core, so that /u3srv1/get/test4 is also available there:

$ sudo lxc exec u2core -- bash -c "metis_control --keystore keystore.pkcs12 --password password cache serve on"

We also need to make u1core aware that the content is available on u2core. Three steps are necessary to do that:

1. Find the IP addresses for the link between u1core and u2core
2. Find the corresponding Metis connection on u1core
3. Add a route to /u3srv1/get/test4 on that connection

To do so, we must find out the IP of the interface of u1core (resp. u2core) connected to u2core (resp. u1core):

$ sudo lxc list
+--------+---------+--------------------------------+------+------------+-----------+
|  NAME  |  STATE  |              IPV4              | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+--------------------------------+------+------------+-----------+
| u1core | RUNNING | 192.168.128.9 (u2core)         |      | PERSISTENT | 0         |
|        |         | 192.168.128.7 (u1srv2)         |      |            |           |
|        |         | 192.168.128.5 (u1srv1)         |      |            |           |
|        |         | 192.168.128.20 (eth0)          |      |            |           |
|        |         | 192.168.128.11 (u3core)        |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+
| u1srv1 | RUNNING | 192.168.128.4 (u1core)         |      | PERSISTENT | 0         |
|        |         | 192.168.128.21 (eth0)          |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+
| u1srv2 | RUNNING | 192.168.128.6 (u1core)         |      | PERSISTENT | 0         |
|        |         | 192.168.128.22 (eth0)          |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+
| u2core | RUNNING | 192.168.128.8 (u1core)         |      | PERSISTENT | 0         |
|        |         | 192.168.128.23 (eth0)          |      |            |           |
|        |         | 192.168.128.15 (u3core)        |      |            |           |
|        |         | 192.168.128.13 (u2srv1)        |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+
| u2srv1 | RUNNING | 192.168.128.24 (eth0)          |      | PERSISTENT | 0         |
|        |         | 192.168.128.12 (u2core)        |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+
| u3core | RUNNING | 192.168.128.25 (eth0)          |      | PERSISTENT | 0         |
|        |         | 192.168.128.19 (u3srv2)        |      |            |           |
|        |         | 192.168.128.17 (u3srv1)        |      |            |           |
|        |         | 192.168.128.14 (u2core)        |      |            |           |
|        |         | 192.168.128.10 (u1core)        |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+
| u3srv1 | RUNNING | 192.168.128.26 (eth0)          |      | PERSISTENT | 0         |
|        |         | 192.168.128.16 (u3core)        |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+
| u3srv2 | RUNNING | 192.168.128.27 (eth0)          |      | PERSISTENT | 0         |
|        |         | 192.168.128.18 (u3core)        |      |            |           |
+--------+---------+--------------------------------+------+------------+-----------+

In this case, it's 192.168.128.9 (resp. 192.168.128.8) (it should be consistent on your system). We can now find the corresponding connection in Metis on u1core:

$ sudo lxc exec u1core -- bash -c "metis_control --keystore keystore.pkcs12 --password password list connections"
metis 1.0.20170411 2017-02-17T15:34:37.319950
Copyright (c) 2017 Cisco and/or its affiliates. 

            __  __        _    _
           |  \/  |  ___ | |_ (_) ___
           | |\/| | / _ \| __|| |/ __|
           | |  | ||  __/| |_ | |\__ \
           |_|  |_| \___| \__||_||___/

Using keystore: keystore.pkcs12
    3   UP inet4://192.168.128.5:6363 inet4://192.168.128.4:6363 UDP
    5   UP inet4://192.168.128.9:6363 inet4://192.168.128.8:6363 UDP
    7   UP inet4://192.168.128.11:6363 inet4://192.168.128.10:6363 UDP
    9   UP inet4://192.168.128.7:6363 inet4://192.168.128.6:6363 UDP
   10   UP inet4://127.0.0.1:9695 inet4://127.0.0.1:34846 TCP
   11   UP inet4://127.0.0.1:9695 inet4://127.0.0.1:34848 TCP

We find that the matching IPs are the connection number 5. We can now add the route:

$ sudo lxc exec u1core -- bash -c "metis_control --keystore keystore.pkcs12 --password password add route 5 ccnx:/u3srv1/get/test4 1"

You can test that the route has been added by running:

$ sudo lxc exec u1core -- bash -c "metis_control --keystore keystore.pkcs12 --password password list routes"

Now proceed to the same manipulation for the connection between u1core and u3core (IP addresses should be 192.168.128.11 and 192.168.128.10). Finally, let's tell Metis to do load-balancing for our prefix:

$ sudo lxc exec u1core -- bash -c "metis_control --keystore keystore.pkcs12 --password password set strategy ccnx:/u3srv1/get/test4 loadbalancer"

We are now ready to start the consumer on u1srv1:

$ screen -r u1srv1
root@u1srv1:~# iget http://u3srv1/test4

Look at the monitoring tool. You can see how most of the load is put on the powerful link between u1core and u2core, while u3srv1 only has to serve around 20Mbps, thus saving bandwith and CPU power.

Cleaning up

Congratulations, you have completed this tutorial! You can now clean your machine. First close your screens:

$ screen -r u1srv1
root@u1srv1:~# exit
$ screen -r u3srv1
# Use CTRL+C to stop the producer
root@u3srv1:~# exit
$ screen -r vicn
# Use CTRL+C to stop vICN
$ exit

Now you can clean the topology using the cleanup script. In the vICN folder, run:

$ sudo ./scripts/topo_cleanup.sh examples/tutorial/tutorial04-caching.json

This script will remove the containers, the virtual bridge and any other remains of your experiment. Your machine is now ready to deploy another topology!