VPP/Code Walkthrough VoD Topic Index

vpp code walkthrough index and notes

@0:09:15 - @0:43:50 VPP initialization

Constructors are declared using VLIB_INIT_FUNCTION macro. These constructors add functions passed via this macro to a global linked list: vlib_main_t->init_function_registrations.

This implies that this linked list is created before main() is called. However, the function themselves are invoked later by vlib/vlib/init.c::vlib_call_all_init_functions() before vlib_main_loop() is executed. Similarly global linked lists are created by constructors declared by macros such as:

• VLIB_API_INIT_FUNCTION
• VLIB_CLI_COMMAND
• VLIB_CONFIG_FUNCTION
• VLIB_EARLY_CONFIG_FUNCTION
• VLIB_MAIN_LOOP_ENTER_FUNCTION
• VLIB_MAIN_LOOP_EXIT_FUNCTION
• VLIB_REGISTER_NODE

vpp/vnet/main.c:main()

This is the executable entry point. Sets a function pointer to vnet/main.c::vnet_get_handoff_structure(), in vlib_plugin_main called handoff_structure_get_cb. This function gets invoked vlib/unix/plugin.c::vnet_get_handoff_structure() is invoked.

vnet_get_handoff_structure() declares a static variable that contains pointers to main data structures like vlib_main, vnet_main and ethernet_main. This hand-off structure is passed to each plug-in via vlib_plugin_register() function, defined in each plug-in when the plug-ins are registered.

It then invokes vlib_unix_main().

vlib/unix/main.c::vlib_unix_main()

Invoke vlib_plugin_early_init() that loads all the plug-ins by performing dlopen for all the libraries found in plug-in directory that can be specified via command line. The default plugin path is /usr/lib/vpp_plugins.

For each plug-in dlopen-ed, VPP gets the symbol address of a function named "vlib_plugin_register". This means each plug-in must implement this function. It passes important data structures as explained above.

Parses all the command line option in function vlib_call_all_config_functions() and also performs any early configurations that are required.

Creates thread stacks for the following types of threads. Mainly there are 3 types of threads are implemented.

1. Regular pthreads: Ex: Stats collector.
2. EAL threads: These are workers that process packets
3. Processes: These are co-operating multitasking threads

that gets executed periodically. For example, DHCP lease renewal thread, etc. These are scheduled by the main VPP thread if its timer has expired.

Performs a long jump to thread0().

vlib/unix/main.c::thread0()

Invokes vlib/main.c::vlib_main()

vlib/main.c::vlib_main()

Graph nodes are created (not linked) by vlib/node.c::vlib_register_all_static_nodes() by walking a linked list. This linked list itself is created by constructors declared via VLIB_REGISTER_NODE macro.

Once these nodes are created, they are connected appropriately by vlib/node.c::vlib_node_main_init(). Before this, all the initialization routines declared via VLIB_INIT_FUNCTION are invoked by vlib_call_all_init_functions().

Invokes vlib_call_all_main_loop_enter_functions() that invokes functions by walking a linked list that are registered via VLIB_MAIN_LOOP_ENTER_FUNCTION.

Calls vlib_main_loop()

vlib/main.c::vlib_main_loop()

Creates all the co-operative multitasking process explained earlier. Now the main while(1) loop starts. Processes different types different types of graph nodes.

• VLIB_NODE_TYPE_PRE_INPUT: These are nodes like DBG_CLI

• VLIB_NODE_TYPE_INPUT: These are the main nodes which inject frames of packets extracted from network interfaces, or from hardware accelerators

• Processes pending signals. This is important as all clients communicate

to VPP via shared memory. This means, that the client puts some API messages in shared memory area and sends a signal (SIGUSR1) to VPP.

Input nodes form frames of packets, and dispatch them to appropriate intermediate nodes. These partially processed packets are further processed by dispatch_pending_node().

@1:16:05 - @1:30:00 VPP API by Dave Barach

During start, VPP spins memclnt_process thread that is used to process any incoming API messages. This includes all API messages sent from any client.

Whenever a client starts, it has to spin a thread to receive any responses asynchronously from VPP. Instantiation of this thread is done as a side-effect of connecting to vpp in connect_to_vpe() function.

The client puts an API message in a unidirectional shared memory queue and sends SIGUSR1 to VPP if the VPP input queue just transitioned from empty to non-empty. The main thread that sent the API will call W; that waits until the side-effect thread either sets vam->reply_ready or will timeout after 1 second.

VPP's memclnt_process invokes appropriate handler and replies via the client's unidirectional shared memory queue. Again, if the queue transitions from empty to non-empty, vpp signals the client RX thread. The client RX thread invokes appropriate handler, which sets vam->reply_ready.

@1:30:00 - @1:45:00 Build and deploy a plug-in by Dave Barach

Explain how a plug-in can be built and how to access it via DBG_CLI or API. Additionally, he explains how plug-in interfaces such as enable/disable etc (not network interfaces) are exposed that can be controlled via VPP API.

@1:45:00 - @2:09:10 Deep-dive into one of the sample plug-in by Dave Barach

A mac-swap sample plug-in is used for explanation and Dave takes this opportunity to explain how a graph node works as the plug-in itself is seen as a graph node by VPP.

@2:09:10 - @2:29:30 VPP Binary API by Dave Barach

• Initialization of binary APIs
• How to hook up a binary API
• Enabling and disabling APIs programatically
• Registering API to be accessible by VPP API Test program

@2:37:00 - @2:43:43 Q & A by Dave Barach

• Request for VPP DPDK interaction, which was done separately.
• Vincent Jardin asked about performance number between DPDK s ENIC driver and VPP s VIC driver
• Maciek and Dave Barach opined that they will publish those when available.
• Vincent Jardin suggested tools to test VPP subsystem via DPDK test framework.

@3:07:00 - @3:14:38 Misc Discussions

• Comparison of HW and SW RSS that is being implemented in VPP.
• Configuration with IO threads and workers threads is referred as SW-RSS.

@3:33:30 - @3:45:45 Discussion on rte_mbuf structure

Suggestions for reorganization for better performance as current structure organization leads to cachet thrashing. The "next" field in the structure currently causes an additional cache-line fetch.

@3:47:13 - @3:57:00 Q & A

Topics covered:

1. Mailing List
2. Running Coverity
3. DPDK's ENIC driver vs VPP's VIC driver
4. Request to provide pictorial representation of VPP
5. Why autoconf tools  ?
6. Vhost drivers