VPP/Code Walkthrough VoD Topic Index

vpp code walkthrough index and notes

@0:09:15 - @0:43:50 VPP initialization

Constructors are declared using VLIB_INIT_FUNCTION macro. These constructors add functions passed via this macro to a global linked list: vlib_main_t->init_function_registrations.

This implies that this linked list is created before main() is called. However, the function themselves are invoked later by vlib/vlib/init.c::vlib_call_all_init_functions() before vlib_main_loop() is executed. Similarly global linked lists are created by constructors declared by macros such as:

• VLIB_API_INIT_FUNCTION
• VLIB_CLI_COMMAND
• VLIB_CONFIG_FUNCTION
• VLIB_EARLY_CONFIG_FUNCTION
• VLIB_MAIN_LOOP_ENTER_FUNCTION
• VLIB_MAIN_LOOP_EXIT_FUNCTION
• VLIB_REGISTER_NODE

vpp/vnet/main.c:main()

This is the executable entry point. Sets a function pointer to vnet/main.c::vnet_get_handoff_structure(), in vlib_plugin_main called handoff_structure_get_cb. This function gets invoked vlib/unix/plugin.c::vnet_get_handoff_structure() is invoked.

vnet_get_handoff_structure() declares a static variable that contains pointers to main data structures like vlib_main, vnet_main and ethernet_main. This hand-off structure is passed to each plug-in via vlib_plugin_register() function, defined in each plug-in when the plug-ins are registered.

It then invokes vlib_unix_main().

vlib/unix/main.c::vlib_unix_main()

Invoke vlib_plugin_early_init() that loads all the plug-ins by performing dlopen for all the libraries found in plug-in directory that can be specified via command line. The default plugin path is /usr/lib/vpp_plugins.

For each plug-in dlopen-ed, VPP gets the symbol address of a function named "vlib_plugin_register". This means each plug-in must implement this function. It passes important data structures as explained above.

Parses all the command line option in function vlib_call_all_config_functions() and also performs any early configurations that are required.

Creates thread stacks for the following types of threads. Mainly there are 3 types of threads are implemented.

1. Regular pthreads: Ex: Stats collector. 2. EAL threads: These are workers that process packets 3. Processes: These are co-operating multitasking threads that gets executed periodically. For example, DHCP lease renewal thread, etc. These are scheduled by the main VPP thread if its timer has expired.

Performs a long jump to thread0().

vlib/unix/main.c::thread0() Just invokes vlib/main.c::vlib_main()

vlib/main.c::vlib_main() Graph nodes are created (not linked) by vlib/node.c::vlib_register_all_static_nodes() by walking a linked list. This linked list itself is created by constructors declared via VLIB_REGISTER_NODE macro.

Once these nodes are created, they are connected appropriately by vlib/node.c::vlib_node_main_init(). Before this, all the initialization routines declared via VLIB_INIT_FUNCTION are invoked by vlib_call_all_init_functions().

Invokes vlib_call_all_main_loop_enter_functions() that invokes functions by walking a linked list that are registered via VLIB_MAIN_LOOP_ENTER_FUNCTION.

Calls vlib_main_loop()

vlib/main.c::vlib_main_loop() Creates all the co-operative multitasking process explained earlier. Now the main while(1) loop starts. Processes different types different types of graph nodes.

VLIB_NODE_TYPE_PRE_INPUT: These are nodes like DBG_CLI

VLIB_NODE_TYPE_INPUT: These are the main nodes which reads packets out of the network interfaces.

Processes pending signals. This is important as all clients communicate to VPP via shared memory. This means, that the client puts some API messages in shared memory area and sends a signal (SIGUSR1) to VPP.

The input nodes would have sucked packets out of the interface and have moved it to appropriate intermediate nodes. These partially processed packets are further processed by dispatch_pending_node().

This never exits.

@0:43:50 - @1:02:00 Performance and Measurements by Maciek Konstantynowicz

@1:02:55 - @1:15:30 VPP bring-up plus a simple ping test by Dave Barach

@1:16:05 - @1:30:00 VPP API by Dave Barach

During start, VPP spins memclnt_process thread that is used to process any incoming API messages. This includes all API messages sent from any client.

When ever a client starts, it has to spin a thread to receive any responses asynchronously from VPP. Instantiation of this thread is done as a side-effect of connecting to vpp in connect_to_vpe() function. Let us called it side-effect thread.

The client puts an API message in a shared memory mailbox and sends a signal (SIGUSR1) to VPP. The main thread that sent the API will call W; that waits until the side-effect thread either sets vam->reply_ready or will timeout after 1 second.

VPP's memclnt_process invokes appropriate handler and puts any response back in the shared memory mailbox and sends a signal back to the client that is handled by side-effect thread. The client's side-effect thread invokes appropriate handler, which sets vam->reply_ready.

@1:30:00 - @1:45:00 Build and deploy a plug-in by Dave Barach This explains how a plug-in can be built and how to access it via DBG_CLI or API. Additionally, he explains how plug-in interfaces such as enable/disable etc (not network interfaces) are exposed that can be controlled via VPP API.

@1:45:00 - @2:09:10 Deep-dive into one of the sample plug-in by Dave Barach A mac-swap sample plug-in is used for explanation and Dave takes this opportunity to explain how a graph node works as the plug-in itself is seen as a graph node by VPP.

@2:09:10 - @2:29:30 VPP Binary API by Dave Barach Initialization of binary APIs How to hook an API Enabling and disabling API programatically Registering API to be accessible by VPP API Test program

@2:29:30 - @2:35:00 Detour to explain more of VPP API test program by Dave Barach

@2:37:00 - @2:43:43 Q & A by Dave Barach

              Request for VPP DPDK interaction, which was done separately.
              Vincent asked about performance number between DPDK s ENIC driver and VPP s VIC driver
              Maciek and Dave Barach opined that they will publish those when available.
              Vincent suggested tools to test VPP subsystem via DPDK test framework.

@2:43:43 - @3:07:00 Thread support in VPP by Damjan Marion

@3:07:00 - @3:14:38 Random Chat Comparison of HW and SW RSS that is being implemented in VPP. Configuration with IO threads and workers threads is referred as SW-RSS.

@3:14:38 - @3:33:30 DPDK + VPP interaction by Dave Barach

@3:33:30 - @3:45:45 Discussion on rte_mbuf structure Why Dave Barach thinks that a structure in DPDK should be re-organized for better performance as current structure organization leads to cache trashing because one field in the structure (next pointer) crosses cache-line boundary.

@3:45:45 - @3:47:13 How DPDK is patched and compiled in VPP by Damjan Marion

@3:47:13 - @3:57:00 Q & A Topics covered: 1. Mailing List 2. Running Coverity 3. DPDK's ENIC driver vs VPP's VIC driver 4. Request to provide pictorial representation of VPP 5. Why autoconf tools  ? 6. Vhost drivers

@3:57:00 - @3:58:00 Thank You note by Mike O'Gorman