VPP/Modifying The Packet Processing Directed Graph

Introduction

The VPP platform uses a frame dispatcher and a directed node graph to process packets. Teaching the VPP platform how to process new kinds of traffic - or how to process known traffic in a different way - boils down to modifying graph trajectories.

There are several ways to do this. At the simple end of the spectrum, vpp device drivers implement a per-interface RX redirect function. All packets from the indicated interface are delivered to the supplied node.

For-us packets sent to a specific (address family, protocol-ID) can be delivered to a specific graph node. Implementing a new IP protocol works this way.

Protocols such as vxlan which involve processing for-us packets sent to a specific udp dst port have a similar (completely straighforward) API.

One can add per-interface configurable feature arcs at specific points in the forwarding graph. Example: ip4-input and ip4-input-no-checksum share a set of per-configurable features. Depending on configuration, packets visit ip4-input -> ip4-lookup, ip4-input -> ip4-source-check-via-rx, ip4-input -> ip4-source-check-via-any, ip4-input -> ipsec-input-ip4, or combinations thereof.

Full N-tuple classification provides another degree of freedom.

The show vlib graph debug CLI command shows the entire graph. It's not unknown for graph arcs which "should have been created" to end up missing.

NOTE: Be aware of warning messages such as:

 Complaints at application start time of the form "node XYZ not found" are not to be ignored.

If you see such a message, the forwarding graph will not have been correctly constructed. Crashes in vlib_get_next_frame (...) are almost always the result of missing successor nodes. Typos in VLIB_REGISTER_NODE(...) macros can easily cause this kind of issue.

Forgetting to include a VLIB_INIT_FUNCTION(...) macro is another way to prevent graph arcs or other kinds of registrations from occurring.

Per-interface RX Redirection Details

Use the vnet_hw_interface_rx_redirect_to_node API to direct all packets from a certain interface to the graph node of your choice, or to cancel redirection:

 vnet_hw_interface_rx_redirect_to_node 
 (vnet_main, hw_if_index, my_graph_node.index /* redirect to my_graph_node */);

or

 vnet_hw_interface_rx_redirect_to_node 
 (vnet_main, hw_if_index, ~0 /* disable redirection */);

The my_graph_node index might hand "uninteresting" traffic to the normal code path, specifically ethernet-input, ip4-input / ip4-input-no-checksum, or ip6-input.

If you need to build a "bump-in-the-wire" application, use the rx redirect API to capture input packets and send them to my_graph_node. The my_graph node index performs the required analysis, and sends traffic to the output node for a specific TX interface.

Capturing Packets with Particular Ethertypes

Registering a node to handle packets with specific, currently unimplemented Ethertypes is easy. The following registration would send Cisco Discovery Protocol (ethertype 0x2000) packets to cdp_input_node:

 ethernet_register_input_type (vm, ETHERNET_TYPE_CDP, cdp_input_node.index);

The easiest way to replace the existing ip4, ip6, or mpls stacks would be to modify vpp/vnet/main.c so that it doesn't link the current stack(s) from vnet, and to supply functionally equivalent nodes with the same names. For performance, device drivers know how to send ip4 packets, for example to ip4-input-no-checksum. It would be best not to make things more difficult by insisting on other driver-to-stack entry-point names.

Capturing for-us Packets for Newly-implemented IP Protocols

Take a look at the .../vpp/vnet/vnet/gre/node.c file. We register the gre_input_node index to capture IP_PROTOCOL_GRE packets through the ip4_register_protocol API:

 ip4_register_protocol (IP_PROTOCOL_GRE, gre_input_node.index);

The ip4 and ip6 implementations are quite symmetrical, so it makes sense that there is an ip6_register_protocol API. Here's an example from the .../vpp/vnet/vnet/l2tp/decap.c source file:

 ip6_register_protocol (IP_PROTOCOL_L2TP, l2t_decap_node.index);

Capturing ip-for-us Packets Sent to Specific UDP dst Ports

Use the udp_register_dst_port API. Here's an example from .../vpp/vnet/vnet/vxlan/vxlan.c, which sends ip4-udp-for-us packets sent to udp dst port 4789 to the vxlan_input_node:

udp_register_dst_port (vm, UDP_DST_PORT_vxlan, 
                       vxlan_input_node.index, 1 /* is_ip4 */);

ip4 and ip6 have separate local-stack udp dst port decode tables.

Dispatch Function Hackery

It may seem attractive to change the vlib_node_runtime_t function pointer in arbitrary ways. One can imagine adding an API to track down the entire set of graph / node runtime replicas, change the dispatch function, and return the previous dispatch function. In a multi-threaded case, the node graph will have been replicated: vlib_update_node_runtime(...).