VPP/Documentation

The VPP project currently uses Doxygen as the mechanism to generate documentation. This is predominantly focused on providing developer-focused information but it can also be used to generate user-focused details.

Doxygen works by parsing source files and identifying special comment blocks that are adjacent to identifiers in the code. We have Doxygen configured to generate output for items that are not yet documented so that at the very least the names of function parameters or structure members are visible.

This wiki page aims to cover how developers can go about producing documentation. Whilst it will cover a number of Doxygen features, readers are encouraged to review the Doxygen documentation and in particular special commands page.

Documentation is automatically generated after changes are merged into the source repository. For the current development branch this is visible at https://docs.fd.io/vpp/16.12/ and for the most recent release branch at https://docs.fd.io/vpp/16.09/.

Documenting the code

Language

The language of written documentation in VPP is English. It is tempting, as developers, to write tersely with the belief that doing so will adequately convey the relevant details. This should be avoided since readers of the resulting documentation may find the text halting and lacking context. This is especially true if English is not the readers' native tongue.

As a consequence documentation should be written with structurally correct English complete with capitalization, sentences and paragraphs where appropriate. Assumptions regarding context should be avoided. Spelling mistakes and grammar are easily fixed later; structure and context are much harder to fill in.

What to document

In an ideal world we would document everything; however given the realities of life and open source projects here are a list of items that should be prioritized for documenting and where the better the documentation the better able VPP is to mature:

• Graph node functions.
  • See the graph node template below.
• Widely-used library functions, structures and pre-processor macros.
  • Most of vppinfra and vlib.
  • Node and feature lookup, insertion or other manipulation functions.
  • *_format and *_unformat functions.
  • FIB and other centralized data store and lookup mechanisms.
• *_main_t and other primary data structures used for interacting with modules.
• CLI short_help and long_help.
  • These will eventually be extracted for automatic documentation.
• API definitions.
  • At the moment most API documentation just describes the parameters; better description of what API calls do is necessary in many cases.
• Anything else that may be of use to the developer who comes after you.
  • Be kind to your future self.

In general, static and inline functions within modules are not visible outside that module and thus adding a documentation block may be of limited value (with exceptions, such as when used for function pointers for consumption elsewhere); though this does not excuse a developer from writing useful comments. However static and inline functions in header files are much more widely visible and should be documented.

In general, even though the purpose of an item may be obvious from its name, when rendered as documentation this implicit meaning is not always forthcoming. Thus it is recommended that such items not be skipped from consideration merely because the meaning seems plain. For example, structures (and unions) require documentation describing both the structure itself as well as all of its members. Similarly enumerations should have documentation both for the enumeration as a whole as well as individual items defined by it. It is very apparent in the generated documentation when developers only partially document these things.

Global variables must be documented. Since global variables are evil but are sometimes necessary we should encourage developers to justify their existence, to apologize to future programmers and beg their forbearance for the indiscretion.

Documentation conventions

To promote consistency both in the VPP source code and in the generated text, these are the conventions VPP have adopted for writing documentation.

Doxygen comments/commands

Doxygen requires the use of specially marked comment blocks to identify documentation and recognizes several special commands inside those blocks. It provides several ways to indicate both of these things. Whilst any will work, VPP as a project has adopted the following as conventions:

• Pre-identifier comment blocks are signified with /** ... */.
  • These are documentation blocks that come immediately before the code that they provide documentation for.
• Post-identifier comment blocks are given with /**< ... */.
  • These are documentation blocks that come immediately after the code that they provide documentation for.
  • In some situations using ///< ... may make the code more readable and this is acceptable.
• Special commands are prefixed with the at-sign, for example @param or @brief.
  • Though you may often find the use of a back-slash in the existing code (especially \brief) these are being migrated to using at-sign.
  • Note that the Doxygen documentation will show use of back-slash throughout, but it notes that the two symbols are equal and interchangeable.

Referring to identifiers in the text

Doxygen will automatically spot the names of known functions in the text and link those to the definition of that function. For other identifiers you must tell it to using @ref and this is encouraged in all places; note that not all things are linkable in this way (refer to the Doxygen documentation) but it is harmless to use @ref in such places.

For other items, such as function parameters, local variables or constants the use of @c is recommended. This will mono-space the text thus providing contrast with the surrounding text. Note that this works only for a single word; <tt> ... </tt> can be used for multiple words or @code ... @endcode for a C code block.

Describing functions and their parameters

Functions are typically documented with a docblock immediately before the function definition. They contain a brief description, a fuller description, a description of each parameter and information on any return values.

This is straightforward to use, like so:

/**
 * @brief A widget coloring function.
 *
 * This is the function for changing the color of widgets.
 *
 * @note It is possible to 
 *
 * @param widget A reference to the widget to operate on.
 * @param type   The type of operation. The widget will be obliterated
 *               if this value is prime. Colors will be opaque if
 *               the value is narcissistic. Transparency can be determined
 *               by rotating the number 90 degrees. Any other value will
 *               change its color. Negative values have undesirable effects
 *               if you are afraid of spiders.
 *
 * @returns 42 on good days.
 * @returns Random numbers otherwise.
 */
static word
widget_colorizer (widget_t *widget,
            word value)
{
  ...
  return 42;
}

Things to note:

• The @brief line is exactly that: brief. It is a one-sentence description of the purpose of the function. In particular note that it ends with a period '.'. This is important since that is what ends a sentence; otherwise the text from the next paragraph would run-on as part of the brief description when Doxygen parses it. It is also possible to dispense with the @brief command; when Doxygen notices it is missing it will use the first sentence of the long description for the brief sentence instead.
• The @param commands do not provide extra formatting (like hyphens '-') nor do they repeat the type information from the signature (since that would be redundant and often leads to errors. Refer to the Doxygen documentation for more information on how to use @param.
  • Doxygen automatically includes type information in the function signature section.
  • There are some places that do include type information, including the boiler plate node documentation template, but only because that is an apt way to describe the parameter. It is not encouraged.
  • It's important to remember that this information is captured semantically for use by other tools, not just HTML page rendering. Use only Doxygen formatting commands if formatting is needed.
  • The start of parameter descriptions has been horizontally aligned to aid readability of the source.
• The @returns (or @return) also does not include type information.
  • Often just one @returns is sufficient but it is also valid to use more than one if that reads better, as in this example.

Otherwise common-sense should prevail. Describe what a function does, what its side-effects are and how it should be used. For more complex functions try to include examples of how to use it.

Describing function-like macros and their parameters

Macros can be documented just like functions with one significant difference: Doxygen cannot guess the type of each parameter or any return value and thus the documentation must mention parameter typing if it is important.

It's also important to mention the side-effects a macro may have or other local variables that must be present for it to work.

Documenting Debug CLI commands

Much of the documentation for CLI commands is extracted automatically from the VLIB_CLI_COMMAND() initialization block that is used to create one in the first place. We have augmented this with our own documentation block format that can be used to provide verbose documentation for CLI commands without needing to compile them into the VPP binary.

<detail on convention coming soon>

Extra commands

These commands have been added to Doxygen (using the ALIAS mechanism) to assist with documenting specific items in a consistent way.

@node: Indexing graph node identifiers

The @node command is used to document the name of graph nodes. The command makes use of the Doxygen @xrefitem command to generate an index of graph nodes and renders the name of the node in mono-spaced type within the node documentation.

See below in the node template section for an example of how this is used and to see now the node name is rendered within the documentation output. The index, when rendered, will look something like this:

Graph node index

Documentation templates

Source code file header

All of our source code files are required to include the Apache 2.0 license preamble. To properly document files they should also contain at minimum a brief description of the contents of that module and preferably a longer description to give some detail; for example, describing how to interact with it, the key functionality it provides and any special items of note.

/*
 * Copyright (c) <current-year> <your-affiliation-here>.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at:
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
/**
 * @file
 * @brief Your brief description goes here.
 *
 * Your much longer description goes here. This should be written in
 * structurally correct English, with proper capitalization, sentences
 * and paragraphs.
 */

An example of the @file section might read:

/**
 * @file
 * @brief Unix stdin/socket command line interface.
 *
 * Provides a command line interface so humans can interact with VPP.
 * This is predominantly a debugging and testing mechanism.
 */

Pay careful attention that the @file directive is provided as-is. It's tempting to add a filename after this directive but please do not do this, it only confuses Doxygen.

Also note that all @brief directives need to end in a '.' to terminate the directive.

Directory description

So that directories are appropriately described in the file browser each directory with source code in should have a file with the name dir.dox. At minimum that file should contain a brief description of the contents of the directory but it could also contain a much longer description.

Refer to the file doxygen/dir.dox.sample:

/*
 * Copyright (c) <current-year> <your-affiliation-here>.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at:
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
/**
 * @dir
 * @brief Someone please fix this description.
 * @todo This directory needs a description.
 *
 * This is where you would document the contents of a directory.
 *
 * This looks like a C file but it is not part of the build; it is purely
 * for documentation.
 */
/*? %%clicmd:group_label CLI section description%% ?*/
/*? %%syscfg:group_label Startup config section description%% ?*/

Pay careful attention to the @dir directive; it must be provided as-is. It is tempting to provide a path after it but please do not do this; it often confuses Doxygen.

Also note that all @brief directives need to end in a '.' to terminate the directive.

Documenting a graph node function

Nodes form the core of what VPP does and providing meaningful documentation about their function is important. This is doubly so for those nodes which are direct ancestors or descendants of a node a developer is working; that individual will be looking for details on the features the surrounding nodes make available. Examples may include the metadata passed along in the buffer opaque data which will help a node understand where a packet came from. Some nodes will expect data in the opaque section in order to steer their behavior.

By way of example, here is the documentation block for the "ip4-lookup" node and which may be used as a template:

/**
 * @brief IPv4 lookup node.
 * @node ip4-lookup
 *
 * This is the main IPv4 lookup dispatch node.
 *
 * @param vm    vlib_main_t corresponding to the current thread.
 * @param node  vlib_node_runtime_t data for this node.
 * @param frame vlib_frame_t whose contents should be dispatched.
 *
 * @par Graph mechanics: buffer metadata, next index usage
 *
 * <em>Uses:</em>
 * - <code>vnet_buffer(b)->sw_if_index[VLIB_RX]</code>
 *     - Indicates the @c sw_if_index value of the interface that the
 *       packet was received on.
 * - <code>vnet_buffer(b)->sw_if_index[VLIB_TX]</code>
 *     - When the value is @c ~0 then the node performs a longest prefix
 *       match (LPM) for the packet destination address in the FIB attached
 *       to the receive interface.
 *     - Otherwise perform LPM for the packet destination address in the
 *       indicated FIB. In this case <code>[VLIB_TX]</code> is a FIB index
 *       value (0, 1, ...) and not a VRF id.
 *
 * <em>Sets:</em>
 * - <code>vnet_buffer(b)->ip.adj_index[VLIB_TX]</code>
 *     - The lookup result adjacency index.
 *
 * <em>Next Index:</em>
 * - Dispatches the packet to the node index found in
 *   ip_adjacency_t @c adj->lookup_next_index
 *   (where @c adj is the lookup result adjacency).
 */
static uword
ip4_lookup (vlib_main_t * vm,
            vlib_node_runtime_t * node,
            vlib_frame_t * frame)
{
  ...
}

When rendered, this documentation block will look something like this:

Rendered documentation for the "ip4-lookup" node

Of particular note this example describes these things:

• The node name is documented with the special "@node" command.
• A description of what extra data it uses as inputs; in this case the text describes how ip4-lookup makes use of vnet_buffer(b)‑>sw_if_index[VLIB_RX] and vnet_buffer(b)‑>sw_if_index[VLIB_TX].
• A description of what extra data is passed on to descendants; in this case vnet_buffer(b)‑>sw_if_index[VLIB_TX].
• A description of how the next node index is determined.

In this case the bulk of the node functionality can be described in terms of the sw_if_index sections; other nodes may need to provide more verbiage in other sections of the text. In particular, where nodes have other side-effects those should be clearly documented.

Documenting a Debug CLI command

Each CLI Command is defined and documented using the VLIB_CLI_COMMAND pre-processor macro. A special documentation comment, delimited by comment markers that look like /*? text-goes-here... ?*/, is added just before the VLIB_CLI_COMMAND macro call. A mechanism extracts this text along with the contents of the structure and passes it to Doxygen. Below is an example of how to document a CLI Command:

/*?
 * The set of '<em>test l2fib</em>' commands allow the L2 FIB table of the default
 * bridge domain (bridge-domain-id of 0) to be modified.
 *
 * @cliexpar
 * @parblock
 * Example of how to add a set of 4 sequential MAC Address entries to L2
 * FIB table of the default bridge-domain:
 * @cliexcmd{test l2fib add mac 52:54:00:53:00:00 count 4}
 *
 * Show the set of 4 sequential MAC Address entries that were added:
 * @cliexstart{show l2fib verbose}
 *     Mac Address     BD Idx           Interface           Index  static  filter  bvi  refresh  timestamp
 * 52:54:00:53:00:00    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
 * 52:54:00:53:00:01    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
 * 52:54:00:53:00:03    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
 * 52:54:00:53:00:02    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
 * 4 l2fib entries
 * @cliexend
 *
 * Example of how to check that the set of 4 sequential MAC Address
 * entries were added to L2 FIB table of the default
 * bridge-domain. Used a count of 5 to produce an error:
 *
 * @cliexcmd{test l2fib check mac 52:54:00:53:00:00 count 5}
 * The output of the check command is in the log files. Log file
 * location may vary based on your OS and Version:
 *
 * <b><em># tail -f /var/log/messages | grep l2fib_test_command_fn</em></b>
 *
 * Sep  7 17:15:24 localhost vnet[4952]: l2fib_test_command_fn:446: key 52:54:00:53:00:04 AWOL
 *
 * Example of how to delete a set of 4 sequential MAC Address entries
 * from L2 FIB table of the default bridge-domain:
 * @cliexcmd{test l2fib del mac 52:54:00:53:00:00 count 4}
 * @endparblock
?*/
/* *INDENT-OFF* */
VLIB_CLI_COMMAND (l2fib_test_command, static) = {
  .path = "test l2fib",
  .short_help = "test l2fib [add|del|check] mac <base-addr> count <nn>",
  .function = l2fib_test_command_fn,
};
/* *INDENT-ON* */

Once the documentation is generated, the above Doxygen encoding will produce the following output:

oxygen generated documentation for the CLI command "test l2fib".

Below is a description of each section of the generated document.

Summary/usage

The Summary/usage section is taken from .short_help variable of the VLIB_CLI_COMMAND section. The text associated with this variable is also used in the CLI help output. The current usage of this variable is to provide the command syntax. Longer term, new variables will be introduced so that there can be both a syntax and an abbreviated command description. For now, it is used as the command syntax.

VLIB_CLI_COMMAND (l2fib_test_command, static) = {
  .path = "test l2fib",
  .short_help = "test l2fib [add|del|check] mac <base-addr> count <nn>",
  .function = l2fib_test_command_fn,
};

NOTE: If the declaration name in the the VLIB_CLI_COMMAND pre-processor macro is real long, it may be tempting to break it across multiple lines. This causes issues in the parsing and will cause the associated generated documentation to end up on the wrong command.

Correct:

VLIB_CLI_COMMAND (set_interface_ip_source_and_port_range_check_command, static) = {
   :
};

Incorrect:

VLIB_CLI_COMMAND (set_interface_ip_source_and_port_range_check_command,
                  static) = {
   :
};

Description

The Description section is the text preceding the @cliexpar line. This section can span multiple lines and contain multiple paragraphs, separated by blank lines.

/*?
 * The set of '<em>test l2fib</em>' commands allow the L2 FIB table of the default
 * bridge domain (bridge-domain-id of 0) to be modified.
 *
 * @cliexpar
 :
?*/

Example usage

The Example usage section is the text from the @cliexpar line to the closing ?*/. Here are some points of interest in this section:

• Required: @cliexpar prints the heading and causes the remaining text to be indented.
• Optional: @parblock and @endparblock allow the section to have blank lines. Without this, a blank line will terminate the Example usage section.
  • If the blank line is following a @cliexcmd{} or @cliexend, the blank line is not honoured in the generated document but only helps in the readability of the comment itself.
  • Otherwise, multiple paragraphs can be added to this section using a blank line and the @parblock and @endparblock.
• Single line examples (like set commands) should be enclosed in @cliexcmd{<cmd>}. This macro prepends the #vpp and makes the text bold.
• Multiple line examples (like show commands) have following format:

@cliexstart{<cmd>}

show output

@cliexend

The @cliexstart{<cmd>} macro prepends the #vpp and makes the text bold.

• Don't have to give an example of every optional parameter, but if something isn't straight forward, give an extra example.
• Attempt to document tear down as well as creation. Sometimes deleting objects can be a bit tricky.

/*?
:
* @cliexpar
* @parblock
* Example of how to add a set of 4 sequential MAC Address entries to L2
* FIB table of the default bridge-domain:
* @cliexcmd{test l2fib add mac 52:54:00:53:00:00 count 4}
*                                                                     <== Sample Blank Line - NOT Displayed
* Show the set of 4 sequential MAC Address entries that were added:
* @cliexstart{show l2fib verbose}
*     Mac Address     BD Idx           Interface           Index  static  filter  bvi  refresh  timestamp
* 52:54:00:53:00:00    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
* 52:54:00:53:00:01    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
* 52:54:00:53:00:03    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
* 52:54:00:53:00:02    0       GigabitEthernet0/8/0.300     8       0       0     0      0         0
* 4 l2fib entries
* @cliexend
*
* Example of how to check that the set of 4 sequential MAC Address
* entries were added to L2 FIB table of the default
* bridge-domain. Used a count of 5 to produce an error:
*
* @cliexcmd{test l2fib check mac 52:54:00:53:00:00 count 5}
* The output of the check command is in the log files. Log file
* location may vary based on your OS and Version:
*                                                                     <== Sample Blank Line - Yes Displayed
* # tail -f /var/log/messages | grep l2fib_test_command_fn
*
* Sep  7 17:15:24 localhost vnet[4952]: l2fib_test_command_fn:446: key 52:54:00:53:00:04 AWOL
*
* Example of how to delete a set of 4 sequential MAC Address entries
* from L2 FIB table of the default bridge-domain:
* @cliexcmd{test l2fib del mac 52:54:00:53:00:00 count 4}
* @endparblock
?*/

Declaration and implementation

The Declaration and implementation section is derived from .function variable of the VLIB_CLI_COMMAND section.

VLIB_CLI_COMMAND (l2fib_test_command, static) = {
  .path = "test l2fib",
  .short_help = "test l2fib [add|del|check] mac <base-addr> count <nn>",
  .function = l2fib_test_command_fn,
};

Build tools

All documentation generation related dependencies are installed with the install-dep target of the top level Makefile.

Building the documentation

The top level Makefile of the VPP build tree also includes a doxygen build target that will run Doxygen over the complete VPP source tree. Invoking that target will produce a significant amount of output and would look something like this:

$ make doxygen
Adding custom extension mapping: .def will be treated as language c
Adding custom extension mapping: .api will be treated as language c
Parsing layout file /home/chrisy/dev/fdio/vpp/doxygen/layout.xml...
Searching for include files...
Searching for files in directory /home/chrisy/dev/fdio/vpp/vppinfra
Searching for files in directory /home/chrisy/dev/fdio/vpp/vppinfra/autom4te.cache
Searching for files in directory /home/chrisy/dev/fdio/vpp/vppinfra/config
...
Patching output file 10218/10219
Patching output file 10219/10219
lookup cache used 25684/65536 hits=502842 misses=26575
finished...

Previewing the generated documentation

By default only HTML documentation is produced and it is built in build-root/docs/html. If you build the documentation on a workstation then you can simply open index.html in that directory with a web browser to view it, for example:

$ make doxygen
...
$ sensible-browser build-root/docs/html/index.html

Alternatively if you generate your documentation on a remote machine then an approach might be to add an HTTP server (such as Nginx or Apache HTTPD) to that host and configure it to expose the build-root/docs/html directory; one mechanism is to symbolically link to this directory from inside some other directory already visible from the HTTP server, assuming the use of symbolic links is allowed. Details on this are beyond the scope of this page.

Building documentation for a specific directory or file

Producing the documentation for the entire build tree can take some time which can become an issue if a developer is working on documentation for a specific module or directory tree. As a convenience it is possible to override the set of directories into which Doxygen will descend when searching for files to parse. For example:

$ make doxygen DOXY_INPUT=vppinfra

or, if you wanted just one file:

$ make doxygen DOXY_INPUT=vppinfra/vppinfra/pool.h

You can also indicate several locations to look in by enclosing a list or directories and/or files in quotes:

$ make doxygen DOXY_INPUT="vlib/vlib/unix README.md"

Note that specifying wildcards here won't work unless the shell expands them first.

Cleaning up after Doxygen

It is also possible to clean up the Doxygen output directory. This can become useful since Doxygen does not itself remove files from the build directory on subsequent runs should they become redundant.

$ make wipe-doxygen

This is especially worth noting should you have previously built the documentation for the whole source tree and then want to build it for only one directory or file (using DOXY_INPUT); whilst unlikely, the remnants of previous documentation may interfere with subsequent Doxygen runs for a narrower scope.