Difference between revisions of "VPP/Command-line Arguments"

From fd.io
< VPP
Jump to: navigation, search
m (break long lines)
 
(18 intermediate revisions by 8 users not shown)
Line 1: Line 1:
=== Introduction  ===
+
== Introduction  ==
  
The VPP network stack comes with several configuration options that can be provided either on the command line or in a configuration file.  
+
The VPP network stack comes with several configuration options that can be provided and applications built on the VPP stack have been known to require a dozen arguments, depending on requirements. This page describes commonly used configuration options. Please help maintain and improve this document!
  
Specific applications built on the stack have been known to require a dozen arguments, depending on requirements. This page describes commonly-used options and parameters.
+
The configuration options can be provided to VPP either as [[VPP/Command-line_Arguments#Command-line_arguments|command-line arguments]] or in a [[VPP/Command-line_Arguments#Configuration_file|configuration file]].
  
You can find command-line argument parsers in the source code by searching for instances of the <code>VLIB_CONFIG_FUNCTION</code> macro. The invocation <code>VLIB_CONFIG_FUNCTION (foo_config, "foo")</code> will cause the function <code>foo_config</code> to receive all parameters given in a parameter block named "<code>foo</code>", for example: "<code>foo { arg1 arg2 arg3 ... }</code>".
+
== Command-line arguments  ==
 
+
Please help maintain and improve this document!
+
 
+
=== Command-line arguments  ===
+
  
 
Parameters are grouped by a section name. When providing more than one parameter to a section all parameters for that section must be wrapped in curly braces.
 
Parameters are grouped by a section name. When providing more than one parameter to a section all parameters for that section must be wrapped in curly braces.
Line 22: Line 18:
 
  vpp# &lt;start-typing&gt;
 
  vpp# &lt;start-typing&gt;
  
When providing only one such parameter the braces are optional. For example, the following command argument, <code>unix interactive</code> does not have braces:  
+
When providing only one such parameter the braces are optional. For example, the following command argument, <code>unix interactive</code> does not need braces:  
  
 
  # /usr/bin/vpp unix interactive
 
  # /usr/bin/vpp unix interactive
&lt;startup diagnostic messages&gt;
 
<span style="color:red">    _______    _    </span>  _  _____  ___
 
<span style="color:red"> __/ __/ _ \  (_)__  </span>  | | / / _ \/ _ \
 
<span style="color:red"> _/ _// // / / / _ \ </span>  | |/ / ___/ ___/
 
<span style="color:red"> /_/ /____(_)_/\___/ </span>  |___/_/  /_/   
 
 
vpp# &lt;start-typing&gt;
 
  
 
The command line can be presented as a single string or as several; anything given on the command line is concatenated with spaces into a single string before parsing.
 
The command line can be presented as a single string or as several; anything given on the command line is concatenated with spaces into a single string before parsing.
Line 39: Line 28:
 
When developing with VPP, in gdb it's often sufficient to start an application like this:  
 
When developing with VPP, in gdb it's often sufficient to start an application like this:  
  
  (gdb) run unix interactive
+
  # (gdb) run unix interactive
  
=== Configuration file ===
+
== Configuration file ==
  
It is also possible to supply parameters in a startup configuration file the path of which is provided to the VPP application on its command line.
+
It is also possible to supply parameters in a startup configuration file the path of which is provided to the VPP application on its command line. The format of the configuration file is a simple text file with the same content as the command line but with the benefit of being able to use newlines to make the content easier to read.  
  
The format of the configuration file is a simple text file with the same content as the command line but with the benefit of being able to use newlines to make the content easier to read. For example:
+
The configuration file is commonly stored at <code>/etc/vpp/startup.conf</code>.
  
<code>
+
Here is an example of the configuration file:
 +
<code>  
 
  unix {
 
  unix {
 
   nodaemon
 
   nodaemon
   log /tmp/vpp.log
+
   log /var/log/vpp/vpp.log
 
   full-coredump
 
   full-coredump
 
   cli-listen localhost:5002
 
   cli-listen localhost:5002
Line 66: Line 56:
 
VPP is then instructed to load this file with the <code>-c</code> option:
 
VPP is then instructed to load this file with the <code>-c</code> option:
  
  /usr/bin/vpp -c /etc/vpp/startup.conf
+
  # /usr/bin/vpp -c /etc/vpp/startup.conf
  
=== Configuration parameters ===
+
'''''Note''''': In VPP 18.04, the default log file location was moved from "<code>/tmp/vpp.log</code>" to "<code>/var/log/vpp/vpp.log</code>".  The VPP code is indifferent to the file location. However, if SELinux is enabled, then the new location is required for the file to be properly labelled. Check your local <code>startup.conf</code> file for the log file location on your system.
 +
 
 +
== Configuration parameters ==
  
 
This is not an exhaustive list of parameters available. Readers who find information is missing or inaccurate are encouraged to add to this wiki page!
 
This is not an exhaustive list of parameters available. Readers who find information is missing or inaccurate are encouraged to add to this wiki page!
  
 +
You can find argument parsers in the source code by searching for instances of the <code>VLIB_CONFIG_FUNCTION</code> or <code>VLIB_EARLY_CONFIG_FUNCTION</code> macros. Their invocation <code>(foo_config, "foo")</code> will cause the function <code>foo_config</code> to receive all parameters given in a parameter block named "<code>foo</code>", for example: "<code>foo { arg1 arg2 arg3 ... }</code>".
  
==== "unix" parameters  ====
+
=== <code>unix { ... }</code> ===
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>interactive</code>
 
| <code>interactive</code>
 
| Attach CLI to stdin/out and provide a debugging command line interface. Implies <code>nodaemon</code>.
 
| Attach CLI to stdin/out and provide a debugging command line interface. Implies <code>nodaemon</code>.
 
|-
 
|-
 
| <code>nodaemon</code>
 
| <code>nodaemon</code>
| Do not fork / background the vpp process. Typical when invoking VPP applications from a process monitor.  
+
| Do not fork / background the vpp process. Typical when invoking VPP applications from a process monitor. Set by default in the default <code>startup.conf</code> file.
 
|-
 
|-
 
| <code>exec &lt;filename&gt;</code>
 
| <code>exec &lt;filename&gt;</code>
 
<code>startup-config &lt;filename&gt;</code>
 
<code>startup-config &lt;filename&gt;</code>
 
| Read startup operational configuration from <code>filename</code>. The contents of the file will be performed as though entered at the CLI.
 
| Read startup operational configuration from <code>filename</code>. The contents of the file will be performed as though entered at the CLI.
The two keywords are aliases for the same function; if both are specified, the last will take precedence.
+
The two keywords are aliases for the same function; if both are specified, only the last will have an effect.
 
|-
 
|-
 
| <code>log &lt;filename&gt;</code>
 
| <code>log &lt;filename&gt;</code>
Line 94: Line 90:
 
| Ask the Linux kernel to dump all memory-mapped address regions, instead of just text+data+bss.
 
| Ask the Linux kernel to dump all memory-mapped address regions, instead of just text+data+bss.
 
|-
 
|-
| <code>cli-listen localhost:5002</code>
+
| <code>cli-listen <addr:port> <nowiki>|</nowiki> <filename></code>
| Bind the CLI to listen at address <code>localhost</code> on TCP port <code>5002</code>.
+
| Bind the CLI to listen at address <code>addr</code> on TCP port <code>port</code>.
 +
This will accept an <code>ipaddress:port</code> pair or a filesystem path; in the latter case
 +
a local unix socket is opened instead. The default 'startup.conf' file is to open the socket <code>/run/vpp/cli.sock</code>.
 
|-
 
|-
 
| <code>cli-line-mode</code>
 
| <code>cli-line-mode</code>
Line 116: Line 114:
 
|}
 
|}
  
==== "tuntap" parameters  ====
+
=== <code>tuntap { ... }</code> ===
  
 
The "tuntap" driver configures a point-to-point interface between the vpp engine and the local Linux kernel stack. This allows e.g. users to ssh to the host / VM / container via vpp "revenue" interfaces. It's marginally useful, and is currently disabled by default. To [dynamically] create TAP interfaces - the preferred scheme - see the "tap_connect" binary API  
 
The "tuntap" driver configures a point-to-point interface between the vpp engine and the local Linux kernel stack. This allows e.g. users to ssh to the host / VM / container via vpp "revenue" interfaces. It's marginally useful, and is currently disabled by default. To [dynamically] create TAP interfaces - the preferred scheme - see the "tap_connect" binary API  
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>tuntap disable</code>
 
| <code>tuntap disable</code>
 
| Disable the tun/tap driver if desired  
 
| Disable the tun/tap driver if desired  
Line 139: Line 140:
 
   tuntap { ethernet have-normal-interface name vpp1 }
 
   tuntap { ethernet have-normal-interface name vpp1 }
  
==== "api-trace" parameters  ====
+
=== <code>api-trace { ... }</code> ===
  
 
The ability to trace, dump, and replay control-plane API traces makes all the difference in the world when trying to understand what the control-plane has tried to ask the forwarding-plane to do.  
 
The ability to trace, dump, and replay control-plane API traces makes all the difference in the world when trying to understand what the control-plane has tried to ask the forwarding-plane to do.  
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>on</code>
 
| <code>on</code>
 +
<code>enable</code>
 
| Enable API trace capture from the beginning of time, and arrange for a post-mortem dump of the API trace if the application terminates abnormally.
 
| Enable API trace capture from the beginning of time, and arrange for a post-mortem dump of the API trace if the application terminates abnormally.
By default, the (circular) trace buffer will be configured to capture 256K traces
+
The two keywords are aliases for the same thing.
 
|-
 
|-
 
| <code>nitems &lt;nnn&gt;</code>
 
| <code>nitems &lt;nnn&gt;</code>
 
| Configure the circular trace buffer to contain the last &lt;nnn&gt; entries. By default, the trace buffer captures the last 256K API messages received.
 
| Configure the circular trace buffer to contain the last &lt;nnn&gt; entries. By default, the trace buffer captures the last 256K API messages received.
 +
|-
 +
| <code>save-api-table &lt;filename&gt;</code>
 +
| To save the api message table decode tables, configure a filename. Results in /tmp/<code><filename></code>
 +
Very handy for understanding api message changes between versions, identifying missing
 +
plugins, and so forth.
 
|}
 
|}
  
Line 156: Line 166:
 
  api-trace { on }
 
  api-trace { on }
  
==== "cpu" parameters  ====
+
=== <code>api-segment { ... }</code> ===
 +
 
 +
These values control various aspects of the binary API interface to VPP.
 +
 
 +
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 +
| <code>prefix &lt;path&gt;</code>
 +
| Sets the prefix prepended to the name used for shared memory (SHM) segments.
 +
The default is empty, meaning shared memory segments are created directly in the SHM directory <code>/dev/shm</code>.
 +
 
 +
It is worth noting that on many systems <code>/dev/shm</code> is a symbolic link to somewhere else in the file system;
 +
 
 +
Ubuntu links it to <code>/run/shm</code>.
 +
|-
 +
| <code>uid &lt;number<nowiki>|</nowiki>name&gt;</code>
 +
| Sets the user ID or name that should be used to set the ownership of the shared memory segments.
 +
Defaults to the same user that VPP is started with, probably <code>root</code>.
 +
|-
 +
| <code>gid &lt;number<nowiki>|</nowiki>name&gt;</code>
 +
| Sets the group ID or name that should be used to set the ownership of the shared memory segments.
 +
Defaults to the same group that VPP is started with, probably <code>root</code>.
 +
|}
 +
 
 +
=== <code>cpu { ... }</code> ===
  
 
Command-line CPU configuration controls the creation of named thread types, and the cpu affinity thereof. As of this writing, the cpu configuration function supports these parameters:  
 
Command-line CPU configuration controls the creation of named thread types, and the cpu affinity thereof. As of this writing, the cpu configuration function supports these parameters:  
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>workers &lt;nn&gt;</code>
 
| <code>workers &lt;nn&gt;</code>
 
| Create &lt;nn&gt; worker threads.
 
| Create &lt;nn&gt; worker threads.
Line 177: Line 215:
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>main-core &lt;n&gt;</code>
 
| <code>main-core &lt;n&gt;</code>
 
| Assign main thread to specific core.
 
| Assign main thread to specific core.
Line 206: Line 247:
 
  pid 16291's current affinity mask: 10 # worker thread 1
 
  pid 16291's current affinity mask: 10 # worker thread 1
  
==== "dpdk" parameters ====
+
=== <code>dpdk  { ... }</code> ===
  
 
Command line DPDK configuration controls a number of parameters, including device whitelisting, the number of CPUs available for launching dpdk-eal-controlled threads, the number of I/O buffers, and the process affinity mask.  
 
Command line DPDK configuration controls a number of parameters, including device whitelisting, the number of CPUs available for launching dpdk-eal-controlled threads, the number of I/O buffers, and the process affinity mask.  
Line 215: Line 256:
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>coremask &lt;hex&gt;</code>
 
| <code>coremask &lt;hex&gt;</code>
 
| Sets the process-level coremask. See [[VPP/Command-line_Arguments#.22cpu.22_parameters|previous section on cpu parameters]] for more details.  
 
| Sets the process-level coremask. See [[VPP/Command-line_Arguments#.22cpu.22_parameters|previous section on cpu parameters]] for more details.  
Line 228: Line 272:
 
* <code>F</code> = Function
 
* <code>F</code> = Function
 
This is the same format used in the linux sysfs tree (i.e. <code>/sys/bus/pci/devices</code>) for PCI device directory names.  
 
This is the same format used in the linux sysfs tree (i.e. <code>/sys/bus/pci/devices</code>) for PCI device directory names.  
 +
 +
VLAN stripping for each PCI device on the white-list can be specified using [{vlan-strip-offload on|off}]. VLAN stripping is off by default for all NICs except VICs, using ENIC driver, which has VLAN stripping on by default. To disable VLAN stripping for a VIC vNIC on PCI address 0000:0c:00.0, for example, specify "dev 0000:0c:00.0 {vlan-strip-offload off}" on the white-list.
 
|-
 
|-
 
| <code>vdev &lt;eal-command&gt;</code>
 
| <code>vdev &lt;eal-command&gt;</code>
| Provide a DPDK EAL command topecify bonded Ethernet interfaces, operating modes and PCI address of slave links. For example:
+
| Provide a DPDK EAL command to specify bonded Ethernet interfaces, operating modes and PCI addresses of slave links. For example:
 
  vdev eth_bond0,mode=2,slave=0000:0f:00.0,slave=0000:11:00.0,xmit_policy=l34
 
  vdev eth_bond0,mode=2,slave=0000:0f:00.0,slave=0000:11:00.0,xmit_policy=l34
 
  vdev eth_bond1,mode=2,slave=0000:10:00.0,slave=0000:12:00.0,xmit_policy=l34
 
  vdev eth_bond1,mode=2,slave=0000:10:00.0,slave=0000:12:00.0,xmit_policy=l34
Line 236: Line 282:
 
|-
 
|-
 
| <code>num-mbufs &lt;nn&gt;</code>
 
| <code>num-mbufs &lt;nn&gt;</code>
| Number of I/O buffers. You need to increase this if you have a lot of physical interfaces with multiple RSS queues (for example, <code>131072</code>).
+
| Number of I/O buffers. The default is <code>16384</code>.
 +
You need to increase this if you have a lot of physical interfaces with multiple RSS queues.
 
|-
 
|-
 
| <code>kni &lt;nn&gt;</code>
 
| <code>kni &lt;nn&gt;</code>
Line 247: Line 294:
 
All of the DPDK EAL options should be available. See <code>.../open-repo/vnet/vnet/devices/dpdk/dpdk.h</code>, look at the set of "<code>foreach_eal_XXX</code>" macros.
 
All of the DPDK EAL options should be available. See <code>.../open-repo/vnet/vnet/devices/dpdk/dpdk.h</code>, look at the set of "<code>foreach_eal_XXX</code>" macros.
  
==== "l2tp" parameters  ====
+
=== <code>l2tp { ... }</code> ===
  
 
ip6-l2tpv3 configuration controls the method used to locate a specific ip6-l2tpv3 tunnel. The following settings are mutually exclusive  
 
ip6-l2tpv3 configuration controls the method used to locate a specific ip6-l2tpv3 tunnel. The following settings are mutually exclusive  
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>lookup-v6-src</code>
 
| <code>lookup-v6-src</code>
 
| Lookup tunnel by ip6 source address.
 
| Lookup tunnel by ip6 source address.
Line 264: Line 314:
 
Refer to <code>.../open-repo/vnet/vnet/devices/dpdk/{dpdk.h, init.c}</code> for more details.
 
Refer to <code>.../open-repo/vnet/vnet/devices/dpdk/{dpdk.h, init.c}</code> for more details.
  
==== "oam" parameters  ====
+
=== <code>oam { ... }</code> ===
  
 
Oam configuration controls the (ip4-icmp) interval, and number of misses allowed before reporting an oam target down to any registered listener.
 
Oam configuration controls the (ip4-icmp) interval, and number of misses allowed before reporting an oam target down to any registered listener.
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>interval &lt;floating-point-seconds&gt;</code>
 
| <code>interval &lt;floating-point-seconds&gt;</code>
 
| Interval between sending oam ip4 icmp messages.
 
| Interval between sending oam ip4 icmp messages.
Line 276: Line 329:
 
|}
 
|}
  
==== "heapsize" parameter  ====
+
=== <code>heapsize { ... }</code> ===
  
 
Heapsize configuration controls the size of the main heap. The heap size is configured very early in the boot sequence, before loading plug-ins or doing much of anything else.  
 
Heapsize configuration controls the size of the main heap. The heap size is configured very early in the boot sequence, before loading plug-ins or doing much of anything else.  
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>heapsize &lt;nn&gt;M <nowiki>|</nowiki> &lt;nn&gt;G</code>
 
| <code>heapsize &lt;nn&gt;M <nowiki>|</nowiki> &lt;nn&gt;G</code>
 
| Specifies the size of the heap in mb or gb.
 
| Specifies the size of the heap in mb or gb.
Line 286: Line 342:
 
|}
 
|}
  
==== "plugin_path" parameter ====
+
=== <code>plugins { path ... }</code> ===
  
Plugin_path configuration controls the set of directories searched for vlib plugins. Supply a colon-separated list of (absolute) directory names:  
+
Plugin path configuration controls the set of directories searched for vlib plugins. Supply a colon-separated list of (absolute) directory names:  
  
 
{| class="wikitable"
 
{| class="wikitable"
| <code>plugin_path dir1:dir2:...:dirN</code>
+
! Parameter
 +
! Description
 +
|-
 +
| <code>plugins { path dir1:dir2:...:dirN } </code>
 
| Configure the search path for plugins.
 
| Configure the search path for plugins.
 
|}
 
|}
  
==== "cj" parameters  ====
+
=== <code>cj { ... }</code> ===
  
 
The cj thread-safe circular log buffer scheme is occasionally useful when chasing bugs. Calls to it should not be checked in. See <code>.../vlib/vlib/unix/cj.c</code>.
 
The cj thread-safe circular log buffer scheme is occasionally useful when chasing bugs. Calls to it should not be checked in. See <code>.../vlib/vlib/unix/cj.c</code>.
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>records &lt;nn&gt;</code>
 
| <code>records &lt;nn&gt;</code>
 
| Configure the number of cj records in the circular buffer.
 
| Configure the number of cj records in the circular buffer.
Line 307: Line 369:
 
|}
 
|}
  
==== "vhost-user" parameters  ====
+
=== <code>vhost-user { ... }</code> ===
  
 
Vhost-user configuration parameters control the vhost-user driver.  
 
Vhost-user configuration parameters control the vhost-user driver.  
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>coalesce-frames &lt;nn&gt;</code>
 
| <code>coalesce-frames &lt;nn&gt;</code>
 
| Subject to deadline-timer expiration - see next item - attempt to transmit at least <code>nn</code> packet frames.  
 
| Subject to deadline-timer expiration - see next item - attempt to transmit at least <code>nn</code> packet frames.  
Line 322: Line 387:
 
|}
 
|}
  
==== "l2learn" parameter  ====
+
=== <code>l2learn { ... }</code> ===
  
 
Configures the number of L2 (MAC) addresses in the L2 FIB at any one time. Defaults to 1M entries.  
 
Configures the number of L2 (MAC) addresses in the L2 FIB at any one time. Defaults to 1M entries.  
  
 
{| class="wikitable"
 
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 
| <code>limit &lt;nn&gt;</code>
 
| <code>limit &lt;nn&gt;</code>
 
| Limits the size of the L2 FIB to <code>nn</code> concurrent entries.
 
| Limits the size of the L2 FIB to <code>nn</code> concurrent entries.
 +
|}
 +
 +
=== <code>socksvr { ... }</code> ===
 +
 +
Configure the socket server for the binary API. (See <code>.../vlibmemory/socket_api.c</code>)
 +
 +
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 +
| <code>socket-name &lt;filename&gt;</code>
 +
| Configure API socket filename
 +
|-
 +
| <code>default</code>
 +
| Default API filename: <code>/run/vpp-api.sock</code>
 +
|}
 +
 +
=== <code>statseg { ... }</code> ===
 +
 +
Configure the shared memory stats segment. (See <code>.../vpp/stats/stat_segment.c</code>)
 +
 +
{| class="wikitable"
 +
! Parameter
 +
! Default
 +
! Description
 +
|-
 +
| <code>socket-name &lt;filename&gt;</code>
 +
| <code>/run/vpp/stats.sock</code>
 +
| Configure the stats segment socket filename.
 +
|-
 +
| <code>default</code>|| <code>-</code>|| Sets the default value for <code>socket-name</code> parameter.
 +
|-
 +
| <code>per-node-counters <on<nowiki>|</nowiki>off></code>
 +
| <code>off</code>
 +
| Enable per node performance counters (<code>/sys/node</code>).
 +
|-
 +
| <code><nowiki>size <n>G | <n>M | <n>K | <n></nowiki></code>
 +
| <code>33554432 (roughly 32MB)</code>
 +
| Sets the size of the memory mapped stats segment object stat_segment. The input value can be set in GB, MB, KB or bytes.
 +
|}
 +
 +
=== <code>punt { ... }</code> ===
 +
 +
Configure the punt socket for the binary API. (See <code>.../vnet/ip/punt.c</code>)
 +
 +
{| class="wikitable"
 +
! Parameter
 +
! Description
 +
|-
 +
| <code>socket &lt;filename&gt;</code>
 +
| Configure socket filename
 
|}
 
|}

Latest revision as of 12:27, 7 June 2019

Introduction

The VPP network stack comes with several configuration options that can be provided and applications built on the VPP stack have been known to require a dozen arguments, depending on requirements. This page describes commonly used configuration options. Please help maintain and improve this document!

The configuration options can be provided to VPP either as command-line arguments or in a configuration file.

Command-line arguments

Parameters are grouped by a section name. When providing more than one parameter to a section all parameters for that section must be wrapped in curly braces.

# /usr/bin/vpp unix { interactive cli-listen 127.0.0.1:5002 }
<startup diagnostic messages>
    _______    _        _   _____  ___ 
 __/ __/ _ \  (_)__    | | / / _ \/ _ \
 _/ _// // / / / _ \   | |/ / ___/ ___/
 /_/ /____(_)_/\___/   |___/_/  /_/    

vpp# <start-typing>

When providing only one such parameter the braces are optional. For example, the following command argument, unix interactive does not need braces:

# /usr/bin/vpp unix interactive

The command line can be presented as a single string or as several; anything given on the command line is concatenated with spaces into a single string before parsing.

VPP applications must be able to locate their own executable images. The simplest way to ensure this will work is to invoke a VPP application by giving its absolute path; for example: /usr/bin/vpp <options>. At startup, VPP applications parse through their own ELF-sections [primarily] to make lists of init, configuration, and exit handlers.

When developing with VPP, in gdb it's often sufficient to start an application like this:

# (gdb) run unix interactive

Configuration file

It is also possible to supply parameters in a startup configuration file the path of which is provided to the VPP application on its command line. The format of the configuration file is a simple text file with the same content as the command line but with the benefit of being able to use newlines to make the content easier to read.

The configuration file is commonly stored at /etc/vpp/startup.conf.

Here is an example of the configuration file:

unix {
  nodaemon
  log /var/log/vpp/vpp.log
  full-coredump
  cli-listen localhost:5002
}

api-trace {
  on
}

dpdk {
  dev 0000:03:00.0
}

VPP is then instructed to load this file with the -c option:

# /usr/bin/vpp -c /etc/vpp/startup.conf

Note: In VPP 18.04, the default log file location was moved from "/tmp/vpp.log" to "/var/log/vpp/vpp.log". The VPP code is indifferent to the file location. However, if SELinux is enabled, then the new location is required for the file to be properly labelled. Check your local startup.conf file for the log file location on your system.

Configuration parameters

This is not an exhaustive list of parameters available. Readers who find information is missing or inaccurate are encouraged to add to this wiki page!

You can find argument parsers in the source code by searching for instances of the VLIB_CONFIG_FUNCTION or VLIB_EARLY_CONFIG_FUNCTION macros. Their invocation (foo_config, "foo") will cause the function foo_config to receive all parameters given in a parameter block named "foo", for example: "foo { arg1 arg2 arg3 ... }".

unix { ... }

Parameter Description
interactive Attach CLI to stdin/out and provide a debugging command line interface. Implies nodaemon.
nodaemon Do not fork / background the vpp process. Typical when invoking VPP applications from a process monitor. Set by default in the default startup.conf file.
exec <filename>

startup-config <filename>

Read startup operational configuration from filename. The contents of the file will be performed as though entered at the CLI.

The two keywords are aliases for the same function; if both are specified, only the last will have an effect.

log <filename> Logs the startup configuration and all subsequent CLI commands in filename.

Very useful in situations where folks don't remember or can't be bothered to include CLI commands in bug reports.

full-coredump Ask the Linux kernel to dump all memory-mapped address regions, instead of just text+data+bss.
cli-listen <addr:port> | <filename> Bind the CLI to listen at address addr on TCP port port.

This will accept an ipaddress:port pair or a filesystem path; in the latter case a local unix socket is opened instead. The default 'startup.conf' file is to open the socket /run/vpp/cli.sock.

cli-line-mode Disable character-by-character I/O on stdin. Useful when combined with, for example, emacs M-x gud-gdb.
cli-prompt <string> Configure the CLI prompt to be string.
cli-history-limit <nn> Limit commmand history to nn lines. A value of 0 disables command history. Default value: 50
cli-no-banner Disable the login banner on stdin and Telnet connections.
cli-no-pager Disable the output pager.
cli-pager-buffer-limit <nn> Limit pager buffer to nn lines of output. A value of 0 disables the pager. Default value: 100000

tuntap { ... }

The "tuntap" driver configures a point-to-point interface between the vpp engine and the local Linux kernel stack. This allows e.g. users to ssh to the host / VM / container via vpp "revenue" interfaces. It's marginally useful, and is currently disabled by default. To [dynamically] create TAP interfaces - the preferred scheme - see the "tap_connect" binary API

Parameter Description
tuntap disable Disable the tun/tap driver if desired
tuntap ethernet Create a tap device (ethernet MAC) instead of a tun device (point-to-point tunnel)
tuntap have-normal-interface Treat the host Linux stack as a routing peer instead of programming vpp interface L3 addresses onto the tun/tap devices.

You'll need to manually configure the Linux network stack "vnet" interface, configure vlan subinterfaces if desired, etc.

tuntap name ishmael Call the tun/tap device ishmael

Here's a typical multiple parameter invocation:

 tuntap { ethernet have-normal-interface name vpp1 }

api-trace { ... }

The ability to trace, dump, and replay control-plane API traces makes all the difference in the world when trying to understand what the control-plane has tried to ask the forwarding-plane to do.

Parameter Description
on

enable

Enable API trace capture from the beginning of time, and arrange for a post-mortem dump of the API trace if the application terminates abnormally.

The two keywords are aliases for the same thing.

nitems <nnn> Configure the circular trace buffer to contain the last <nnn> entries. By default, the trace buffer captures the last 256K API messages received.
save-api-table <filename> To save the api message table decode tables, configure a filename. Results in /tmp/<filename>

Very handy for understanding api message changes between versions, identifying missing plugins, and so forth.

Typically, one simply enables the API message trace scheme:

api-trace { on }

api-segment { ... }

These values control various aspects of the binary API interface to VPP.

Parameter Description
prefix <path> Sets the prefix prepended to the name used for shared memory (SHM) segments.

The default is empty, meaning shared memory segments are created directly in the SHM directory /dev/shm.

It is worth noting that on many systems /dev/shm is a symbolic link to somewhere else in the file system;

Ubuntu links it to /run/shm.

uid <number|name> Sets the user ID or name that should be used to set the ownership of the shared memory segments.

Defaults to the same user that VPP is started with, probably root.

gid <number|name> Sets the group ID or name that should be used to set the ownership of the shared memory segments.

Defaults to the same group that VPP is started with, probably root.

cpu { ... }

Command-line CPU configuration controls the creation of named thread types, and the cpu affinity thereof. As of this writing, the cpu configuration function supports these parameters:

Parameter Description
workers <nn> Create <nn> worker threads.
io <nn> Create nn i/o threads.
main-thread-io Handle i/o devices from thread 0, hand off traffic to worker threads. Requires "workers <nn>".
skip-cores <nn> Leave the low nn bits of the process affinity mask clear.

In some cases, it may be necessary to place thread instances manually:

Parameter Description
main-core <n> Assign main thread to specific core.
coremask-workers <hex-mask> Place worker threads according to the bitmap hex-mask.
corelist-workers <list> Same as coremask-workers but accepts a list of cores instead of a bitmap.
coremask-io <hex-mask> Place I/O threads according to the bitmap hex-mask.
corelist-io <list> Same as coremask-io but accepts a list of cores instead of a bitmap.

Note that the "main" thread always occupies the lowest core-id specified in the DPDK [process-level] coremask.

Here's a full-bore manual placement example:

/cisco/bin/vpe  unix interactive tuntap disable cpu { main-thread-io coremask-workers 18 coremask-stats 4 } dpdk {coremask 1e}

# taskset -a -p <vpe-pid>
pid 16251's current affinity mask: 2   # main thread
pid 16288's current affinity mask: ffffff # DPDK interrupt thread (not bound to a core)
pid 16289's current affinity mask: 4 # stats thread
pid 16290's current affinity mask: 8 # worker thread 0
pid 16291's current affinity mask: 10 # worker thread 1

dpdk { ... }

Command line DPDK configuration controls a number of parameters, including device whitelisting, the number of CPUs available for launching dpdk-eal-controlled threads, the number of I/O buffers, and the process affinity mask.

In addition, the DPDK configuration function attempts to support all of the DPDK EAL configuration parameters.

Popular options include:

Parameter Description
coremask <hex> Sets the process-level coremask. See previous section on cpu parameters for more details.
no-hugetlb Don't use huge TLB pages. Potentially useful for running simulator images.
dev <pci-dev> White-list [as in, attempt to drive] a specific PCI device. PCI-dev is a string of the form "DDDD:BB:SS.F" where:
  • DDDD = Domain
  • BB = Bus Number
  • SS = Slot number
  • F = Function

This is the same format used in the linux sysfs tree (i.e. /sys/bus/pci/devices) for PCI device directory names.

VLAN stripping for each PCI device on the white-list can be specified using [{vlan-strip-offload on|off}]. VLAN stripping is off by default for all NICs except VICs, using ENIC driver, which has VLAN stripping on by default. To disable VLAN stripping for a VIC vNIC on PCI address 0000:0c:00.0, for example, specify "dev 0000:0c:00.0 {vlan-strip-offload off}" on the white-list.

vdev <eal-command> Provide a DPDK EAL command to specify bonded Ethernet interfaces, operating modes and PCI addresses of slave links. For example:
vdev eth_bond0,mode=2,slave=0000:0f:00.0,slave=0000:11:00.0,xmit_policy=l34
vdev eth_bond1,mode=2,slave=0000:10:00.0,slave=0000:12:00.0,xmit_policy=l34

Only XOR balanced (mode 2) mode is supported.

num-mbufs <nn> Number of I/O buffers. The default is 16384.

You need to increase this if you have a lot of physical interfaces with multiple RSS queues.

kni <nn> Number of KNI interfaces. Refer to the DPDK documentation
uio-driver <name> Defaults to "igb_uio" [the Ubuntu default]. The names "uio_pci_generic" or "vfio-pci" are other possible values.

All of the DPDK EAL options should be available. See .../open-repo/vnet/vnet/devices/dpdk/dpdk.h, look at the set of "foreach_eal_XXX" macros.

l2tp { ... }

ip6-l2tpv3 configuration controls the method used to locate a specific ip6-l2tpv3 tunnel. The following settings are mutually exclusive

Parameter Description
lookup-v6-src Lookup tunnel by ip6 source address.
lookup-v6-dst Lookup tunnel by ip6 destination address.
lookup-session-id Lookup tunnel by l2tpv3 session identifier.

Refer to .../open-repo/vnet/vnet/devices/dpdk/{dpdk.h, init.c} for more details.

oam { ... }

Oam configuration controls the (ip4-icmp) interval, and number of misses allowed before reporting an oam target down to any registered listener.

Parameter Description
interval <floating-point-seconds> Interval between sending oam ip4 icmp messages.
misses-allowed <nn> Number of misses before declaring an oam target down.

heapsize { ... }

Heapsize configuration controls the size of the main heap. The heap size is configured very early in the boot sequence, before loading plug-ins or doing much of anything else.

Parameter Description
heapsize <nn>M | <nn>G Specifies the size of the heap in mb or gb.

Setting the main heap size to 4gb or more requires recompilation of the entire system with CLIB_VEC64 > 0. See .../clib/clib/vec_bootstrap.h.

plugins { path ... }

Plugin path configuration controls the set of directories searched for vlib plugins. Supply a colon-separated list of (absolute) directory names:

Parameter Description
plugins { path dir1:dir2:...:dirN } Configure the search path for plugins.

cj { ... }

The cj thread-safe circular log buffer scheme is occasionally useful when chasing bugs. Calls to it should not be checked in. See .../vlib/vlib/unix/cj.c.

Parameter Description
records <nn> Configure the number of cj records in the circular buffer.
on Turns on logging at the earliest possible moment.

vhost-user { ... }

Vhost-user configuration parameters control the vhost-user driver.

Parameter Description
coalesce-frames <nn> Subject to deadline-timer expiration - see next item - attempt to transmit at least nn packet frames.
coalesce-time <seconds> Hold packets no longer than (floating-point) seconds before transmitting them.
dont-dump-memory vhost-user shared-memory segments can add up to a large amount of memory, so it's handy to avoid adding them to corefiles when using a significant number of such interfaces.

l2learn { ... }

Configures the number of L2 (MAC) addresses in the L2 FIB at any one time. Defaults to 1M entries.

Parameter Description
limit <nn> Limits the size of the L2 FIB to nn concurrent entries.

socksvr { ... }

Configure the socket server for the binary API. (See .../vlibmemory/socket_api.c)

Parameter Description
socket-name <filename> Configure API socket filename
default Default API filename: /run/vpp-api.sock

statseg { ... }

Configure the shared memory stats segment. (See .../vpp/stats/stat_segment.c)

Parameter Default Description
socket-name <filename> /run/vpp/stats.sock Configure the stats segment socket filename.
default - Sets the default value for socket-name parameter.
per-node-counters <on|off> off Enable per node performance counters (/sys/node).
size <n>G | <n>M | <n>K | <n> 33554432 (roughly 32MB) Sets the size of the memory mapped stats segment object stat_segment. The input value can be set in GB, MB, KB or bytes.

punt { ... }

Configure the punt socket for the binary API. (See .../vnet/ip/punt.c)

Parameter Description
socket <filename> Configure socket filename