Difference between revisions of "VPP/Command-line Arguments"
(experimentally tabularize this section) |
(tabularize the rest of the page) |
||
Line 66: | Line 66: | ||
The "tuntap" driver configures a point-to-point interface between the vpp engine and the local Linux kernel stack. This allows e.g. users to ssh to the host / VM / container via vpp "revenue" interfaces. It's marginally useful, and is currently disabled by default. To [dynamically] create TAP interfaces - the preferred scheme - see the "tap_connect" binary API | The "tuntap" driver configures a point-to-point interface between the vpp engine and the local Linux kernel stack. This allows e.g. users to ssh to the host / VM / container via vpp "revenue" interfaces. It's marginally useful, and is currently disabled by default. To [dynamically] create TAP interfaces - the preferred scheme - see the "tap_connect" binary API | ||
− | + | {| class="wikitable" | |
− | + | | <code>tuntap disable</code> | |
− | + | | Disable the tun/tap driver if desired | |
− | + | |- | |
+ | | <code>tuntap ethernet</code> | ||
+ | | Create a tap device (ethernet MAC) instead of a tun device (point-to-point tunnel) | ||
+ | |- | ||
+ | | <code>tuntap have-normal-interface</code> | ||
+ | | Treat the host Linux stack as a routing peer instead of programming vpp interface L3 addresses onto the tun/tap devices. You'll need to manually configure the Linux network stack "vnet" interface, configure vlan subinterfaces if desired, etc. | ||
+ | |- | ||
+ | | <code>tuntap name ishmael</code> | ||
+ | | Call the tun/tap device ishmael | ||
+ | |} | ||
Here's a typical multiple parameter invocation: | Here's a typical multiple parameter invocation: | ||
Line 79: | Line 88: | ||
The ability to trace, dump, and replay control-plane API traces makes all the difference in the world when trying to understand what the control-plane has tried to ask the forwarding-plane to do. | The ability to trace, dump, and replay control-plane API traces makes all the difference in the world when trying to understand what the control-plane has tried to ask the forwarding-plane to do. | ||
− | + | {| class="wikitable" | |
− | + | | <code>on</code> | |
− | + | | Enable API trace capture from the beginning of time, and arrange for a post-mortem dump of the API trace if the application terminates abnormally. By default, the (circular) trace buffer will be configured to capture 256K traces | |
+ | |- | ||
+ | | <code>nitems <nnn></code> | ||
+ | | Configure the circular trace buffer to contain the last <nnn> entries. By default, the trace buffer captures the last 256K API messages received. | ||
+ | |} | ||
Typically, one simply enables the API message trace scheme: | Typically, one simply enables the API message trace scheme: | ||
Line 91: | Line 104: | ||
Command-line CPU configuration controls the creation of named thread types, and the cpu affinity thereof. As of this writing, the cpu configuration function supports these parameters: | Command-line CPU configuration controls the creation of named thread types, and the cpu affinity thereof. As of this writing, the cpu configuration function supports these parameters: | ||
− | + | {| class="wikitable" | |
− | + | | <code>workers <nn></code> | |
− | + | | Create <nn> worker threads. | |
− | + | |- | |
+ | | <code>io <nn></code> | ||
+ | | Create <code>nn</code> i/o threads. | ||
+ | |- | ||
+ | | <code>main-thread-io</code> | ||
+ | | Handle i/o devices from thread 0, hand off traffic to worker threads. Requires "workers <nn>". | ||
+ | |- | ||
+ | | <code>skip-cores <nn></code> | ||
+ | | Leave the low <code>nn</code> bits of the process affinity mask clear. | ||
+ | |} | ||
+ | |||
In some cases, it may be necessary to place thread instances manually: | In some cases, it may be necessary to place thread instances manually: | ||
− | + | {| class="wikitable" | |
− | + | | <code>main-core <n></code> | |
− | + | | Assign main thread to specific core. | |
− | + | |- | |
− | + | | <code>coremask-workers <hex-mask></code> | |
+ | | Place worker threads according to the bitmap <code>hex-mask</code>. | ||
+ | |- | ||
+ | | <code>corelist-workers <list></code> | ||
+ | | Same as <code>coremask-workers</code> but accepts a list of cores instead of a bitmap. | ||
+ | |- | ||
+ | | <code>coremask-io <hex-mask></code> | ||
+ | | Place I/O threads according to the bitmap <code>hex-mask</code>. | ||
+ | |- | ||
+ | | <code>corelist-io <list></code> | ||
+ | | Same as <code>coremask-io</code> but accepts a list of cores instead of a bitmap. | ||
+ | |} | ||
− | Note that the "main" thread always occupies the lowest core-id specified in the | + | Note that the "main" thread always occupies the lowest core-id specified in the DPDK [process-level] coremask. |
Here's a full-bore manual placement example: | Here's a full-bore manual placement example: | ||
Line 120: | Line 154: | ||
Command line DPDK configuration controls a number of parameters, including device whitelisting, the number of CPUs available for launching dpdk-eal-controlled threads, the number of I/O buffers, and the process affinity mask. | Command line DPDK configuration controls a number of parameters, including device whitelisting, the number of CPUs available for launching dpdk-eal-controlled threads, the number of I/O buffers, and the process affinity mask. | ||
− | In addition, the | + | In addition, the DPDK configuration function attempts to support all of the DPDK EAL configuration parameters. |
Popular options include: | Popular options include: | ||
− | + | {| class="wikitable" | |
− | + | | <code>coremask <hex></code> | |
− | + | | Sets the process-level coremask. See [[VPP/Command-line_Arguments#.22cpu.22_parameters|previous section on cpu parameters]] for more details. | |
− | + | |- | |
− | + | | <code>no-hugetlb</code> | |
− | + | | Don't use huge TLB pages. Potentially useful for running simulator images. | |
− | + | |- | |
+ | | <code>dev <pci-dev></code> | ||
+ | | White-list [as in, attempt to drive] a specific PCI device. PCI-dev is a string of the form "<code>DDDD:BB:SS.F</code>" where <code>DDDD</code> = Domain, <code>BB</code> = Bus Number, <code>SS</code> = Slot number, and <code>F</code> = Function. This is the same format used in the linux sysfs tree (i.e. <code>/sys/bus/pci/devices</code>) for PCI device directory names. | ||
+ | |- | ||
+ | | <code>vdev <DPDK EAL command to create link bonding interfaces></code> | ||
+ | | Specify bonded Ethernet interfaces, operating modes and PCI address of slave links. For example, "<code>vdev eth_bond0,mode=2,slave=0000:0f:00.0,slave=0000:11:00.0,xmit_policy=l34 vdev eth_bond1,mode=2,slave=0000:10:00.0,slave=0000:12:00.0,xmit_policy=l34</code>". Only XOR balanced (mode 2) mode is supported. | ||
+ | |- | ||
+ | | <code>num-mbufs <nn></code> | ||
+ | | Number of I/O buffers. You need to increase this if you have a lot of physical interfaces with multiple RSS queues (for example, <code>131072</code>). | ||
+ | |- | ||
+ | | <code>kni <nn></code> | ||
+ | | Number of KNI interfaces. Refer to the DPDK documentation | ||
+ | |- | ||
+ | | <code>uio-driver <name></code> | ||
+ | | Defaults to "<code>igb_uio</code>" [the Ubuntu default]. The names "<code>uio_pci_generic</code>" or "<code>vfio-pci</code>" are other possible values. | ||
+ | |} | ||
− | All of the DPDK EAL options should be available. See .../open-repo/vnet/vnet/devices/dpdk/dpdk.h, look at the set of "foreach_eal_XXX" macros. | + | All of the DPDK EAL options should be available. See <code>.../open-repo/vnet/vnet/devices/dpdk/dpdk.h</code>, look at the set of "<code>foreach_eal_XXX</code>" macros. |
==== "l2tp" parameters ==== | ==== "l2tp" parameters ==== | ||
Line 138: | Line 187: | ||
ip6-l2tpv3 configuration controls the method used to locate a specific ip6-l2tpv3 tunnel. The following settings are mutually exclusive | ip6-l2tpv3 configuration controls the method used to locate a specific ip6-l2tpv3 tunnel. The following settings are mutually exclusive | ||
− | + | {| class="wikitable" | |
− | + | | <code>lookup-v6-src</code> | |
− | + | | Lookup tunnel by ip6 source address. | |
+ | |- | ||
+ | | <code>lookup-v6-dst</code> | ||
+ | | Lookup tunnel by ip6 destination address. | ||
+ | |- | ||
+ | | <code>lookup-session-id</code> | ||
+ | | Lookup tunnel by l2tpv3 session identifier. | ||
+ | |} | ||
− | Refer to .../open-repo/vnet/vnet/devices/dpdk/{dpdk.h, init.c} for more details | + | Refer to <code>.../open-repo/vnet/vnet/devices/dpdk/{dpdk.h, init.c}</code> for more details. |
==== "oam" parameters ==== | ==== "oam" parameters ==== | ||
− | Oam configuration controls the (ip4-icmp) interval, and number of misses allowed before reporting an oam target down to any registered listener | + | Oam configuration controls the (ip4-icmp) interval, and number of misses allowed before reporting an oam target down to any registered listener. |
− | + | {| class="wikitable" | |
− | + | | <code>interval <floating-point-seconds></code> | |
+ | | Interval between sending oam ip4 icmp messages. | ||
+ | |- | ||
+ | | <code>misses-allowed <nn></code> | ||
+ | | Number of misses before declaring an oam target down. | ||
+ | |} | ||
==== "heapsize" parameter ==== | ==== "heapsize" parameter ==== | ||
Line 155: | Line 216: | ||
Heapsize configuration controls the size of the main heap. The heap size is configured very early in the boot sequence, before loading plug-ins or doing much of anything else. | Heapsize configuration controls the size of the main heap. The heap size is configured very early in the boot sequence, before loading plug-ins or doing much of anything else. | ||
− | + | {| class="wikitable" | |
+ | | <code>heapsize <nn>M | <nn>G</code> | ||
+ | | Specifies the size of the heap in mb or gb. Setting the main heap size to 4gb or more requires recompilation of the entire system with <code>CLIB_VEC64 > 0</code>. See <code>.../clib/clib/vec_bootstrap.h</code>. | ||
+ | |} | ||
==== "plugin_path" parameter ==== | ==== "plugin_path" parameter ==== | ||
Line 161: | Line 225: | ||
Plugin_path configuration controls the set of directories searched for vlib plugins. Supply a colon-separated list of (absolute) directory names: | Plugin_path configuration controls the set of directories searched for vlib plugins. Supply a colon-separated list of (absolute) directory names: | ||
− | + | {| class="wikitable" | |
+ | | <code>plugin_path dir1:dir2:...:dirN</code> | ||
+ | | Configure the search path for plugins. | ||
+ | |} | ||
==== "cj" parameters ==== | ==== "cj" parameters ==== | ||
− | The cj thread-safe circular log buffer scheme is occasionally useful when chasing bugs. Calls to it should not be checked in. See .../vlib/vlib/unix/cj.c. | + | The cj thread-safe circular log buffer scheme is occasionally useful when chasing bugs. Calls to it should not be checked in. See <code>.../vlib/vlib/unix/cj.c</code>. |
− | + | {| class="wikitable" | |
− | + | | <code>records <nn></code> | |
+ | | Configure the number of cj records in the circular buffer. | ||
+ | |- | ||
+ | | <code>on</code> | ||
+ | | Turns on logging at the earliest possible moment. | ||
+ | |} | ||
==== "vhost-user" parameters ==== | ==== "vhost-user" parameters ==== | ||
Line 174: | Line 246: | ||
Vhost-user configuration parameters control the vhost-user driver. | Vhost-user configuration parameters control the vhost-user driver. | ||
− | + | {| class="wikitable" | |
− | + | | <code>coalesce-frames <nn></code> | |
− | + | | Subject to deadline-timer expiration - see next item - attempt to transmit at least <code>nn</code> packet frames. | |
+ | |- | ||
+ | | <code>coalesce-time <seconds></code> | ||
+ | | Hold packets no longer than (floating-point) <code>seconds</coode> before transmitting them. | ||
+ | |- | ||
+ | | <code>dont-dump-memory</code> | ||
+ | | vhost-user shared-memory segments can add up to a large amount of memory, so it's handy to avoid adding them to corefiles when using a significant number of such interfaces. | ||
+ | |} | ||
==== "l2learn" parameter ==== | ==== "l2learn" parameter ==== | ||
Line 182: | Line 261: | ||
Configures the number of L2 (MAC) addresses in the L2 FIB at any one time. Defaults to 1M entries. | Configures the number of L2 (MAC) addresses in the L2 FIB at any one time. Defaults to 1M entries. | ||
− | limit <nn> | + | {| class="wikitable" |
+ | | <code>limit <nn></code> | ||
+ | | Limits the size of the L2 FIB to <code>nn</code> concurrent entries. | ||
+ | |} |
Revision as of 03:02, 28 April 2016
Contents
[hide]Introduction
The vpp network stack comes with a command-line interface that can be used for debugging purposes.
Please help maintain and improve this document!
Command-line arguments
The vpp stack can be configured from the command-line in a number of ways. Specific applications built on the stack have been known to require a dozen arguments, depending on requirements. This section describes commonly-used command-line options. These options are parsed by individual bits of code, similar to the action functions of the CLI parser.
You can find command-line argument parsers in the source code by searching for instances of the VLIB_CONFIG_FUNCTION macro. The actual command-line invocation syntax is extremely flexible.
macro invocation: VLIB_CONFIG_FUNCTION (foo_config, "foo") command-line: "foo { arg1 arg2 arg3 ... }"
When using the command on the command line, surround the list of arguments with curly braces.
NOTE: If you supply a single argument to a specific configuration function, you do not need to surround the argument with curly braces. For example, the following command argument does not have braces:
# /cisco/bin/swt unix interactive SWT: <start-typing..>
Vpp applications must be able to locate their own executable images. The simplest way to ensure this will work: invoke a vpp application by giving its absolute path. At startup, vpp aplications parse through their own Elf-sections [primarily] to make lists of init, configuration, and exit handlers. In gdb, it's often sufficient to start an application like this:
(gdb) r unix interactive
"unix" parameters
interactive
|
Attach CLI to stdin/out and provide a debugging command line interface. Implies nodaemon .
|
nodaemon
|
Do not fork / background the vpp process. Typical when invoking VPP applications from a process monitor. |
cli-listen localhost:5002
|
Bind the CLI to listen at address localhost on TCP port 5002 .
|
exec <filename>
|
Read startup configuration from filename .
|
log <filename>
|
Logs the startup configuration and all subsequent CLI commands in filename . Very useful in situations where folks don't remember or can't be bothered to include CLI commands in bug reports.
|
cli-line-mode
|
Disable character-by-character I/O on stdin. Useful when combined with, for example, emacs M-x gud-gdb .
|
cli-history-limit <nn>
|
Limit commmand history to nn lines. A value of 0 disables command history. Default value: 50
|
cli-no-banner
|
Disable the login banner on stdin and Telnet connections. |
cli-no-pager
|
Disable the output pager. |
cli-pager-buffer-limit <nn>
|
Limit pager buffer to nn lines of output. A value of 0 disables the pager. Default value: 100000
|
full-coredump
|
Ask the Linux kernel to dump all memory-mapped address regions, instead of just text+data+bss. |
"tuntap" parameters
The "tuntap" driver configures a point-to-point interface between the vpp engine and the local Linux kernel stack. This allows e.g. users to ssh to the host / VM / container via vpp "revenue" interfaces. It's marginally useful, and is currently disabled by default. To [dynamically] create TAP interfaces - the preferred scheme - see the "tap_connect" binary API
tuntap disable
|
Disable the tun/tap driver if desired |
tuntap ethernet
|
Create a tap device (ethernet MAC) instead of a tun device (point-to-point tunnel) |
tuntap have-normal-interface
|
Treat the host Linux stack as a routing peer instead of programming vpp interface L3 addresses onto the tun/tap devices. You'll need to manually configure the Linux network stack "vnet" interface, configure vlan subinterfaces if desired, etc. |
tuntap name ishmael
|
Call the tun/tap device ishmael |
Here's a typical multiple parameter invocation:
tuntap { ethernet have-normal-interface name vpp1 }
"api-trace" parameters
The ability to trace, dump, and replay control-plane API traces makes all the difference in the world when trying to understand what the control-plane has tried to ask the forwarding-plane to do.
on
|
Enable API trace capture from the beginning of time, and arrange for a post-mortem dump of the API trace if the application terminates abnormally. By default, the (circular) trace buffer will be configured to capture 256K traces |
nitems <nnn>
|
Configure the circular trace buffer to contain the last <nnn> entries. By default, the trace buffer captures the last 256K API messages received. |
Typically, one simply enables the API message trace scheme:
api-trace { on }
"cpu" parameters
Command-line CPU configuration controls the creation of named thread types, and the cpu affinity thereof. As of this writing, the cpu configuration function supports these parameters:
workers <nn>
|
Create <nn> worker threads. |
io <nn>
|
Create nn i/o threads.
|
main-thread-io
|
Handle i/o devices from thread 0, hand off traffic to worker threads. Requires "workers <nn>". |
skip-cores <nn>
|
Leave the low nn bits of the process affinity mask clear.
|
In some cases, it may be necessary to place thread instances manually:
main-core <n>
|
Assign main thread to specific core. |
coremask-workers <hex-mask>
|
Place worker threads according to the bitmap hex-mask .
|
corelist-workers <list>
|
Same as coremask-workers but accepts a list of cores instead of a bitmap.
|
coremask-io <hex-mask>
|
Place I/O threads according to the bitmap hex-mask .
|
corelist-io <list>
|
Same as coremask-io but accepts a list of cores instead of a bitmap.
|
Note that the "main" thread always occupies the lowest core-id specified in the DPDK [process-level] coremask.
Here's a full-bore manual placement example:
/cisco/bin/vpe unix interactive tuntap disable cpu { main-thread-io coremask-workers 18 coremask-stats 4 } dpdk {coremask 1e} # taskset -a -p <vpe-pid> pid 16251's current affinity mask: 2 # main thread pid 16288's current affinity mask: ffffff # DPDK interrupt thread (not bound to a core) pid 16289's current affinity mask: 4 # stats thread pid 16290's current affinity mask: 8 # worker thread 0 pid 16291's current affinity mask: 10 # worker thread 1
"dpdk" parameters
Command line DPDK configuration controls a number of parameters, including device whitelisting, the number of CPUs available for launching dpdk-eal-controlled threads, the number of I/O buffers, and the process affinity mask.
In addition, the DPDK configuration function attempts to support all of the DPDK EAL configuration parameters.
Popular options include:
coremask <hex>
|
Sets the process-level coremask. See previous section on cpu parameters for more details. |
no-hugetlb
|
Don't use huge TLB pages. Potentially useful for running simulator images. |
dev <pci-dev>
|
White-list [as in, attempt to drive] a specific PCI device. PCI-dev is a string of the form "DDDD:BB:SS.F " where DDDD = Domain, BB = Bus Number, SS = Slot number, and F = Function. This is the same format used in the linux sysfs tree (i.e. /sys/bus/pci/devices ) for PCI device directory names.
|
vdev <DPDK EAL command to create link bonding interfaces>
|
Specify bonded Ethernet interfaces, operating modes and PCI address of slave links. For example, "vdev eth_bond0,mode=2,slave=0000:0f:00.0,slave=0000:11:00.0,xmit_policy=l34 vdev eth_bond1,mode=2,slave=0000:10:00.0,slave=0000:12:00.0,xmit_policy=l34 ". Only XOR balanced (mode 2) mode is supported.
|
num-mbufs <nn>
|
Number of I/O buffers. You need to increase this if you have a lot of physical interfaces with multiple RSS queues (for example, 131072 ).
|
kni <nn>
|
Number of KNI interfaces. Refer to the DPDK documentation |
uio-driver <name>
|
Defaults to "igb_uio " [the Ubuntu default]. The names "uio_pci_generic " or "vfio-pci " are other possible values.
|
All of the DPDK EAL options should be available. See .../open-repo/vnet/vnet/devices/dpdk/dpdk.h
, look at the set of "foreach_eal_XXX
" macros.
"l2tp" parameters
ip6-l2tpv3 configuration controls the method used to locate a specific ip6-l2tpv3 tunnel. The following settings are mutually exclusive
lookup-v6-src
|
Lookup tunnel by ip6 source address. |
lookup-v6-dst
|
Lookup tunnel by ip6 destination address. |
lookup-session-id
|
Lookup tunnel by l2tpv3 session identifier. |
Refer to .../open-repo/vnet/vnet/devices/dpdk/{dpdk.h, init.c}
for more details.
"oam" parameters
Oam configuration controls the (ip4-icmp) interval, and number of misses allowed before reporting an oam target down to any registered listener.
interval <floating-point-seconds>
|
Interval between sending oam ip4 icmp messages. |
misses-allowed <nn>
|
Number of misses before declaring an oam target down. |
"heapsize" parameter
Heapsize configuration controls the size of the main heap. The heap size is configured very early in the boot sequence, before loading plug-ins or doing much of anything else.
<nn>G | Specifies the size of the heap in mb or gb. Setting the main heap size to 4gb or more requires recompilation of the entire system with CLIB_VEC64 > 0 . See .../clib/clib/vec_bootstrap.h .
|
"plugin_path" parameter
Plugin_path configuration controls the set of directories searched for vlib plugins. Supply a colon-separated list of (absolute) directory names:
plugin_path dir1:dir2:...:dirN
|
Configure the search path for plugins. |
"cj" parameters
The cj thread-safe circular log buffer scheme is occasionally useful when chasing bugs. Calls to it should not be checked in. See .../vlib/vlib/unix/cj.c
.
records <nn>
|
Configure the number of cj records in the circular buffer. |
on
|
Turns on logging at the earliest possible moment. |
"vhost-user" parameters
Vhost-user configuration parameters control the vhost-user driver.
coalesce-frames <nn>
|
Subject to deadline-timer expiration - see next item - attempt to transmit at least nn packet frames.
|
coalesce-time <seconds>
|
Hold packets no longer than (floating-point) seconds</coode> before transmitting them.
|
<code>dont-dump-memory | vhost-user shared-memory segments can add up to a large amount of memory, so it's handy to avoid adding them to corefiles when using a significant number of such interfaces. |
"l2learn" parameter
Configures the number of L2 (MAC) addresses in the L2 FIB at any one time. Defaults to 1M entries.
limit <nn>
|
Limits the size of the L2 FIB to nn concurrent entries.
|