Latest revision as of 13:26, 20 March 2024

VPP Security Groups

Introduction

Features are tracked as they are developed in the following VPP-427.

The 1.2 version of the plugin is committed via change 5805

The 1.1 version of the plugin is committed via change 3423

ACL Node architecture in 17.01

Changes/News in the version 1.2

Unified L2 / L3 processing path

This version adds the processing for the packets in the routed data path in addition to the switching data path by the same code with the same API.

It also collapses the entire processing into the single node - per-AF, per-L2/L3, per-direction. (So in total there are 8 nodes using the same core code).

The node architecture on the diagram here is thus now simplified, with an ACL node combining the session lookup and ACL lookup.

(1704: The old processing path is still there, but is not used and is left for the time being for potential backwards compatibility/ fallback purposes. The goal is to have it removed by 17.07 version.)

IPv6 extension header skipping

This version adds the skipping of most of the known extension headers during the ACL processing, so one can for example skip the Segment Routing header when matching.

In this version, the extension headers to be skipped are treated with the semantics of https://tools.ietf.org/html/rfc6564 semantics.

For a given extension header value, one can choose - either to skip it, or to treat it as a L4 protocol number - so as to be able to match on that within the ACL.

Performance

The initial performance tests of 1.2 version show the comparable performance to 1.1, which is expected since the main focus for this version was getting to feature completeness. The performance (along side with multithread-safety) will be the target of the 1.3 version for the release 17.07.

Example packet traces

The below packet trace shows the inbound security ACL in the L3 packet path and an outbound security ACL in L2 packet path, this can be reproduced as part of the unit tests, by doing "TEST=acl_plugin_l2l3 make test-debug":

00:00:01:433983: pg-input
  stream pcap3, 73 bytes, 3 sw_if_index
  current data 0, length 73, free-list 6, clone-count 0, trace 0x1
  IP6: 02:03:00:00:ff:02 -> 02:fe:30:f6:b7:4e
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 64, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44
00:00:01:434603: ethernet-input
  IP6: 02:03:00:00:ff:02 -> 02:fe:30:f6:b7:4e
00:00:01:434625: ip6-input
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 64, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44
00:00:01:434636: acl-plugin-in-ip6-fa
  acl-plugin: sw_if_index 3, next index 1, action: 1, match: acl 1 rule 1 trace_bits 00000000
  pkt info 00000000030001fd 0200000000000000 00000000040001fd 0300000000000000 0000001110e204d3 0000000000000420
00:00:01:434683: ip6-lookup
  fib 0 dpo-idx 257 flow hash: 0x00000000
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 64, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44
00:00:01:434699: ip6-rewrite
  tx_sw_if_index 4 adj-idx 257 : ipv6 via fd01:4::3 loop0: 02040000ff03dead0000000086dd flow hash: 0x00000000
  00000000: 02040000ff03dead0000000086dd600000000013113ffd010003000000000000
  00000020: 000000000002fd01000400000000000000000000000304d310e20013da443020
  00000040: 332034202d31202d310000000000000000000000000000000000000000000000
  00000060: 00000000000000000000000000000000000000000000000000000000
00:00:01:434728: loop0-output
  loop0
  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 63, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44
00:00:01:434739: l2-input
  l2-input: sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00
00:00:01:434749: l2-fwd
  l2-fwd:   sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 bd_index 1
00:00:01:434759: l2-flood
  l2-flood: sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 bd_index 1
00:00:01:434775: l2-output
  l2-output: sw_if_index 2 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 data 86 dd 60 00 00 00 00 13 11 3f fd 01
00:00:01:434800: l2-output-classify
  l2-classify: sw_if_index 2, table 5, offset 0, next 2
00:00:01:434816: acl-plugin-out-ip6-l2
  acl-plugin: sw_if_index 2, next index 1, action: 1, match: acl 0 rule 1 trace_bits 00000000
  pkt info 00000000030001fd 0200000000000000 00000000040001fd 0300000000000000 0000001110e204d3 0000000000000420
00:00:01:434848: pg1-output
  pg1
  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 63, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44
00:00:01:434858: pg1-tx
    buffer 0xb4b4: current data 0, length 73, free-list 5, clone-count 0, trace 0x1
  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 63, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44
00:00:01:434941: l2-flood
  l2-flood: sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 bd_index 1
00:00:01:434957: l2-output
  l2-output: sw_if_index 1 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 data 86 dd 60 00 00 00 00 13 11 3f fd 01
00:00:01:434966: l2-output-classify
  l2-classify: sw_if_index 1, table 3, offset 0, next 2
00:00:01:434969: acl-plugin-out-ip6-l2
  acl-plugin: sw_if_index 1, next index 1, action: 1, match: acl 0 rule 1 trace_bits 00000000
  pkt info 00000000030001fd 0200000000000000 00000000040001fd 0300000000000000 0000001110e204d3 0000000000000420
00:00:01:434975: pg0-output
  pg0
  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 63, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44
00:00:01:434978: pg0-tx
    buffer 0xb4b4: current data 0, length 73, free-list 6, clone-count 0, trace 0x1
  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
  UDP: fd01:3::2 -> fd01:4::3
    tos 0x00, flow label 0x0, hop limit 63, payload length 19
  UDP: 1235 -> 4322
    length 19, checksum 0xda44

A trace in the other direction:

00:00:02:616543: pg-input
  stream pcap1, 73 bytes, 1 sw_if_index
  current data 0, length 73, free-list 7, clone-count 0, trace 0x1
  IP6: 02:04:00:00:ff:03 -> de:ad:00:00:00:00
  UDP: fd01:4::3 -> fd01:3::2
    tos 0x00, flow label 0x0, hop limit 64, payload length 19
  UDP: 4322 -> 1235
    length 19, checksum 0xda44
00:00:02:617044: ethernet-input
  IP6: 02:04:00:00:ff:03 -> de:ad:00:00:00:00
00:00:02:617074: l2-input
  l2-input: sw_if_index 1 dst de:ad:00:00:00:00 src 02:04:00:00:ff:03
00:00:02:617086: l2-input-classify
  l2-classify: sw_if_index 1, table 3, offset 0, next 17
00:00:02:617101: acl-plugin-in-ip6-l2
  acl-plugin: sw_if_index 1, next index 7, action: 1, match: acl 8 rule 1 trace_bits 00000000
  pkt info 00000000040001fd 0300000000000000 00000000030001fd 0200000000000000 0000001104d310e2 0000000000000420
00:00:02:617155: l2-learn
  l2-learn: sw_if_index 1 dst de:ad:00:00:00:00 src 02:04:00:00:ff:03 bd_index 1
00:00:02:617173: l2-fwd
  l2-fwd:   sw_if_index 1 dst de:ad:00:00:00:00 src 02:04:00:00:ff:03 bd_index 1
00:00:02:617182: ip6-input
  UDP: fd01:4::3 -> fd01:3::2
    tos 0x00, flow label 0x0, hop limit 64, payload length 19
  UDP: 4322 -> 1235
    length 19, checksum 0xda44
00:00:02:617191: ip6-lookup
  fib 0 dpo-idx 507 flow hash: 0x00000000
  UDP: fd01:4::3 -> fd01:3::2
    tos 0x00, flow label 0x0, hop limit 64, payload length 19
  UDP: 4322 -> 1235
    length 19, checksum 0xda44
00:00:02:617215: ip6-rewrite
  tx_sw_if_index 3 adj-idx 507 : ipv6 via fd01:3::2 pg2: 02030000ff0202fe30f6b74e86dd flow hash: 0x00000000
  00000000: 02030000ff0202fe30f6b74e86dd600000000013113ffd010004000000000000
  00000020: 000000000003fd01000300000000000000000000000210e204d30013da443020
  00000040: 332034202d31202d310000000000000000000000000000000000000000000000
  00000060: 00000000000000000000000000000000000000000000000000000000
00:00:02:617227: acl-plugin-out-ip6-fa
  acl-plugin: sw_if_index 3, next index 1, action: 1, match: acl 9 rule 1 trace_bits 00000000
  pkt info 00000000040001fd 0300000000000000 00000000030001fd 0200000000000000 0000001104d310e2 0000000000000420
00:00:02:617238: pg2-output
  pg2
  IP6: 02:fe:30:f6:b7:4e -> 02:03:00:00:ff:02
  UDP: fd01:4::3 -> fd01:3::2
    tos 0x00, flow label 0x0, hop limit 63, payload length 19
  UDP: 4322 -> 1235
    length 19, checksum 0xda44
00:00:02:617253: pg2-tx
    buffer 0xb4b4: current data 0, length 73, free-list 7, clone-count 0, trace 0x1
  IP6: 02:fe:30:f6:b7:4e -> 02:03:00:00:ff:02
  UDP: fd01:4::3 -> fd01:3::2
    tos 0x00, flow label 0x0, hop limit 63, payload length 19
  UDP: 4322 -> 1235
    length 19, checksum 0xda44

The below is a sample trace from an L2-only path with just an ingress ACL on IPv4, from a "test_acl_plugin.py" run:

00:00:01:189275: pg-input
  stream pcap1, 9014 bytes, 1 sw_if_index
  current data 0, length 9014, free-list 6, clone-count 0, trace 0x1
  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
  UDP: 172.17.101.1 -> 172.17.102.8
    tos 0x00, ttl 64, length 9000, checksum 0x3498
    fragment id 0x0001
  UDP: 11 -> 21541
    length 8980, checksum 0xfa5e
00:00:01:189796: ethernet-input
  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
00:00:01:189879: l2-input
  l2-input: sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01
00:00:01:189901: l2-input-classify
  l2-classify: sw_if_index 1, table 0, offset 0, next 16
00:00:01:189932: acl-plugin-in-ip4-l2
  acl-plugin: sw_if_index 1, next index 7, action: 1, match: acl 2 rule 0 trace_bits 00000000
  pkt info 0000000000000000 016511ac00000000 0000000000000000 086611ac00000000 000000115425000b 0000000000000420
00:00:01:190038: l2-learn
  l2-learn: sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 bd_index 1
00:00:01:190059: l2-fwd
  l2-fwd:   sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 bd_index 1
00:00:01:190076: l2-flood
  l2-flood: sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 bd_index 1
00:00:01:190091: l2-output
  l2-output: sw_if_index 2 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 data 08 00 45 00 23 28 00 01 00 00 40 11
00:00:01:190108: pg1-output
  pg1
  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
  UDP: 172.17.101.1 -> 172.17.102.8
    tos 0x00, ttl 64, length 9000, checksum 0x3498
    fragment id 0x0001
  UDP: 11 -> 21541
    length 8980, checksum 0xfa5e
00:00:01:190202: pg1-tx
    buffer 0x7008: current data 0, length 9014, free-list 6, clone-count 0, trace 0x1
  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
  UDP: 172.17.101.1 -> 172.17.102.8
    tos 0x00, ttl 64, length 9000, checksum 0x3498
    fragment id 0x0001
  UDP: 11 -> 21541
    length 8980, checksum 0xfa5e

CLI

The ACL plugin does not supply the "supported" debug CLI for configuration, but has the full support for talking to it via VAT CLI, which are documented below.

Note from 2024: In contrast with the above statement there seems to be some debug CLI support introduced in https://gerrit.fd.io/r/c/vpp/+/5805 and https://gerrit.fd.io/r/c/vpp/+/8805 . Please see the following vppctl commands:

clear acl-plugin sessions
delete acl-plugin acl
set acl-plugin
set acl-plugin acl
set acl-plugin interface
set flow classify
set interface input acl
set interface output acl
show acl-plugin acl
show acl-plugin decode 5tuple
show acl-plugin interface
show acl-plugin lookup context
show acl-plugin lookup user
show acl-plugin macip acl
show acl-plugin macip interface
show acl-plugin memory
show acl-plugin sessions
show acl-plugin tables
show inacl
show outacl

acl_plugin_get_version : get plugin version

Show the version of the ACL plugin

vat# acl_plugin_get_version
vl_api_acl_plugin_get_version_reply_t_handler:133: ACL plugin version: 1.1
vat#

acl_add_replace : add or replace an ACL

Add a new ACL or replace the existing one. To replace an existing ACL, pass the ID of this ACL. If you prefer a new ACL to be created, pass the 0xffffffff (-1)

Since the single API call defines the entire ACL, all of the entries of this ACL are specified on the same line, separated by commas.

A quasi-BNF syntax of the VAT command line is below:

acl_add_replace <acl-idx> [<ipv4|ipv6> <permit|permit+reflect|deny|action N> [src IP/plen] [dst IP/plen] [sport X-Y] [dport X-Y] [proto P] [tcpflags FL MASK], ... , ...

An example of adding a single ACL permitting IPv4 and IPv6 is below:

vat# acl_add_replace permit, ipv6 permit
vl_api_acl_add_replace_reply_t_handler:107: ACL index: 0
vat#

Replacing this ACL#0 by one with two host entries:

vat# acl_add_replace 0 ipv6 permit dst 2001:db8::1/128, ipv4 permit src 192.0.2.1/32
vl_api_acl_add_replace_reply_t_handler:107: ACL index: 0
vat#

acl_del : delete an ACL

Simply pass the ID of the ACL to delete it:

vat# acl_del 0
vat#

acl_dump : dump ACL(s)

Use the command without any arguments to dump all configured ACLs, or supply the optional ACL ID to dump only that ACL:

vat# acl_dump
vl_api_acl_details_t_handler:193: acl_index: 0, count: 2
   tag {}
   ipv6 action 1 src ::/0 dst 2001:db8::1/128 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 1.1.1.1/32 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
vl_api_acl_details_t_handler:193: acl_index: 1, count: 2
   tag {}
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv6 action 1 src ::/0 dst ::/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
vl_api_acl_details_t_handler:193: acl_index: 2, count: 5
   tag {}
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
vat# acl_dump 2
vl_api_acl_details_t_handler:193: acl_index: 2, count: 5
   tag {}
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
   ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
vat#

Notice that the output format of each ACL is (minus the newlines) suitable for feeding it as input to VAT to recreate that ACL.

acl_interface_add_del (not recommended) : add/delete ACL to/from list of applied ACLs on an interface

This command allows to append/delete one input or output ACL to the list of the ACLs applied to an interface.

Because one would mostly want to predictably set the ACLs, the usage of this API is not recommended.

The pseudo-BNF format is below:

acl_interface_add_del <intfc> | sw_if_index <if-idx> [add|del] [input|output] acl <acl-idx>

acl_interface_set_acl_list : set the list of inbound+outbound ACLs for a given interface

This command sets in one shot the list of input and output ACLs applied to the interface. It unapplies any previously applied ACLs, which makes this command idempotent.

 acl_interface_set_acl_list <intfc> | sw_if_index <if-idx> input [acl-idx list] output [acl-idx list]

An example:

vat# acl_interface_set_acl_list sw_if_index 0 input 0 output 0 1 2
vat#

acl_interface_list_dump : show which interfaces have which lists of which ACLs applied

acl_interface_list_dump [<intfc> | sw_if_index <if-idx>]

This command lists which ACLs are applied on which interface and in which direction.

The below example is a result from setting the ACL on the interface using the previous command:

vat# acl_interface_list_dump
vl_api_acl_interface_list_details_t_handler:152: sw_if_index: 0, count: 4, n_input: 1
   input 0
  output 0 1 2
vat#

macip_acl_add : add a MACIP ACL

macip_acl_add [ipv4|ipv6] [permit|deny|action N] [ip <ADDR>/<PREFIX-LEN> mac <MAC> mask <MAC-MASK>, ...

vat# macip_acl_add ipv4 permit ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask ff:ff:ff:ff:ff:ff, ipv6 permit ip 2001:db8::1/128  mac 00:01:02:03:04:05 mask 00:00:00:00:00:00
vl_api_macip_acl_add_reply_t_handler:107: ACL index: 7
vat#

macip_acl_del : delete a MACIP ACL

Delete the MACIP ACL by its index

vat# macip_acl_del 6
vat#

macip_acl_dump

Dump the MACIP ACLs (or just one if you supply its index)

vat# macip_acl_dump
vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 0, count: 0
   tag {}
vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 1, count: 1
   tag {}
   ipv6 action 1 ip ::/0 mac 00:00:00:00:00:00 mask 00:00:00:00:00:00
vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 2, count: 1
   tag {}
   ipv6 action 1 ip ::/0 mac 00:00:00:00:00:00 mask 00:00:00:00:00:00
vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 3, count: 1
   tag {}
   ipv6 action 1 ip ::/0 mac 00:00:00:00:00:00 mask 00:00:00:00:00:00
vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 4, count: 1
   tag {}
   ipv4 action 1 ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask ff:ff:ff:ff:ff:ff
vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 5, count: 1
   tag {}
   ipv4 action 1 ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask 00:00:00:00:00:00
vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 7, count: 2
   tag {}
   ipv4 action 1 ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask ff:ff:ff:ff:ff:ff,
   ipv6 action 1 ip 2001:db8::1/128 mac 00:01:02:03:04:05 mask 00:00:00:00:00:00
vat#

macip_acl_interface_add_del: apply/unapply the MACIP ACL to/from a given interface

macip_acl_interface_add_del <intfc> | sw_if_index <if-idx> [add|del] acl <acl-idx>

vat# macip_acl_interface_add_del local0 add acl 7
vat#

macip_acl_interface_get: get the list of interfaces and associated MACIP ACLs

vat# macip_acl_interface_add_del local0 add acl 7
vat# macip_acl_interface_get
vl_api_macip_acl_interface_get_reply_t_handler:241: sw_if_index with MACIP ACL count: 1
  macip_acl_interface_add_del sw_if_index 0 add acl 7
  
vat# macip_acl_interface_add_del local0 del acl 7
vat# macip_acl_interface_get
vl_api_macip_acl_interface_get_reply_t_handler:241: sw_if_index with MACIP ACL count: 1
  macip_acl_interface_add_del sw_if_index 0 add acl -1
  
vat#

Initial performance tests

The first version of the ACL plugin was explicitly focused on getting things "correct" as the first priority, even if at some expense of getting them "fast". But understanding the performance is very important, so we did limited performance testing. The performance testing was done using MoonGen, running on the same host as VPP. The VPP was run by doing "make release-build; make release-plugins; make release-run" process. The VPP is configured via VAT.

The performance was done by testing with 200000 unidirectional UDP streams, by submitting the line-rate 10Gbps of traffic by MoonGen (14.88Mpps of 64-byte packets) and observing the amount of traffic received on the other side.

First, we test the baseline configuration which just doing briding between the two interfaces.

 sw_interface_set_flags TenGigabitEthernet81/0/0 admin-up link-up
 sw_interface_set_flags TenGigabitEthernet81/0/1 admin-up link-up
 bridge_domain_add_del bd_id 42 flood 1 uu-flood 1 forward 1 learn 1 arp-term 0
 sw_interface_set_l2_bridge TenGigabitEthernet81/0/0 bd_id 42
 sw_interface_set_l2_bridge TenGigabitEthernet81/0/1 bd_id 42

This configuration exhibited 9.52 Mpps performance.

Then we added a trivial case of a one-line "permit" ACL which is checked on input and output of the packet path, using the following VAT commands:

 acl_add_replace permit
 acl_interface_add_del sw_if_index 1 add input acl 0
 acl_interface_add_del sw_if_index 1 add output acl 0
 acl_interface_add_del sw_if_index 2 add input acl 0
 acl_interface_add_del sw_if_index 2 add output acl 0

This configuration causes two very simple ACL checks - on input of the packet path and on the output, see the below trace(NB: this is a trace from 1701, the trace from 1704 looks different, though the logic is very similar. See the sample traces above):

 00:02:12:167665: dpdk-input
   TenGigabitEthernet81/0/0 rx queue 0
   buffer 0x5358: current data 0, length 60, free-list 0, totlen-nifb 0, trace 0x0
   PKT MBUF: port 0, nb_segs 1, pkt_len 60
     buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 0x6ee4d640
     packet_type 0x0
     Packet Offload Flags
   IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
   UDP: 10.0.80.47 -> 10.1.0.10
     tos 0x00, ttl 64, length 46, checksum 0x1686
     fragment id 0x0000
   UDP: 1234 -> 319
     length 26, checksum 0x956f
 00:02:12:167682: ethernet-input
   IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
 00:02:12:167691: l2-input
   l2-input: sw_if_index 1 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06
 00:02:12:167693: l2-input-classify
   l2-classify: sw_if_index 1, table 0, offset 0, next 9
 00:02:12:167700: acl-plugin-in
   ACL_IN: sw_if_index 1, next index 10, match: inacl 0 rule 0 trace_bits 00000000
 00:02:12:167709: l2-learn
   l2-learn: sw_if_index 1 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06 bd_index 1
 00:02:12:167712: l2-flood
   l2-flood: sw_if_index 1 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06 bd_index 1
 00:02:12:167714: l2-output
   l2-output: sw_if_index 2 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06
 00:02:12:167716: l2-output-classify
   l2-classify: sw_if_index 2, table 6, offset 0, next 5
 00:02:12:167723: acl-plugin-out
   ACL_OUT: sw_if_index 2, next index 4, match: outacl 0 rule 0 trace_bits 00000000
 00:02:12:167734: TenGigabitEthernet81/0/1-output
   TenGigabitEthernet81/0/1
   IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
   UDP: 10.0.80.47 -> 10.1.0.10
     tos 0x00, ttl 64, length 46, checksum 0x1686
     fragment id 0x0000
   UDP: 1234 -> 319
     length 26, checksum 0x956f
 00:02:12:167736: TenGigabitEthernet81/0/1-tx
   TenGigabitEthernet81/0/1 tx queue 0
   buffer 0x5358: current data 0, length 60, free-list 0, totlen-nifb 0, trace 0x0
   IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
   UDP: 10.0.80.47 -> 10.1.0.10
     tos 0x00, ttl 64, length 46, checksum 0x1686
     fragment id 0x0000
   UDP: 1234 -> 319
     length 26, checksum 0x956f

This configuration exhibited the performance of 4.62 Mpps.

To test the impact of the linear match, we add lines to ACL one by one:

 acl_add_replace 0 permit src 1.1.1.1/32,permit

performance: 4.50Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32,permit

performance: 4.34 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32,permit

performance: 4.19 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, permit

performance: 4.04 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, permit

performance: 3.90 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, permit

performance: 3.77 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32,  permit

performance: 3.64 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, permit

performance: 3.55 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, permit

performance: 3.23 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, permit

performance: 3.33 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, permit

performance: 3.24 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, permit

performance: 3.14 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, src 1.1.1.13/32, permit

performance: 3.06 Mpps

 acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, src 1.1.1.13/32, src

1.1.1.14/32, permit

performance: 2.85 Mpps

We can see that the performance impact is not strictly linear, but it gives the impression.

Changing the last match to "permit and reflect" in order to cause the stateful processing path:

acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, src 1.1.1.13/32, src 1.1.1.14/32, permit+reflect

performance: 3.68 Mpps

We can see that the performance of the stateful path is less than the performance of the stateless path with small ACLs - this is most probably due to the current code organization of session tracker nodes, which use some code sharing between the nodes to ease the maintenance of it.

Removing all ACLs:

 acl_interface_add_del sw_if_index 1 del input acl 0
 acl_interface_add_del sw_if_index 1 del output acl 0
 acl_interface_add_del sw_if_index 2 del input acl 0
 acl_interface_add_del sw_if_index 2 del output acl 0

We get to 9.47 Mpps.

Let's test multiple ACL matching:

 acl_add_replace src 1.1.1.1/32
 acl_add_replace src 1.1.1.1/32
 acl_add_replace src 1.1.1.1/32
 acl_add_replace src 1.1.1.1/32
 acl_add_replace src 1.1.1.1/32

First same condition as before, with just one ACL check:

 acl_interface_set_acl_list sw_if_index 1 input 0 output 0
 acl_interface_set_acl_list sw_if_index 2 input 0 output 0

performance: 4.67 Mpps

Two ACLs:

 acl_interface_set_acl_list sw_if_index 1 input 1 0 output 1 0
 acl_interface_set_acl_list sw_if_index 2 input 1 0 output 1 0

performance: 4.16 Mpps

Three ACLs:

 acl_interface_set_acl_list sw_if_index 1 input 2 1 0 output 2 1 0
 acl_interface_set_acl_list sw_if_index 2 input 2 1 0 output 2 1 0

performance: 3.73 Mpps

Four ACLs:

 acl_interface_set_acl_list sw_if_index 1 input 3 2 1 0 output 3 2 1 0
 acl_interface_set_acl_list sw_if_index 2 input 3 2 1 0 output 3 2 1 0

performance: 3.38 Mpps

Five ACLs:

 acl_interface_set_acl_list sw_if_index 1 input 4 3 2 1 0 output 4 3 2 1 0
 acl_interface_set_acl_list sw_if_index 2 input 4 3 2 1 0 output 4 3 2 1 0

Performance: 3.10 Mpps

Six ACLs:

 acl_interface_set_acl_list sw_if_index 1 input 5 4 3 2 1 0 output 5 4 3 2 1 0
 acl_interface_set_acl_list sw_if_index 2 input 5 4 3 2 1 0 output 5 4 3 2 1 0

Performance: 2.85 Mpps

Convert to last "permit" to stateful:

 acl_add_replace 0 permit+reflect

Performance: 3.67 Mpps

These two performance axis might combine 0 i.e. if the ACL 5 in the above test were to be long and not match, the performance will be worse than just with that ACL and worse than with 6 trivial ACLs.

Removing all ACLs:

 acl_interface_set_acl_list sw_if_index 1 
 acl_interface_set_acl_list sw_if_index 2

The performance is 9.45 Mpps.

MACIP ACLs

 macip_acl_add permit
 macip_acl_interface_add_del sw_if_index 1 add acl 0

Performance: 7.30 Mpps

 macip_acl_add permit ip 128.1.0.0/7, permit ip 10.0.0.0/8
 macip_acl_interface_add_del sw_if_index 1 add acl 1

Performance: 7.31 Mpps (the hit on the first classify table)

 macip_acl_add permit ip 128.1.0.0/9, permit ip 10.0.0.0/8
 macip_acl_interface_add_del sw_if_index 1 add acl 1

Performance: 7.31 Mpps (the hit on the first classify table)

(more testing with MACIP ACLs should be done)

Requirements

Support classifiers/filters on any interface type (bridged / routed)
Filter on IP-addresses with address mask or prefix length (IPv4 and IPv6)
Filter on source and destination TCP/UDP port ranges
Filter on source and destination L2 MAC addresses
Support IPv6 with extension headers present
Support fragmented packets and unknown transport layer headers
Combinations of the above filters (e.g. MAC + IP)
Filters on ingress and egress interfaces
Stateful firewall. No application layer filtering.

Work list

Task	Owner	Priority	Status	Description
API definition	Ole	0	Done	VPP-513
Connection tracker	Andrew	0	Done	VPP-514
Stateful ACLs		0		VPP-515
ACL policy matching node (MVP)	Andrew	0	Done	input output
Direct classifier policy matching		-
Control Plane test code (new framework)	Pavel	0	WIP
Data Plane tests (performance + scale)		0

 1. Python tests/examples -> Ole + Pavel
 2a. IPv4 matching in all plugin -> Andrew - done.
 2b. make it “deny by default” -> Andrew - done.
 2c. port range support -> Andrew - done.
 2d. ICMP type/code matching -> Andrew - done.
 3. Performance testing -> Andrew - done.
 --- MVP ---
 4a. Plumbing for stateful sessions from ACL plugin (to be able to specify “match and track” (“permit and create the forward/return session”) -> Andrew - done.
 4b. Stateful session tracking - timeouts -> Andrew - Done.
 4c. Stateful session tracking - lightweight TCP state -> Andrew - Done
 5. MACIP(L2) rules -> Andrew - done.
 6. Code cleanups -> Andrew

 
 PHASE2:
 A. ACL/Sessions support for L3 (routed) mode - (big) !
 B. Can we implement the ACL match purely in terms of classifier tables ? How expensive/(in)efficient that would be ?
 C. Extension header handling during the slow path lookup - easy in ACL plugin
 D. classifier match for the sessions with extension headers - currently no extension headers supported

API

API file as implemented in 17.01

MACIP (formerly "L2") API

MACIP (renamed to avoid confusion) is an ingress-only ACL which permits the traffic based on a mix of MAC and IP address matches.

The use of this mechanism is to prevent spoofing.

API file as implemented

API as implemented supports MAC address masks and prefixes, however, be aware: the current implementation is done using chained classifier tables, so each variation of the masks/prefix lengths means an extra table and hence the performance impact.

These filters are per-packet so you will want to care for performance.

For best performance, use the exact match MAC mask (ff:ff:ff:ff:ff:ff) and the maximum prefix length (/32 for IPv4 and /128 for IPv6).

Design and prototyping

The ACL matching is implemented in this phase as a simple array search, under the assumption that given the rules are per-port, the rule list will be small.

The redirection of the traffic to the node performing the ACL match is done by installing an empty L2 classifier table whose "miss-next" index diverts the traffic to the node.

The ACL match node can also redirect the traffic to the stateful-session setup node (by having a "permit" = 2 in the ACE), which will create the session on that interface.

...TBD: more details...

Examples

YANG model

Open Issues

Closed Issues

Security Group use case specific API. Done in a plugin.

Existing functionality

The existing functionality has a classifier (https://wiki.fd.io/view/VPP/Introduction_To_N-tuple_Classifiers) matching.

As the above document explains, the classifier is a series of chained tables, with each table having a specific mask, but this mask is the same for all entries.

This has been tested to happen in the L2 bridged case (test case: http://stdio.be/vpp/t/aytest-bridge-tap-py.txt).

Therefore, if we have an example policy:

 nova secgroup-create test-secgroup test
 nova secgroup-add-rule test-secgroup icmp -1 -1 0.0.0.0/0
 nova secgroup-add-rule test-secgroup tcp 22 22 0.0.0.0/0

So, assuming we match with offset 0 (from the beginning of the packet) the mask will look like this for the first line:

 000000000000 000000000000 0000 00 00 0000 0000 0000 00 FF 0000 00000000 00000000  00 00 0000 0000 
   eth dst      eth src    et   ihl t  len id    fo ttl pr  cs   ip4src   ip4dst    t  c  cs   id
   +-------- L2 ---------------+----------- L3 IPv4 ------------------------------+--------L4 ICMP -----+

For the TCP matching on port 22 it will look as follows:

 000000000000 000000000000 0000 00 00 0000 0000 0000 00 FF 0000 00000000 00000000  0000 FFFF 00000000 00000000 0000 0000 0000 0000
   eth dst      eth src    et   ihl t  len id    fo ttl pr  cs   ip4src   ip4dst    sp  dp    seq      ack      fl  win   cs   urg
   +-------- L2 ---------------+----------- L3 IPv4 ------------------------------+--------L4 TCP ---------------------------------+

(One would need to round up the number of bytes to the nearest 16-byte boundary that makes sense)

For IPv6 assuming no extension headers, it will look similar, with the L3 header being the IPv6 one:

 000000000000 000000000000 0000 0 00 00000 0000 FF 00 00000000000000000000000000000000 00000000000000000000000000000000 00 00 0000 0000 
   eth dst      eth src    et   v TC  fll  len  nh hl             ipv6 src                   ipv dst                    t  c  cs   id
   +-------- L2 ---------------+----------- L3 IPv6 --------------------------------------------------------------------+--------L4 ICMP -----+

For the TCP matching on port 22 it will look as follows:

 000000000000 000000000000 0000 0 00 00000 0000 FF 00 00000000000000000000000000000000 00000000000000000000000000000000 0000 FFFF 00000000 00000000 0000 0000 0000 0000
   eth dst      eth src    et   v TC  fll  len  nh hl             ipv6 src                   ipv dst                      sp  dp    seq      ack      fl  win   cs   urg
   +-------- L2 ---------------+----------- L3 IPv6 --------------------------------------------------------------------+--------L4 TCP ---------------------------------

Then using these masks one would create 4 tables, by using the API call:

 classify_add_del_table(is_add=1, skip_n_vectors=0, mask=<MMMM>, match_n_vectors=<NNNN>,nbuckets=32,memory_size=20000, next_table_index=-1, miss_next_index=-1)

Let's call these tables "IPv4PROTO", "IPv4PROTO_TCPDPORT", "IPv6PROTO", "IPv6PROTO_TCPDPORT".

One would mention "IPv4PROTO" table as "next_table_index" table for "IPv4PROTO_TCPDPORT", and "IPv6PROTO" as "next_table_index" table for IPv6PROTO_TCPDPORT table.

Then one needs to populate the tables with the correct matches for "ICMP" and "tcp dst port 22". That can be done using API call:

 classify_add_del_session(is_add=1, table_index=<XXXX>, match=<bytes-to-match>, hit-next-index -1)

The bytes "XXXX" above would be the match of one or several vectors, corresponding to the packet contents with the desired value.

WARNING: if the "skip" is nonzero in the table configuration, the match is still the entire bitstring, without skipping any leading bytes !!!

Then one would apply the IPv4PROTO_TCPDPORT and IPv6PROTO_TCPDPORT as l2 input classify tables.

The CLI for that is set interface l2 output classify intfc <name> ip[46]-table <tableid>.

The API for this is

  classify_set_interface_l2_tables(sw_if_index=<INTFC>, ip4_table_index=<IPv4PROTO_TCPDPORT>, ip6_table_index=<IPv6PROTO_TCPDPORT>, other_table_index=-1, is_input=0)

This would allow to create a unidirectional policy, assuming the other policy is "permit all" it would be fine. If not - then a mirror table entries will need to be created using the same logic.

The full script showing this process in detail using the python API is at http://stdio.be/vpp/t/classifier_script_simple_policy.txt

The Java API is located in $ROOT/vpp-api/java..

References

[1]
[2]

@@ Line 5: / Line 5: @@
 Features are tracked as they are developed in the following [https://jira.fd.io/browse/VPP-427 VPP-427].
-Initial development is done on github: [https://github.com/vpp-dev/vpp/tree/acl ACL branch]
+The 1.2 version of the plugin is committed via [https://gerrit.fd.io/r/#/c/5805/ change 5805]
-[[File:ACL Node architecture.png|thumb|ACL Node architecture]]
+The 1.1 version of the plugin is committed via [https://gerrit.fd.io/r/#/c/3423/ change 3423]
+[[File:ACL Node architecture.png|thumb|ACL Node architecture in 17.01]]
+== Changes/News in the version 1.2 ==
+=== Unified L2 / L3 processing path ===
+This version adds the processing for the packets in the routed data path in addition to the switching data path by the same code with the same API.
+It also collapses the entire processing into the single node - per-AF, per-L2/L3, per-direction. (So in total there are 8 nodes using the same core code).
+The node architecture on the diagram here is thus now simplified, with an ACL node combining the session lookup and ACL lookup.
+(1704: The old processing path is still there, but is not used and is left for the time being for potential backwards compatibility/ fallback purposes. The goal is to have it removed by 17.07 version.)
+=== IPv6 extension header skipping ===
+This version adds the skipping of most of the known extension headers during the ACL processing, so one can for example skip the Segment Routing header when matching.
+In this version, the extension headers to be skipped are treated with the semantics of https://tools.ietf.org/html/rfc6564 semantics.
+For a given extension header value, one can choose - either to skip it, or to treat it as a L4 protocol number - so as to be able to match on that within the ACL.
+=== Performance ===
+The initial performance tests of 1.2 version show the comparable performance to 1.1, which is expected since the main focus for this version was getting to feature completeness.
+The performance (along side with multithread-safety) will be the target of the 1.3 version for the release 17.07.
+=== Example packet traces ===
+The below packet trace shows the inbound security ACL in the L3 packet path and an outbound security ACL in L2 packet path, this can be reproduced as part of the unit tests,
+by doing "TEST=acl_plugin_l2l3 make test-debug":
+<pre>
+:00:01:433983: pg-input
+  stream pcap3, 73 bytes, 3 sw_if_index
+  current data 0, length 73, free-list 6, clone-count 0, trace 0x1
+  IP6: 02:03:00:00:ff:02 -> 02:fe:30:f6:b7:4e
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 64, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+:00:01:434603: ethernet-input
+  IP6: 02:03:00:00:ff:02 -> 02:fe:30:f6:b7:4e
+:00:01:434625: ip6-input
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 64, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+:00:01:434636: acl-plugin-in-ip6-fa
+  acl-plugin: sw_if_index 3, next index 1, action: 1, match: acl 1 rule 1 trace_bits 00000000
+  pkt info 00000000030001fd 0200000000000000 00000000040001fd 0300000000000000 0000001110e204d3 0000000000000420
+:00:01:434683: ip6-lookup
+  fib 0 dpo-idx 257 flow hash: 0x00000000
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 64, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+:00:01:434699: ip6-rewrite
+  tx_sw_if_index 4 adj-idx 257 : ipv6 via fd01:4::3 loop0: 02040000ff03dead0000000086dd flow hash: 0x00000000
+  00000000: 02040000ff03dead0000000086dd600000000013113ffd010003000000000000
+  00000020: 000000000002fd01000400000000000000000000000304d310e20013da443020
+  00000040: 332034202d31202d310000000000000000000000000000000000000000000000
+  00000060: 00000000000000000000000000000000000000000000000000000000
+:00:01:434728: loop0-output
+  loop0
+  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 63, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+:00:01:434739: l2-input
+  l2-input: sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00
+:00:01:434749: l2-fwd
+  l2-fwd:   sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 bd_index 1
+:00:01:434759: l2-flood
+  l2-flood: sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 bd_index 1
+:00:01:434775: l2-output
+  l2-output: sw_if_index 2 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 data 86 dd 60 00 00 00 00 13 11 3f fd 01
+:00:01:434800: l2-output-classify
+  l2-classify: sw_if_index 2, table 5, offset 0, next 2
+:00:01:434816: acl-plugin-out-ip6-l2
+  acl-plugin: sw_if_index 2, next index 1, action: 1, match: acl 0 rule 1 trace_bits 00000000
+  pkt info 00000000030001fd 0200000000000000 00000000040001fd 0300000000000000 0000001110e204d3 0000000000000420
+:00:01:434848: pg1-output
+  pg1
+  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 63, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+:00:01:434858: pg1-tx
+    buffer 0xb4b4: current data 0, length 73, free-list 5, clone-count 0, trace 0x1
+  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 63, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+:00:01:434941: l2-flood
+  l2-flood: sw_if_index 4 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 bd_index 1
+:00:01:434957: l2-output
+  l2-output: sw_if_index 1 dst 02:04:00:00:ff:03 src de:ad:00:00:00:00 data 86 dd 60 00 00 00 00 13 11 3f fd 01
+:00:01:434966: l2-output-classify
+  l2-classify: sw_if_index 1, table 3, offset 0, next 2
+:00:01:434969: acl-plugin-out-ip6-l2
+  acl-plugin: sw_if_index 1, next index 1, action: 1, match: acl 0 rule 1 trace_bits 00000000
+  pkt info 00000000030001fd 0200000000000000 00000000040001fd 0300000000000000 0000001110e204d3 0000000000000420
+:00:01:434975: pg0-output
+  pg0
+  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 63, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+:00:01:434978: pg0-tx
+    buffer 0xb4b4: current data 0, length 73, free-list 6, clone-count 0, trace 0x1
+  IP6: de:ad:00:00:00:00 -> 02:04:00:00:ff:03
+  UDP: fd01:3::2 -> fd01:4::3
+    tos 0x00, flow label 0x0, hop limit 63, payload length 19
+  UDP: 1235 -> 4322
+    length 19, checksum 0xda44
+</pre>
+A trace in the other direction:
+<pre>
+:00:02:616543: pg-input
+  stream pcap1, 73 bytes, 1 sw_if_index
+  current data 0, length 73, free-list 7, clone-count 0, trace 0x1
+  IP6: 02:04:00:00:ff:03 -> de:ad:00:00:00:00
+  UDP: fd01:4::3 -> fd01:3::2
+    tos 0x00, flow label 0x0, hop limit 64, payload length 19
+  UDP: 4322 -> 1235
+    length 19, checksum 0xda44
+:00:02:617044: ethernet-input
+  IP6: 02:04:00:00:ff:03 -> de:ad:00:00:00:00
+:00:02:617074: l2-input
+  l2-input: sw_if_index 1 dst de:ad:00:00:00:00 src 02:04:00:00:ff:03
+:00:02:617086: l2-input-classify
+  l2-classify: sw_if_index 1, table 3, offset 0, next 17
+:00:02:617101: acl-plugin-in-ip6-l2
+  acl-plugin: sw_if_index 1, next index 7, action: 1, match: acl 8 rule 1 trace_bits 00000000
+  pkt info 00000000040001fd 0300000000000000 00000000030001fd 0200000000000000 0000001104d310e2 0000000000000420
+:00:02:617155: l2-learn
+  l2-learn: sw_if_index 1 dst de:ad:00:00:00:00 src 02:04:00:00:ff:03 bd_index 1
+:00:02:617173: l2-fwd
+  l2-fwd:   sw_if_index 1 dst de:ad:00:00:00:00 src 02:04:00:00:ff:03 bd_index 1
+:00:02:617182: ip6-input
+  UDP: fd01:4::3 -> fd01:3::2
+    tos 0x00, flow label 0x0, hop limit 64, payload length 19
+  UDP: 4322 -> 1235
+    length 19, checksum 0xda44
+:00:02:617191: ip6-lookup
+  fib 0 dpo-idx 507 flow hash: 0x00000000
+  UDP: fd01:4::3 -> fd01:3::2
+    tos 0x00, flow label 0x0, hop limit 64, payload length 19
+  UDP: 4322 -> 1235
+    length 19, checksum 0xda44
+:00:02:617215: ip6-rewrite
+  tx_sw_if_index 3 adj-idx 507 : ipv6 via fd01:3::2 pg2: 02030000ff0202fe30f6b74e86dd flow hash: 0x00000000
+  00000000: 02030000ff0202fe30f6b74e86dd600000000013113ffd010004000000000000
+  00000020: 000000000003fd01000300000000000000000000000210e204d30013da443020
+  00000040: 332034202d31202d310000000000000000000000000000000000000000000000
+  00000060: 00000000000000000000000000000000000000000000000000000000
+:00:02:617227: acl-plugin-out-ip6-fa
+  acl-plugin: sw_if_index 3, next index 1, action: 1, match: acl 9 rule 1 trace_bits 00000000
+  pkt info 00000000040001fd 0300000000000000 00000000030001fd 0200000000000000 0000001104d310e2 0000000000000420
+:00:02:617238: pg2-output
+  pg2
+  IP6: 02:fe:30:f6:b7:4e -> 02:03:00:00:ff:02
+  UDP: fd01:4::3 -> fd01:3::2
+    tos 0x00, flow label 0x0, hop limit 63, payload length 19
+  UDP: 4322 -> 1235
+    length 19, checksum 0xda44
+:00:02:617253: pg2-tx
+    buffer 0xb4b4: current data 0, length 73, free-list 7, clone-count 0, trace 0x1
+  IP6: 02:fe:30:f6:b7:4e -> 02:03:00:00:ff:02
+  UDP: fd01:4::3 -> fd01:3::2
+    tos 0x00, flow label 0x0, hop limit 63, payload length 19
+  UDP: 4322 -> 1235
+    length 19, checksum 0xda44
+</pre>
+The below is a sample trace from an L2-only path with just an ingress ACL on IPv4, from a "test_acl_plugin.py" run:
+<pre>
+:00:01:189275: pg-input
+  stream pcap1, 9014 bytes, 1 sw_if_index
+  current data 0, length 9014, free-list 6, clone-count 0, trace 0x1
+  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
+  UDP: 172.17.101.1 -> 172.17.102.8
+    tos 0x00, ttl 64, length 9000, checksum 0x3498
+    fragment id 0x0001
+  UDP: 11 -> 21541
+    length 8980, checksum 0xfa5e
+:00:01:189796: ethernet-input
+  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
+:00:01:189879: l2-input
+  l2-input: sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01
+:00:01:189901: l2-input-classify
+  l2-classify: sw_if_index 1, table 0, offset 0, next 16
+:00:01:189932: acl-plugin-in-ip4-l2
+  acl-plugin: sw_if_index 1, next index 7, action: 1, match: acl 2 rule 0 trace_bits 00000000
+  pkt info 0000000000000000 016511ac00000000 0000000000000000 086611ac00000000 000000115425000b 0000000000000420
+:00:01:190038: l2-learn
+  l2-learn: sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 bd_index 1
+:00:01:190059: l2-fwd
+  l2-fwd:   sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 bd_index 1
+:00:01:190076: l2-flood
+  l2-flood: sw_if_index 1 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 bd_index 1
+:00:01:190091: l2-output
+  l2-output: sw_if_index 2 dst 00:00:00:ff:02:08 src 00:00:00:ff:01:01 data 08 00 45 00 23 28 00 01 00 00 40 11
+:00:01:190108: pg1-output
+  pg1
+  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
+  UDP: 172.17.101.1 -> 172.17.102.8
+    tos 0x00, ttl 64, length 9000, checksum 0x3498
+    fragment id 0x0001
+  UDP: 11 -> 21541
+    length 8980, checksum 0xfa5e
+:00:01:190202: pg1-tx
+    buffer 0x7008: current data 0, length 9014, free-list 6, clone-count 0, trace 0x1
+  IP4: 00:00:00:ff:01:01 -> 00:00:00:ff:02:08
+  UDP: 172.17.101.1 -> 172.17.102.8
+    tos 0x00, ttl 64, length 9000, checksum 0x3498
+    fragment id 0x0001
+  UDP: 11 -> 21541
+    length 8980, checksum 0xfa5e
+</pre>
+== CLI ==
+The ACL plugin does not supply the "supported" debug CLI for configuration, but has the full support for talking to it via VAT CLI, which are documented below.
+Note from 2024: In contrast with the above statement there seems to be some debug CLI support introduced in https://gerrit.fd.io/r/c/vpp/+/5805 and https://gerrit.fd.io/r/c/vpp/+/8805 . Please see the following vppctl commands:
+ clear acl-plugin sessions
+ delete acl-plugin acl
+ set acl-plugin
+ set acl-plugin acl
+ set acl-plugin interface
+ set flow classify
+ set interface input acl
+ set interface output acl
+ show acl-plugin acl
+ show acl-plugin decode 5tuple
+ show acl-plugin interface
+ show acl-plugin lookup context
+ show acl-plugin lookup user
+ show acl-plugin macip acl
+ show acl-plugin macip interface
+ show acl-plugin memory
+ show acl-plugin sessions
+ show acl-plugin tables
+ show inacl
+ show outacl
+=== acl_plugin_get_version : get plugin version ===
+Show the version of the ACL plugin
+ vat# '''acl_plugin_get_version'''
+ vl_api_acl_plugin_get_version_reply_t_handler:133: ACL plugin version: 1.1
+ vat#
+=== acl_add_replace : add or replace an ACL ===
+Add a new ACL or replace the existing one.
+To replace an existing ACL, pass the ID of this ACL. If you prefer a new ACL to be created, pass the 0xffffffff (-1)
+Since the single API call defines the entire ACL, all of the entries of this ACL are specified on the same line,
+separated by commas.
+A quasi-BNF syntax of the VAT command line is below:
+acl_add_replace <acl-idx> [<ipv4|ipv6> <permit|permit+reflect|deny|action N> [src IP/plen] [dst IP/plen] [sport X-Y] [dport X-Y] [proto P] [tcpflags FL MASK], ... , ...
+An example of adding a single ACL permitting IPv4 and IPv6 is below:
+ vat# '''acl_add_replace permit, ipv6 permit'''
+ vl_api_acl_add_replace_reply_t_handler:107: ACL index: 0
+ vat#
+Replacing this ACL#0 by one with two host entries:
+ vat# '''acl_add_replace 0 ipv6 permit dst 2001:db8::1/128, ipv4 permit src 192.0.2.1/32'''
+ vl_api_acl_add_replace_reply_t_handler:107: ACL index: 0
+ vat#
+=== acl_del : delete an ACL ===
+Simply pass the ID of the ACL to delete it:
+ vat# acl_del 0
+ vat#
+=== acl_dump : dump ACL(s) ===
+Use the command without any arguments to dump all configured ACLs, or supply the optional ACL ID to dump
+only that ACL:
+ vat# '''acl_dump'''
+ vl_api_acl_details_t_handler:193: acl_index: 0, count: 2
+    tag {}
+    ipv6 action 1 src ::/0 dst 2001:db8::1/128 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 1.1.1.1/32 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
+ vl_api_acl_details_t_handler:193: acl_index: 1, count: 2
+    tag {}
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv6 action 1 src ::/0 dst ::/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
+ vl_api_acl_details_t_handler:193: acl_index: 2, count: 5
+    tag {}
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
+ vat# '''acl_dump 2'''
+ vl_api_acl_details_t_handler:193: acl_index: 2, count: 5
+    tag {}
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0,
+    ipv4 action 1 src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 0
+ vat#
+Notice that the output format of each ACL is (minus the newlines) suitable for feeding it as input to VAT to recreate that ACL.
+=== acl_interface_add_del (not recommended) : add/delete ACL to/from list of applied ACLs on an interface ===
+This command allows to append/delete one input or output ACL to the list of the ACLs applied to an interface.
+Because one would mostly want to predictably set the ACLs, the usage of this API is not recommended.
+The pseudo-BNF format is below:
+ acl_interface_add_del <intfc> | sw_if_index <if-idx> [add|del] [input|output] acl <acl-idx>
+=== acl_interface_set_acl_list : set the list of inbound+outbound ACLs for a given interface ===
+This command sets in one shot the list of input and output ACLs applied to the interface. It unapplies any previously applied ACLs,
+which makes this command idempotent.
+  acl_interface_set_acl_list <intfc> | sw_if_index <if-idx> input [acl-idx list] output [acl-idx list]
+An example:
+ vat# acl_interface_set_acl_list sw_if_index 0 input 0 output 0 1 2
+ vat#
+=== acl_interface_list_dump : show which interfaces have which lists of which ACLs applied ===
+ acl_interface_list_dump [<intfc> | sw_if_index <if-idx>]
+This command lists which ACLs are applied on which interface and in which direction.
+The below example is a result from setting the ACL on the interface using the previous command:
+ vat# acl_interface_list_dump
+ vl_api_acl_interface_list_details_t_handler:152: sw_if_index: 0, count: 4, n_input: 1
+    input 0
+   output 0 1 2
+ vat#
+=== macip_acl_add : add a MACIP ACL ===
+ macip_acl_add [ipv4|ipv6] [permit|deny|action N] [ip <ADDR>/<PREFIX-LEN> mac <MAC> mask <MAC-MASK>, ...
+ vat# '''macip_acl_add ipv4 permit ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask ff:ff:ff:ff:ff:ff, ipv6 permit ip 2001:db8::1/128  mac 00:01:02:03:04:05 mask 00:00:00:00:00:00'''
+ vl_api_macip_acl_add_reply_t_handler:107: ACL index: 7
+ vat#
+=== macip_acl_del : delete a MACIP ACL ===
+Delete the MACIP ACL by its index
+ vat# '''macip_acl_del 6'''
+ vat#
+=== macip_acl_dump ===
+Dump the MACIP ACLs (or just one if you supply its index)
+ vat# macip_acl_dump
+ vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 0, count: 0
+    tag {}
+ vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 1, count: 1
+    tag {}
+    ipv6 action 1 ip ::/0 mac 00:00:00:00:00:00 mask 00:00:00:00:00:00
+ vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 2, count: 1
+    tag {}
+    ipv6 action 1 ip ::/0 mac 00:00:00:00:00:00 mask 00:00:00:00:00:00
+ vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 3, count: 1
+    tag {}
+    ipv6 action 1 ip ::/0 mac 00:00:00:00:00:00 mask 00:00:00:00:00:00
+ vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 4, count: 1
+    tag {}
+    ipv4 action 1 ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask ff:ff:ff:ff:ff:ff
+ vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 5, count: 1
+    tag {}
+    ipv4 action 1 ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask 00:00:00:00:00:00
+ vl_api_macip_acl_details_t_handler:226: MACIP acl_index: 7, count: 2
+    tag {}
+    ipv4 action 1 ip 1.1.1.1/32 mac 00:01:02:03:04:05 mask ff:ff:ff:ff:ff:ff,
+    ipv6 action 1 ip 2001:db8::1/128 mac 00:01:02:03:04:05 mask 00:00:00:00:00:00
+ vat#
+=== macip_acl_interface_add_del: apply/unapply the MACIP ACL to/from a given interface ===
+ macip_acl_interface_add_del <intfc> | sw_if_index <if-idx> [add|del] acl <acl-idx>
+ vat# macip_acl_interface_add_del local0 add acl 7
+ vat#
+=== macip_acl_interface_get: get the list of interfaces and associated MACIP ACLs ===
+ vat# macip_acl_interface_add_del local0 add acl 7
+ vat# '''macip_acl_interface_get'''
+ vl_api_macip_acl_interface_get_reply_t_handler:241: sw_if_index with MACIP ACL count: 1
+   macip_acl_interface_add_del sw_if_index 0 add acl 7
+ vat# macip_acl_interface_add_del local0 del acl 7
+ vat# '''macip_acl_interface_get'''
+ vl_api_macip_acl_interface_get_reply_t_handler:241: sw_if_index with MACIP ACL count: 1
+   macip_acl_interface_add_del sw_if_index 0 add acl -1
+ vat#
+== Initial performance tests ==
+The first version of the ACL plugin was explicitly focused on getting things "correct" as the first priority,
+even if at some expense of getting them "fast". But understanding the performance is very important,
+so we did limited performance testing. The performance testing was done using MoonGen, running on the same host as VPP.
+The VPP was run by doing "make release-build; make release-plugins; make release-run" process. The VPP is configured via
+VAT.
+The performance was done by testing with 200000 unidirectional UDP streams, by submitting the line-rate 10Gbps of
+traffic by MoonGen (14.88Mpps of 64-byte packets) and observing the amount of traffic received on the other side.
+First, we test the baseline configuration which just doing briding between the two interfaces.
+  sw_interface_set_flags TenGigabitEthernet81/0/0 admin-up link-up
+  sw_interface_set_flags TenGigabitEthernet81/0/1 admin-up link-up
+  bridge_domain_add_del bd_id 42 flood 1 uu-flood 1 forward 1 learn 1 arp-term 0
+  sw_interface_set_l2_bridge TenGigabitEthernet81/0/0 bd_id 42
+  sw_interface_set_l2_bridge TenGigabitEthernet81/0/1 bd_id 42
+This configuration exhibited 9.52 Mpps performance.
+Then we added a trivial case of a one-line "permit" ACL which is checked on input and output of the packet path,
+using the following VAT commands:
+  acl_add_replace permit
+  acl_interface_add_del sw_if_index 1 add input acl 0
+  acl_interface_add_del sw_if_index 1 add output acl 0
+  acl_interface_add_del sw_if_index 2 add input acl 0
+  acl_interface_add_del sw_if_index 2 add output acl 0
+This configuration causes two very simple ACL checks - on input of the packet path and on the output, see the below trace(NB: this is a trace from 1701, the trace from 1704 looks different, though the logic is very similar. See the sample traces above):
+:02:12:167665: dpdk-input
+    TenGigabitEthernet81/0/0 rx queue 0
+    buffer 0x5358: current data 0, length 60, free-list 0, totlen-nifb 0, trace 0x0
+    PKT MBUF: port 0, nb_segs 1, pkt_len 60
+      buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 0x6ee4d640
+      packet_type 0x0
+      Packet Offload Flags
+    IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
+    UDP: 10.0.80.47 -> 10.1.0.10
+      tos 0x00, ttl 64, length 46, checksum 0x1686
+      fragment id 0x0000
+    UDP: 1234 -> 319
+      length 26, checksum 0x956f
+:02:12:167682: ethernet-input
+    IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
+:02:12:167691: l2-input
+    l2-input: sw_if_index 1 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06
+:02:12:167693: l2-input-classify
+    l2-classify: sw_if_index 1, table 0, offset 0, next 9
+:02:12:167700: acl-plugin-in
+    ACL_IN: sw_if_index 1, next index 10, match: inacl 0 rule 0 trace_bits 00000000
+:02:12:167709: l2-learn
+    l2-learn: sw_if_index 1 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06 bd_index 1
+:02:12:167712: l2-flood
+    l2-flood: sw_if_index 1 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06 bd_index 1
+:02:12:167714: l2-output
+    l2-output: sw_if_index 2 dst 07:08:09:0a:0b:0c src 01:02:03:04:05:06
+:02:12:167716: l2-output-classify
+    l2-classify: sw_if_index 2, table 6, offset 0, next 5
+:02:12:167723: acl-plugin-out
+    ACL_OUT: sw_if_index 2, next index 4, match: outacl 0 rule 0 trace_bits 00000000
+:02:12:167734: TenGigabitEthernet81/0/1-output
+    TenGigabitEthernet81/0/1
+    IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
+    UDP: 10.0.80.47 -> 10.1.0.10
+      tos 0x00, ttl 64, length 46, checksum 0x1686
+      fragment id 0x0000
+    UDP: 1234 -> 319
+      length 26, checksum 0x956f
+:02:12:167736: TenGigabitEthernet81/0/1-tx
+    TenGigabitEthernet81/0/1 tx queue 0
+    buffer 0x5358: current data 0, length 60, free-list 0, totlen-nifb 0, trace 0x0
+    IP4: 01:02:03:04:05:06 -> 07:08:09:0a:0b:0c
+    UDP: 10.0.80.47 -> 10.1.0.10
+      tos 0x00, ttl 64, length 46, checksum 0x1686
+      fragment id 0x0000
+    UDP: 1234 -> 319
+      length 26, checksum 0x956f
+This configuration exhibited the performance of 4.62 Mpps.
+To test the impact of the linear match, we add lines to ACL one by one:
+  acl_add_replace 0 permit src 1.1.1.1/32,permit
+performance: 4.50Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32,permit
+performance: 4.34 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32,permit
+performance: 4.19 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, permit
+performance: 4.04 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, permit
+performance: 3.90 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, permit
+performance: 3.77 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32,  permit
+performance: 3.64 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, permit
+performance: 3.55 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, permit
+performance: 3.23 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, permit
+performance: 3.33 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, permit
+performance: 3.24 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, permit
+performance: 3.14 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, src 1.1.1.13/32, permit
+performance: 3.06 Mpps
+  acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, src 1.1.1.13/32, src
+.1.1.14/32, permit
+performance: 2.85 Mpps
+We can see that the performance impact is not strictly linear, but it gives the impression.
+Changing the last match to "permit and reflect" in order to cause the stateful processing path:
+acl_add_replace 0 permit src 1.1.1.1/32,src 1.1.1.2/32, src 1.1.1.3/32, src 1.1.1.4/32, src 1.1.1.5/32, src 1.1.1.6/32, src 1.1.1.7/32, src 1.1.1.8/32, src 1.1.1.9/32, src 1.1.1.10/32, src 1.1.1.11/32, src 1.1.1.12/32, src 1.1.1.13/32, src 1.1.1.14/32, permit+reflect
+performance: 3.68 Mpps
+We can see that the performance of the stateful path is less than the performance of the stateless path with small ACLs - this is most probably due to the current code organization of session tracker nodes, which use some code sharing between the nodes to ease the maintenance of it.
+Removing all ACLs:
+  acl_interface_add_del sw_if_index 1 del input acl 0
+  acl_interface_add_del sw_if_index 1 del output acl 0
+  acl_interface_add_del sw_if_index 2 del input acl 0
+  acl_interface_add_del sw_if_index 2 del output acl 0
+We get to 9.47 Mpps.
+Let's test multiple ACL matching:
+  acl_add_replace src 1.1.1.1/32
+  acl_add_replace src 1.1.1.1/32
+  acl_add_replace src 1.1.1.1/32
+  acl_add_replace src 1.1.1.1/32
+  acl_add_replace src 1.1.1.1/32
+First same condition as before, with just one ACL check:
+  acl_interface_set_acl_list sw_if_index 1 input 0 output 0
+  acl_interface_set_acl_list sw_if_index 2 input 0 output 0
+performance: 4.67 Mpps
+Two ACLs:
+  acl_interface_set_acl_list sw_if_index 1 input 1 0 output 1 0
+  acl_interface_set_acl_list sw_if_index 2 input 1 0 output 1 0
+performance: 4.16 Mpps
+Three ACLs:
+  acl_interface_set_acl_list sw_if_index 1 input 2 1 0 output 2 1 0
+  acl_interface_set_acl_list sw_if_index 2 input 2 1 0 output 2 1 0
+performance: 3.73 Mpps
+Four ACLs:
+  acl_interface_set_acl_list sw_if_index 1 input 3 2 1 0 output 3 2 1 0
+  acl_interface_set_acl_list sw_if_index 2 input 3 2 1 0 output 3 2 1 0
+performance: 3.38 Mpps
+Five ACLs:
+  acl_interface_set_acl_list sw_if_index 1 input 4 3 2 1 0 output 4 3 2 1 0
+  acl_interface_set_acl_list sw_if_index 2 input 4 3 2 1 0 output 4 3 2 1 0
+Performance: 3.10 Mpps
+Six ACLs:
+  acl_interface_set_acl_list sw_if_index 1 input 5 4 3 2 1 0 output 5 4 3 2 1 0
+  acl_interface_set_acl_list sw_if_index 2 input 5 4 3 2 1 0 output 5 4 3 2 1 0
+Performance: 2.85 Mpps
+Convert to last "permit" to stateful:
+  acl_add_replace 0 permit+reflect
+Performance: 3.67 Mpps
+These two performance axis might combine 0 i.e. if the ACL 5 in the above test were to be long and not match,
+the performance will be  worse than just with that ACL and worse than with 6 trivial ACLs.
+Removing all ACLs:
+  acl_interface_set_acl_list sw_if_index 1
+  acl_interface_set_acl_list sw_if_index 2
+The performance is 9.45 Mpps.
+MACIP ACLs
+  macip_acl_add permit
+  macip_acl_interface_add_del sw_if_index 1 add acl 0
+Performance: 7.30 Mpps
+  macip_acl_add permit ip 128.1.0.0/7, permit ip 10.0.0.0/8
+  macip_acl_interface_add_del sw_if_index 1 add acl 1
+Performance: 7.31 Mpps (the hit on the first classify table)
+  macip_acl_add permit ip 128.1.0.0/9, permit ip 10.0.0.0/8
+  macip_acl_interface_add_del sw_if_index 1 add acl 1
+Performance: 7.31 Mpps (the hit on the first classify table)
+(more testing with MACIP ACLs should be done)
 == Requirements ==
@@ Line 33: / Line 722: @@
 | Ole
 | 0
-| WIP
+| Done
 | [https://jira.fd.io/browse/VPP-513 VPP-513]
 |-
@@ Line 39: / Line 728: @@
 | Andrew
 | 0
-| WIP
+| Done
 | [https://jira.fd.io/browse/VPP-514 VPP-514]
 |-
@@ Line 78: / Line 767: @@
 a. IPv4 matching in all plugin -> Andrew - done.
 b. make it “deny by default” -> Andrew - done.
-. Performance testing -> Pavel ?
+c. port range support -> Andrew - done.
+d. ICMP type/code matching -> Andrew - done.
+. Performance testing -> Andrew - done.
    --- MVP ---
 a. Plumbing for stateful sessions from ACL plugin (to be able to specify “match and track” (“permit and create the forward/return session”) -> Andrew - done.
-b. Stateful session tracking -> Andrew
+b. Stateful session tracking - timeouts -> Andrew - Done.
-. L2 rules - TBD
+c. Stateful session tracking - lightweight TCP state -> Andrew - Done
-. ACL/Sessions support for L3 (routed) mode - (big)!
+. MACIP(L2) rules -> Andrew - done.
-. Can we implement the ACL match purely in terms of classifier tables ? How expensive/(in)efficient that would be ?
+. Code cleanups -> Andrew
-. Extension header handling during the slow path lookup - easy in ACL plugin
-. classifier match for the sessions with extension headers - currently no extension headers supported
+  PHASE2:
+  A. ACL/Sessions support for L3 (routed) mode - (big) !
+   B. Can we implement the ACL match purely in terms of classifier tables ? How expensive/(in)efficient that would be ?
+   C. Extension header handling during the slow path lookup - easy in ACL plugin
+   D. classifier match for the sessions with extension headers - currently no extension headers supported
 == API ==
-[https://github.com/vpp-dev/vpp/blob/acl/plugins/acl-plugin/acl/acl.api API file as implemented]
+[https://gerrit.fd.io/r/gitweb?p=vpp.git;a=blob;f=plugins/acl-plugin/acl/acl.api;h=58a5a17180e087efec829621368b2111e406a22e;hb=refs/heads/stable/1701 API file as implemented in 17.01]
 == MACIP (formerly "L2") API ==
@@ Line 99: / Line 795: @@
 The use of this mechanism is to prevent spoofing.
-[https://github.com/vpp-dev/vpp/blob/acl/plugins/acl-plugin/acl/acl.api#L133-L191 API file as implemented]
+[https://gerrit.fd.io/r/gitweb?p=vpp.git;a=blob;f=plugins/acl-plugin/acl/acl.api;h=58a5a17180e087efec829621368b2111e406a22e;hb=refs/heads/stable/1701#l308 API file as implemented]
 API as implemented supports MAC address masks and prefixes, however, be aware: the current implementation is done using chained classifier tables,
@@ Line 110: / Line 806: @@
 == Design and prototyping ==
-The stateful design is being prototyped in https://github.com/vpp-dev/vpp-lua-plugin/blob/master/samples/polua.lua
+The ACL matching is implemented in this phase as a simple array search, under the assumption that given the rules are per-port, the rule list will be small.
-The goal for this prototype was to minimize the amount of changes to the main forwarding path and explore for the later possible optimizations.
-Also one of the primary design criteria is to avoid creating a separate forwarding path as much as possible.
-The main idea with the stateful design is to use the L2 classifier for storing the sessions.
-For this, we create two chained tables per interface per direction: TCP/UDP then ICMP, and hook them into the processing path of the packet.
-If the session is not in the table, it means we need to do the policy check - thus the miss_next index of the ICMP table is set to one of the nodes taking care of the policy checks: there are four of them because of {ingress/egress, ip4/ip6}.
-Each of the nodes is very simple: it checks the policy and if the policy permits the packet, then it adds the session and recirculates the packet
-back into the lookup - then that packet will hit the session and be processed by "fast path".
-For the purposes of this document we refer to the policy check path as "slow path" and the path using the established state as "fast path" even as you see if it is a bit of a misnomer - the "slow path" does not really pass the packets through the box, it merely sets up the fast path and recirculates the packets to hit it.
-Besides adding the forward flow, if there is a policy in the reverse direction, then the slow path also sets up the mirror flow in the tables of the opposite direction - so as to avoid having to do the policy check for the return packets of the flow. The only type of ICMP that are considered to have the "return" packets are echo/echo-reply.
-When the ingress packet processing is done, the forwarding is done as usual by VPP, and then the similar check against the flow table is done on egress in the l2-output-lookup - if there is a policy applied. Again, the missing session results to a redirect to a "slow path" node, which inserts a session and a return session, and recirculates the packet.
-This highlights a particularity - if there is a policy in one direction that is other than "permit everything" and has some deny rules, then for the proper functioning, there needs to be a "permit everything" policy applied in the opposite direction on the same interface - so that the return packets did not hit the policy lookup. However, this can be easily hidden from the user by implementation, so is probably not a big problem.
-However, some more distinct shortcomings:
-) not very frugal about the memory. With policies applied, each connection consumes 4 session slots. How bad is it ? Certainly not very cache friendly. On the other hand this kind of approach could handle in the future even the features that change the packet.
-An idea for possible optimizations: use the same tables for in+out on the same interface, and insert two sessions. This might be questionable from the security standpoint in some corner cases.
-) no TCP state tracking nor UDP timeout tracking.
-) No any cleanup at all for the classifier tables. Only additions are performed.
-this MUST be taken care of and is TBD. Note that it is intentionally separate from (2), because it covers the scenarios like just simple high resource utilization as well.
-) No support for IPv6 EH or IPv4 fragments. This is a general issue with using the "simple" bitmask/match type of classifier, and so far the solution is is TBD.
-) two checks of policy as opposed to one.
-An idea for possible optimization: have the ingress path merely set the flag "need to check the policy" and perform all the policy checking on egress - including the flow - thus combining both of the policies lookup. If the packets are guaranteed to never be translated, then this can be a possible strategy. However, this means an activation of ingress policy MUST trigger the activation of the check on egress on all the interfaces within the forwarding domain. And if the egress interface has a deny-something in the inbound direction, then still a reverse flow check must be done.
-A possible optimized implementation that will take care to some extent of (1) and (5):
-Perform a flow check on ingress.
-If the flow exists, retrieve the egress interface from the flow record, and mark the packet "check the TX interface before output".
-If the flow does not exist, mark the packet "Check the policy", and forward as usual.
-On egress, if "Check the TX interface" is set - verify that the previously saved TX interface matches.
-if "Check the policy" matches, then check the policy - if it does not permit the traffic then drop the packet.
-Else, create an inbound flow using a mirrored information from the packet, and send the packet along.
-As an example of how it is prototyped today, below goes the example trace of the ICMP flow permitted by the policy - first and second packet.
-<pre>
-------------------- Start of thread 0 vpp_main -------------------
-Packet 1
-:12:27:661397: af-packet-input
-  af_packet: hw_if_index 1 next-index 1
-    tpacket2_hdr:
-      status 0x20000001 len 98 snaplen 98 mac 66 net 80
-      sec 0x5804fd0a nsec 0x2660d6db vlan 0
-:12:27:661438: ethernet-input
-  IP4: 7a:01:9a:05:7b:b7 -> ca:52:50:fb:e5:82
-:12:27:661449: l2-input
-  l2-input: sw_if_index 1 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7
-:12:27:661455: l2-input-classify
-  l2-classify: sw_if_index 1, table 18, offset 0, next 16
-:12:27:661462: lua-polua-ip4-input
-  LUA_plugin: sw_if_index 1, next index 1
-:12:27:661592: l2-input-classify
-  l2-classify: sw_if_index 1, table 18, offset c0, next 11
-:12:27:661595: l2-learn
-  l2-learn: sw_if_index 1 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7 bd_index 1
-:12:27:661602: l2-fwd
-  l2-fwd:   sw_if_index 1 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7 bd_index 1
-:12:27:661607: l2-output
-  l2-output: sw_if_index 2 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7
-:12:27:661611: l2-output-classify
-  l2-classify: sw_if_index 2, table 22, offset 0, next 6
-:12:27:661616: lua-polua-ip4-output
-  LUA_plugin: sw_if_index 1, next index 1
-:12:27:661718: l2-output-classify
-  l2-classify: sw_if_index 2, table 22, offset c0, next 9
-:12:27:661727: host-s0_s2-output
-  host-s0_s2
-  IP4: 7a:01:9a:05:7b:b7 -> ca:52:50:fb:e5:82
-  ICMP: 192.0.2.1 -> 192.0.2.2
-    tos 0x00, ttl 64, length 84, checksum 0x37d5
-    fragment id 0x7ed0, flags DONT_FRAGMENT
-  ICMP echo_request checksum 0x8b86
-Packet 2
-:12:27:661784: af-packet-input
-  af_packet: hw_if_index 2 next-index 1
-    tpacket2_hdr:
-      status 0x20000001 len 98 snaplen 98 mac 66 net 80
-      sec 0x5804fd0a nsec 0x2660d6db vlan 0
-:12:27:661790: ethernet-input
-  IP4: ca:52:50:fb:e5:82 -> 7a:01:9a:05:7b:b7
-:12:27:661795: l2-input
-  l2-input: sw_if_index 2 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82
-:12:27:661798: l2-input-classify
-  l2-classify: sw_if_index 2, table 26, offset c0, next 11
-:12:27:661802: l2-learn
-  l2-learn: sw_if_index 2 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82 bd_index 1
-:12:27:661803: l2-fwd
-  l2-fwd:   sw_if_index 2 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82 bd_index 1
-:12:27:661807: l2-output
-  l2-output: sw_if_index 1 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82
-:12:27:661809: host-s0_s1-output
-  host-s0_s1
-  IP4: ca:52:50:fb:e5:82 -> 7a:01:9a:05:7b:b7
-  ICMP: 192.0.2.2 -> 192.0.2.1
-    tos 0x00, ttl 64, length 84, checksum 0x9113
-    fragment id 0x6592
-  ICMP echo_reply checksum 0x9386
-Packet 3
-:12:28:663937: af-packet-input
-  af_packet: hw_if_index 1 next-index 1
-    tpacket2_hdr:
-      status 0x20000001 len 98 snaplen 98 mac 66 net 80
-      sec 0x5804fd0b nsec 0x275a0081 vlan 0
-:12:28:664014: ethernet-input
-  IP4: 7a:01:9a:05:7b:b7 -> ca:52:50:fb:e5:82
-:12:28:664030: l2-input
-  l2-input: sw_if_index 1 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7
-:12:28:664040: l2-input-classify
-  l2-classify: sw_if_index 1, table 18, offset c0, next 11
-:12:28:664053: l2-learn
-  l2-learn: sw_if_index 1 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7 bd_index 1
-:12:28:664060: l2-fwd
-  l2-fwd:   sw_if_index 1 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7 bd_index 1
-:12:28:664071: l2-output
-  l2-output: sw_if_index 2 dst ca:52:50:fb:e5:82 src 7a:01:9a:05:7b:b7
-:12:28:664078: l2-output-classify
-  l2-classify: sw_if_index 2, table 22, offset c0, next 9
-:12:28:664086: host-s0_s2-output
-  host-s0_s2
-  IP4: 7a:01:9a:05:7b:b7 -> ca:52:50:fb:e5:82
-  ICMP: 192.0.2.1 -> 192.0.2.2
-    tos 0x00, ttl 64, length 84, checksum 0x3797
-    fragment id 0x7f0e, flags DONT_FRAGMENT
-  ICMP echo_request checksum 0xc145
-Packet 4
-:12:28:664164: af-packet-input
-  af_packet: hw_if_index 2 next-index 1
-    tpacket2_hdr:
-      status 0x20000001 len 98 snaplen 98 mac 66 net 80
-      sec 0x5804fd0b nsec 0x275a0081 vlan 0
-:12:28:664172: ethernet-input
-  IP4: ca:52:50:fb:e5:82 -> 7a:01:9a:05:7b:b7
-:12:28:664179: l2-input
-  l2-input: sw_if_index 2 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82
-:12:28:664184: l2-input-classify
-  l2-classify: sw_if_index 2, table 26, offset c0, next 11
-:12:28:664191: l2-learn
-  l2-learn: sw_if_index 2 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82 bd_index 1
-:12:28:664193: l2-fwd
-  l2-fwd:   sw_if_index 2 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82 bd_index 1
-:12:28:664198: l2-output
-  l2-output: sw_if_index 1 dst 7a:01:9a:05:7b:b7 src ca:52:50:fb:e5:82
-:12:28:664201: host-s0_s1-output
-  host-s0_s1
-  IP4: ca:52:50:fb:e5:82 -> 7a:01:9a:05:7b:b7
-  ICMP: 192.0.2.2 -> 192.0.2.1
-    tos 0x00, ttl 64, length 84, checksum 0x904b
-    fragment id 0x665a
-  ICMP echo_reply checksum 0xc945
-</pre>
-== CLI ==
- set interface input acl intfc <int> [ip4-table <index>] [ip6-table <index>] [l2-table <index>] [del]
- show inacl type [ip4|ip6|l2]
- classify table [miss-next|l2-miss_next|acl-miss-next <next_index>] mask <mask-value> buckets <nn> [skip <n>] [match <n>] [del]
- show classify tables [index <nn>]
- classify session [hit-next|l2-hit-next|acl-hit-next <next_index>|policer-hit-next <policer_name>] table-index <nn> match [hex] [l2] [l3 ip4] [opaque-index <index>]
- test classify [src <ip>] [sessions <nn>] [buckets <nn>] [table <nn>] [del]
- set ip classify intfc <int> table-index <index>
- set interface ip6 table <intfc> <table-id>
- set interface l2 input classify intfc <interface-name> [ip4-table <n>] [ip6-table <n>] [other-table <n>]
- set interface l2 output classify intfc <<interface-name>> [ip4-table <n>] [ip6-table <n>] [other-table <n>]
+The redirection of the traffic to the node performing the ACL match is done by installing an empty L2 classifier table whose "miss-next" index diverts the traffic
+to the node.
- set ip source-and-port-range-check
+The ACL match node can also redirect the traffic to the stateful-session setup node (by having a "permit" = 2 in the ACE), which will create the session on that interface.
- show ip source-and-port-range-check vrf <nn> <ip-addr> <port>
+...TBD: more details...
 == Examples ==
@@ Line 320: / Line 821: @@
 == Open Issues ==
-* Security Group use case specific API. Done in VPP or control plane plugin?
+== Closed Issues ==
+* Security Group use case specific API. Done in a plugin.
 == Existing functionality ==

Difference between revisions of "VPP/SecurityGroups"