Difference between revisions of "CSIT/VPP-16.06 Test Report Sandbox"

From fd.io
Jump to: navigation, search
Line 36: Line 36:
 
==Functional tests results==
 
==Functional tests results==
 
Dump of a vpp-csit-verify-virl-1606 job from console in ASCII.
 
Dump of a vpp-csit-verify-virl-1606 job from console in ASCII.
 +
 +
===Tests===
 +
===Suites===
 +
===Bridge Domain===
 +
===Bridge Domain Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|TC01: DUT reports active interfaces
 +
|Report VPP interfaces on DUT
 +
|
 +
|PASS
 +
|-
 +
|TC02: DUT with L2BD (MAC learning) switch ICMPv4 between two TG links
 +
|TG-DUT Eth-IPv4-ICMPv4: On DUT configure L2 bridge-domain
 +
(MAC learning enabled), with two interfaces to TG. Make
 +
TG send ICMPv4 Echo Req in both directions between two of
 +
its interfaces to be switched by DUT, verify all packets
 +
are received.
 +
|
 +
|PASS
 +
|-
 +
|TC03: DUT1 and DUT2 with L2BD (MAC learning) switch between two TG links
 +
|Eth-IPv4-ICMPv4 on links DUT1-DUT2, TG-DUT1, TG-DUT2: On
 +
DUT1 and DUT2 configure L2 bridge-domain (MAC learning
 +
enabled), each with one interface to TG and one Ethernet
 +
interface towards the other DUT. Make TG send ICMPv4 Echo
 +
Req in both directions between two of its interfaces to
 +
be switched by DUT1 and DUT2, verify all packets are
 +
received.
 +
|
 +
|PASS
 +
|-
 +
|TC04: DUT1 and DUT2 with L2BD (static MACs) switch between two TG links
 +
|Eth-IPv4-ICMPv4 on links DUT1-DUT2, TG-DUT1, TG-DUT2: On
 +
DUT1 and DUT2 configure L2 bridge-domain (static MAC
 +
entries), each with one interface to TG and one Ethernet
 +
interface towards the other DUT. Make TG send ICMPv4 Echo
 +
Req in both directions between two of its interfaces to
 +
be switched by DUT1 and DUT2, verify all packets are
 +
received.
 +
|
 +
|PASS
 +
|-
 +
|TC05: DUT1 and DUT2 with L2BD (MAC learn) and SHG switch between four TG links
 +
|Eth-IPv4-ICMPv4 on links DUT1-DUT2, TG-DUT1, TG-DUT2: On
 +
DUT1 and DUT2 configure L2 bridge-domain (MAC learning
 +
enabled), each with two untagged interfaces to TG and
 +
untagged interface towards the other DUT. On each DUT put
 +
both interfaces towards TG into the same
 +
Split-Horizon-Group (SHG). Make TG send ICMPv4 Echo Reqs
 +
between all four of its interfaces to be switched by DUT1
 +
and DUT2, and verify packets are not switched between TG
 +
interfaces isolated by SHG on DUT1 and DUT2.
 +
|
 +
|PASS
 +
|-
 +
|TC06: DUT with two L2BDs (MAC learn) switches ICMPv4 between TG and VM links
 +
|Eth-IPv4-ICMPv4 on links DUT-VM, TG-DUT: On DUT configure
 +
two L2 bridge-domains (MAC learning enabled), each with
 +
one untagged interface to TG and untagged interface to
 +
local VM over vhost-user. Make TG send ICMPv4 Echo Reqs
 +
in both directions between two of its interfaces to be
 +
switched by DUT to and from VM, and verify all packets
 +
are received.
 +
|
 +
|PASS
 +
|-
 +
|TC07: DUT with two L2BDs (MAC learn) switches ICMPv6 between TG and VM links
 +
|Eth-IPv6-ICMPv6 on links DUT-VM, TG-DUT: On DUT configure
 +
two L2 bridge-domains (MAC learning enabled), each with
 +
one untagged interface to TG and untagged interface to
 +
local VM over vhost-user. Make TG send ICMPv6 Echo Reqs
 +
in both directions between two of its interfaces to be
 +
switched by DUT to and from VM, and verify all packets
 +
are received.
 +
|
 +
|PASS
 +
|-
 +
|TC08: DUT with two L2BDs (static MACs) switches ICMPv4 between TG and VM links
 +
|Eth-IPv6-ICMPv4 on links DUT-VM, TG-DUT: On DUT configure
 +
two L2 bridge-domains (static MAC entries), each with one
 +
untagged interface to TG and untagged interface to local
 +
VM over vhost-user. Make TG send ICMPv4 Echo Reqs in both
 +
directions between two of its interfaces to be switched
 +
by DUT to and from VM, and verify all packets are
 +
received.
 +
|
 +
|PASS
 +
|-
 +
|TC09: DUT with two L2BDs (static MACs) switches ICMPv6 between TG and VM links
 +
|Eth-IPv6-ICMPv6 on links DUT-VM, TG-DUT: On DUT configure
 +
two L2 bridge-domains (static MAC entries), each with one
 +
untagged interface to TG and untagged interface to local
 +
VM over vhost-user. Make TG send ICMPv6 Echo Reqs in both
 +
directions between two of its interfaces to be switched
 +
by DUT to and from VM, and verify all packets are
 +
received.
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Cop===
 +
===Cop Whitelist Blacklist===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP permits packets based on IPv4 src addr
 +
|COP Whitelist test with basic setup.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 src addr
 +
|COP blacklist test with basic setup.
 +
|
 +
|PASS
 +
|}
 +
===cop whitelist blacklist IPv6===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP permits packets based on IPv6 src addr
 +
|COP Whitelist test with basic setup.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 src addr
 +
|COP blacklist test with basic setup.
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Dhcp===
 +
===Dhcp Client===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP sends a DHCP DISCOVER
 +
|Configure DHCP client on interface to TG without hostname
 +
and check if DHCP DISCOVER message contains all required
 +
fields with expected values.
 +
 +
|
 +
|PASS
 +
|-
 +
|VPP sends a DHCP DISCOVER with hostname
 +
|Configure DHCP client on interface to TG with hostname
 +
and check if DHCP DISCOVER message contains all required
 +
fields with expected values.
 +
 +
|
 +
|PASS
 +
|-
 +
|VPP sends DHCP REQUEST after OFFER
 +
|Configure DHCP client on interface to TG and check if
 +
DHCP REQUEST message contains all required fields.
 +
 +
|
 +
|PASS
 +
|-
 +
|VPP doesn't send DHCP REQUEST after OFFER with wrong XID
 +
|Configure DHCP client on interface to TG. If server sends
 +
DHCP OFFER with different XID as in DHCP DISCOVER,
 +
DHCP REQUEST message shouldn't be sent.
 +
 +
|Expected error 'DHCP REQUEST Rx timeout' but got 'Traffic script execution failed'.
 +
|FAIL
 +
|-
 +
|VPP honors DHCPv4 lease time
 +
|Send IP configuration to the VPP client via DHCP. Address
 +
is checked with ICMP echo request and there should be no
 +
reply for echo request when lease has expired.
 +
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Fds Related Tests===
 +
===Provider Network===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|Provider network test cases with provider physical networks (VLAN)
 +
|Ping among all ports inside the same network should pass.
 +
a) test l2 connectivity inside every network
 +
b) test l2 connectivity between networks
 +
|
 +
|PASS
 +
|}
 +
===Tenant Network===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|L2 test cases with tenant networks (VXLAN)
 +
|Ping among all ports inside the same network should pass.
 +
a) test l2 connectivity inside every network
 +
b) test l2 connectivity between networks
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Gre===
 +
===Gre Encapsulation===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP can route IPv4 traffic from GRE tunnel
 +
|Create GRE configuration on 2 DUTs, send IP traffic and
 +
check IP is not changed.
 +
|
 +
|PASS
 +
|-
 +
|VPP can encapsulate IPv4 traffic in GRE
 +
|Create GRE configuration on DUT, send IP traffic and
 +
check IP is correctly encapsulate into GRE.
 +
|
 +
|PASS
 +
|-
 +
|VPP can decapsulate IPv4 traffic in GRE
 +
|Create GRE configuration on DUT, send GRE encapsulated
 +
traffic and check IP is received without GRE headers.
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Ipv4===
 +
===Ipv4 Arp Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP sends ARP requests for unknown destinations
 +
|Setup IP addresses and route.
 +
Send ICMP packet and check if VPP sends ARP request.
 +
|
 +
|PASS
 +
|}
 +
===Ipv4 Iacl Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP drops packets based on IPv4 source addresses
 +
|Create classify table on VPP, add source IP address
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 destination addresses
 +
|Create classify table on VPP, add destination IP address
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 src-addr and dst-addr
 +
|Create classify table on VPP, add source and destination
 +
IP address of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 protocol (TCP)
 +
|Create classify table on VPP, add mask for TCP port
 +
into table and setup 'deny' traffic
 +
and check if TCP traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 protocol (UDP)
 +
|Create classify table on VPP, add mask for UDP port
 +
into table and setup 'deny' traffic
 +
and check if UDP traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 TCP src ports
 +
|Create classify table on VPP, add source TCP port
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 TCP dst ports
 +
|Create classify table on VPP, add destination TCP port
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 TCP src + dst ports
 +
|Create classify table on VPP, add source and destination
 +
TCP port of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 UDP src ports
 +
|Create classify table on VPP, add source UDP port
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 UDP dst ports
 +
|Create classify table on VPP, add destination UDP port
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv4 UDP src + dst ports
 +
|Create classify table on VPP, add source and destination
 +
UDP port of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|}
 +
===Ipv4 Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP replies to ICMPv4 echo request
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to DUT egress interface
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to DUT2 through DUT1
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to DUT2 egress interface through DUT1
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to TG through DUT1 and DUT2
 +
|
 +
|
 +
|PASS
 +
|-
 +
|VPP can process ICMP echo request from min to 1500B packet size with 1B increment
 +
|
 +
|
 +
|PASS
 +
|-
 +
|VPP responds to ARP request
 +
|
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Ipv6===
 +
===Ipv6 Iacl Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP drops packets based on IPv6 source addresses
 +
|Create classify table on VPP, add source IP address
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 destination addresses
 +
|Create classify table on VPP, add destination IP address
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 src-addr and dst-addr
 +
|Create classify table on VPP, add source and destination
 +
IP address of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 protocol (TCP)
 +
|Create classify table on VPP, add mask for TCP port
 +
into table and setup 'deny' traffic
 +
and check if TCP traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 protocol (UDP)
 +
|Create classify table on VPP, add mask for UDP port
 +
into table and setup 'deny' traffic
 +
and check if UDP traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 TCP src ports
 +
|Create classify table on VPP, add source IP address
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 TCP dst ports
 +
|Create classify table on VPP, add destination TCP port
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 TCP src + dst ports
 +
|Create classify table on VPP, add source and destination
 +
TCP port of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 UDP src ports
 +
|Create classify table on VPP, add source UDP port
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 UDP dst ports
 +
|Create classify table on VPP, add destination UDP port
 +
of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|-
 +
|VPP drops packets based on IPv6 UDP src + dst ports
 +
|Create classify table on VPP, add source and destination
 +
UDP port of traffic into table and setup 'deny' traffic
 +
and check if traffic is dropped.
 +
|
 +
|PASS
 +
|}
 +
===Ipv6 Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP replies to ICMPv6 echo request
 +
|
 +
|
 +
|PASS
 +
|-
 +
|VPP can process ICMPv6 echo request from min to 1500B packet size with 1B increment
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to first DUT egress interface
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to second DUT through first DUT
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to second DUT egress interface through first DUT
 +
|
 +
|
 +
|PASS
 +
|-
 +
|TG can route to TG through first and second DUT
 +
|
 +
|
 +
|PASS
 +
|-
 +
|VPP replies to IPv6 Neighbor Solicitation
 +
|
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===L2 Xconnect===
 +
===L2 Xconnect Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|Vpp forwards ICMPv4 packets via L2 xconnect in circular topology
 +
|Setup single link path with X-connect
 +
and send ICMPv4 packet.
 +
|
 +
|PASS
 +
|-
 +
|Vpp forwards ICMPv6 packets via L2 xconnect in circular topology
 +
|Setup single link path with X-connect
 +
and send ICMPv6 packet.
 +
|
 +
|PASS
 +
|-
 +
|VPP forwards ICMPv4 packets through VM via L2 x-connect
 +
|Setup double link path with X-connect via Vhost user
 +
and send ICMPv4 packet.
 +
|
 +
|PASS
 +
|-
 +
|VPP forwards ICMPv6 packets through VM via L2 x-connect
 +
|Setup double link path with X-connect via Vhost user
 +
and send ICMPv6 packet.
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Lisp===
 +
===Lisp Api Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP can add and delete locator_set
 +
|Test lisp locator_set API
 +
Set locator_set and locator on the VPP node,
 +
check the configured data and then remove it.
 +
Check if all locator_set and locators was unset
 +
from the VPP node.
 +
 +
|
 +
|PASS
 +
|-
 +
|VPP can add, reset and delete locator_set
 +
|Test lisp locator_set API
 +
Set locator_set and locator on the VPP node,
 +
then reset locator_set and set it again.
 +
Check the configured data and then remove it.
 +
Check if all locator_set and locators was unset
 +
from the VPP node.
 +
 +
|
 +
|PASS
 +
|-
 +
|Vpp can add and delete eid address
 +
|Test lisp eid API
 +
Set lisp eid IP address on the VPP node,
 +
check the configured data and then remove it.
 +
Check if all eid IP address was unset
 +
from the VPP node.
 +
 +
|
 +
|PASS
 +
|-
 +
|Vpp can add and delete lisp map resolver address
 +
|Test lisp map resolver address API
 +
Set lisp map resolver address on the VPP node,
 +
check the configured data and then remove it.
 +
Check if all map resolver address was unset
 +
from the VPP node.
 +
 +
|
 +
|PASS
 +
|}
 +
===Lisp Dataplane Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP can pass IPv4 bidirectionally through LISP
 +
|Test IP4 Lisp remote static mapping.
 +
Set IP4 lisp topology and check if packet passes through
 +
Lisp topology.
 +
|
 +
|PASS
 +
|-
 +
|VPP can pass IPv6 bidirectionally through LISP
 +
|Test IP6 Lisp remote static mapping.
 +
Set IP6 lisp topology and check if packet passes through
 +
Lisp topology.
 +
|
 +
|PASS
 +
|-
 +
|VPP can pass IPv4 over IPv6 bidirectionally through LISP
 +
|Test IP4 over IP6 in Lisp remote static mapping.
 +
Set IP6 topology and check if the IP4 packet
 +
passes through IP6 Lisp topology.
 +
|
 +
|PASS
 +
|-
 +
|VPP can pass IPv6 over IPv4 bidirectionally through LISP
 +
|Test IP6 over IP4 in Lisp remote static mapping.
 +
Set IP4 topology and check if the IP6 packet
 +
passes through IP4 Lisp topology.
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Tagging===
 +
===Qinq L2 Xconnect===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP can push and pop two VLAN tags to traffic transferring through xconnect
 +
|Push two tags on DUT 1 to traffic sent from TG,
 +
pop two tags on DUT 2 from this traffic
 +
and receive untagged traffic on TG.
 +
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
===Vxlan===
 +
===Vxlan Bd Dot1Q===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP can encapsulate L2 in VXLAN over IPv4 over Dot1Q
 +
|
 +
|
 +
|PASS
 +
|}
 +
===Vxlan Bd Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP can pass IPv4 bidirectionally through VXLAN using bridge domain
 +
|Create VXLAN interface on both VPP nodes. Create one
 +
bridge domain (learning enabled) on both VPP nodes, add
 +
VXLAN interface and interface toward TG to bridge domains
 +
and check traffic bidirectionally.
 +
|
 +
|PASS
 +
|-
 +
|Vpp forwards ICMPv4 packets through VXLAN tunnels in the same split-horizon group of one L2 bridge domain
 +
|Create two VXLAN interfaces on both VPP nodes. Create one
 +
bridge domain (learning enabled) on the first VPP node,
 +
add VXLAN interfaces to the same split-horizon group of
 +
the bridge domain where interfaces toward TG are added to.
 +
Create two bridge domains (learning enabled) on the second
 +
VPP node and add one VXLAN interface and one interface
 +
toward TG to each of them. Check traffic bidirectionally.
 +
|
 +
|PASS
 +
|-
 +
|Vpp forwards ICMPv4 packets through VXLAN tunnels in different split-horizon groups of one L2 bridge domain
 +
|Create two VXLAN interfaces on both VPP nodes. Create one
 +
bridge domain (learning enabled) on the first VPP node,
 +
add VXLAN interfaces to different split-horizon groups of
 +
the bridge domain where interfaces toward TG are added to.
 +
Create two bridge domains (learning enabled) on the second
 +
VPP node and add one VXLAN interface and one interface
 +
toward TG to each of them. Check traffic bidirectionally.
 +
|
 +
|PASS
 +
|-
 +
|VPP can pass IPv6 bidirectionally through VXLAN using bridge domain
 +
|Create VXLAN interface on both VPP nodes. Create one
 +
bridge domain (learning enabled) on both VPP nodes, add
 +
VXLAN interface and interface toward TG to bridge domains
 +
and check traffic bidirectionally.
 +
|
 +
|PASS
 +
|-
 +
|Vpp forwards ICMPv6 packets through VXLAN tunnels in the same split-horizon group of one L2 bridge domain
 +
|Create two VXLAN interfaces on both VPP nodes. Create one
 +
bridge domain (learning enabled) on the first VPP node,
 +
add VXLAN interfaces to the same split-horizon group of
 +
the bridge domain where interfaces toward TG are added to.
 +
Create two bridge domains (learning enabled) on the second
 +
VPP node and add one VXLAN interface and one interface
 +
toward TG to each of them. Check traffic bidirectionally.
 +
|
 +
|PASS
 +
|-
 +
|Vpp forwards ICMPv6 packets through VXLAN tunnels in different split-horizon groups of one L2 bridge domain
 +
|Create two VXLAN interfaces on both VPP nodes. Create one
 +
bridge domain (learning enabled) on the first VPP node,
 +
add VXLAN interfaces to different split-horizon groups of
 +
the bridge domain where interfaces toward TG are added to.
 +
Create two bridge domains (learning enabled) on the second
 +
VPP node and add one VXLAN interface and one interface
 +
toward TG to each of them. Check traffic bidirectionally.
 +
|
 +
|PASS
 +
|}
 +
===Vxlan Xconnect Untagged===
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|-
 +
|VPP can pass IPv4 bidirectionally through VXLAN
 +
|
 +
|
 +
|PASS
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
 +
{| class="wikitable"
 +
|Name||Documentation||Message||Status
 +
|}
  
 
==Performance tests description==
 
==Performance tests description==

Revision as of 12:48, 16 June 2016

DRAFT

Introduction

This report aims to provide a comprehensive and self-explanatory summary of all CSIT test cases that have been executed against FD.io VPP-16.06 code release, driven by the automated test infrastructure developed within the FD.io CSIT project (Continuous System and Integration Testing).

CSIT test cases have been grouped into the following test suites: {editor: use table instead of list below}

  1. <test_suite_name>: <test_case_name>, [<test_case_name>]
  2. bridge_domain: bridge_domain_untagged
  3. cop: cop_whitelist_blacklist, cop_whitelist_blacklist_IPv6
  4. dhcp: dhcp_client
  5. fds_related_tests: provider_network, tenant_network
  6. gre: gre_encapsulation
  7. honeycomb: interface_management, vxlan, bridge_domain, tap, interface_vhost_user, sub_interface, persistence, vxlan_gpe.
  8. ipv4: ipv4_arp_untagged, ipv4_iacl_untagged, ipv4_untagged
  9. ipv6: ipv6_iacl_untagged, ipv6_untagged
  10. l2_xconnect: l2_xconnect_untagged
  11. lisp: lisp_api_untagged, lisp_dataplane_untagged
  12. performance: Long_Bridge_Domain_Intel-X520-DA2, Long_IPv4_Cop_Intel-X520-DA2, Long_IPv4_Intel-X520-DA2, Long_IPv6_Cop_Intel-X520-DA2, Long_IPv6_Intel-X520-DA2, Long_Xconnect_Dot1q_Intel-X520-DA2, Long_Xconnect_Intel-X520-DA2, Short_Bridge_Domain_Intel-X520-DA2, Short_IPv4_Cop_Intel-X520-DA2, Short_IPv4_Intel-X520-DA2, Short_IPv6_Cop_Intel-X520-DA2, Short_IPv6_Intel-X520-DA2, Short_Xconnect_Dot1q_Intel-X520-DA2, Short_Xconnect_Intel-X520-DA2
  13. tagging: qinq_l2_xconnect
  14. vxlan: vxlan_bd_dot1q, vxlan_bd_untagged, vxlan_xconnect_untagged

CSIT source code for test cases listed above is available in CSIT branch stable/1606 in directory ./tests/suites/<name_of_the_test_suite>. Local copy of CSIT source code can be obtained by cloning CSIT git repository ("git clone https://gerrit.fd.io/r/csit"). CSIT testing virtual environment can be reproduced using Vagrant by following the instructions in CSIT tutorials.

Following sections provide description of CSIT test cases executed against VPP-16.06 release (vpp branch stable/1606), followed by summary test results and links to more detailed test results data. LF FD.io test environment and VPP DUT configuration specifics are provided later in this report to aid anyone interested in reproducing the complete LF FD.io CSIT testing environment, in either virtual or physical test beds.

Functional tests description

   description of topologies tested
   description of each test case
       taken from documentation from .robot files(?)
       reference specs of tested functionality i.e. IETF RFC
   list of executed test cases and summary of results PASS/FAIL/ISSUE

Functional tests results

Dump of a vpp-csit-verify-virl-1606 job from console in ASCII.

Tests

Suites

Bridge Domain

Bridge Domain Untagged

Name Documentation Message Status
TC01: DUT reports active interfaces Report VPP interfaces on DUT PASS
TC02: DUT with L2BD (MAC learning) switch ICMPv4 between two TG links TG-DUT Eth-IPv4-ICMPv4: On DUT configure L2 bridge-domain

(MAC learning enabled), with two interfaces to TG. Make TG send ICMPv4 Echo Req in both directions between two of its interfaces to be switched by DUT, verify all packets are received.

PASS
TC03: DUT1 and DUT2 with L2BD (MAC learning) switch between two TG links Eth-IPv4-ICMPv4 on links DUT1-DUT2, TG-DUT1, TG-DUT2: On

DUT1 and DUT2 configure L2 bridge-domain (MAC learning enabled), each with one interface to TG and one Ethernet interface towards the other DUT. Make TG send ICMPv4 Echo Req in both directions between two of its interfaces to be switched by DUT1 and DUT2, verify all packets are received.

PASS
TC04: DUT1 and DUT2 with L2BD (static MACs) switch between two TG links Eth-IPv4-ICMPv4 on links DUT1-DUT2, TG-DUT1, TG-DUT2: On

DUT1 and DUT2 configure L2 bridge-domain (static MAC entries), each with one interface to TG and one Ethernet interface towards the other DUT. Make TG send ICMPv4 Echo Req in both directions between two of its interfaces to be switched by DUT1 and DUT2, verify all packets are received.

PASS
TC05: DUT1 and DUT2 with L2BD (MAC learn) and SHG switch between four TG links Eth-IPv4-ICMPv4 on links DUT1-DUT2, TG-DUT1, TG-DUT2: On

DUT1 and DUT2 configure L2 bridge-domain (MAC learning enabled), each with two untagged interfaces to TG and untagged interface towards the other DUT. On each DUT put both interfaces towards TG into the same Split-Horizon-Group (SHG). Make TG send ICMPv4 Echo Reqs between all four of its interfaces to be switched by DUT1 and DUT2, and verify packets are not switched between TG interfaces isolated by SHG on DUT1 and DUT2.

PASS
TC06: DUT with two L2BDs (MAC learn) switches ICMPv4 between TG and VM links Eth-IPv4-ICMPv4 on links DUT-VM, TG-DUT: On DUT configure

two L2 bridge-domains (MAC learning enabled), each with one untagged interface to TG and untagged interface to local VM over vhost-user. Make TG send ICMPv4 Echo Reqs in both directions between two of its interfaces to be switched by DUT to and from VM, and verify all packets are received.

PASS
TC07: DUT with two L2BDs (MAC learn) switches ICMPv6 between TG and VM links Eth-IPv6-ICMPv6 on links DUT-VM, TG-DUT: On DUT configure

two L2 bridge-domains (MAC learning enabled), each with one untagged interface to TG and untagged interface to local VM over vhost-user. Make TG send ICMPv6 Echo Reqs in both directions between two of its interfaces to be switched by DUT to and from VM, and verify all packets are received.

PASS
TC08: DUT with two L2BDs (static MACs) switches ICMPv4 between TG and VM links Eth-IPv6-ICMPv4 on links DUT-VM, TG-DUT: On DUT configure

two L2 bridge-domains (static MAC entries), each with one untagged interface to TG and untagged interface to local VM over vhost-user. Make TG send ICMPv4 Echo Reqs in both directions between two of its interfaces to be switched by DUT to and from VM, and verify all packets are received.

PASS
TC09: DUT with two L2BDs (static MACs) switches ICMPv6 between TG and VM links Eth-IPv6-ICMPv6 on links DUT-VM, TG-DUT: On DUT configure

two L2 bridge-domains (static MAC entries), each with one untagged interface to TG and untagged interface to local VM over vhost-user. Make TG send ICMPv6 Echo Reqs in both directions between two of its interfaces to be switched by DUT to and from VM, and verify all packets are received.

PASS
Name Documentation Message Status

Cop

Cop Whitelist Blacklist

Name Documentation Message Status
VPP permits packets based on IPv4 src addr COP Whitelist test with basic setup. PASS
VPP drops packets based on IPv4 src addr COP blacklist test with basic setup. PASS

cop whitelist blacklist IPv6

Name Documentation Message Status
VPP permits packets based on IPv6 src addr COP Whitelist test with basic setup. PASS
VPP drops packets based on IPv6 src addr COP blacklist test with basic setup. PASS
Name Documentation Message Status

Dhcp

Dhcp Client

Name Documentation Message Status
VPP sends a DHCP DISCOVER Configure DHCP client on interface to TG without hostname

and check if DHCP DISCOVER message contains all required fields with expected values.

PASS
VPP sends a DHCP DISCOVER with hostname Configure DHCP client on interface to TG with hostname

and check if DHCP DISCOVER message contains all required fields with expected values.

PASS
VPP sends DHCP REQUEST after OFFER Configure DHCP client on interface to TG and check if

DHCP REQUEST message contains all required fields.

PASS
VPP doesn't send DHCP REQUEST after OFFER with wrong XID Configure DHCP client on interface to TG. If server sends

DHCP OFFER with different XID as in DHCP DISCOVER, DHCP REQUEST message shouldn't be sent.

Expected error 'DHCP REQUEST Rx timeout' but got 'Traffic script execution failed'. FAIL
VPP honors DHCPv4 lease time Send IP configuration to the VPP client via DHCP. Address

is checked with ICMP echo request and there should be no reply for echo request when lease has expired.

PASS
Name Documentation Message Status

Fds Related Tests

Provider Network

Name Documentation Message Status
Provider network test cases with provider physical networks (VLAN) Ping among all ports inside the same network should pass.

a) test l2 connectivity inside every network b) test l2 connectivity between networks

PASS

Tenant Network

Name Documentation Message Status
L2 test cases with tenant networks (VXLAN) Ping among all ports inside the same network should pass.

a) test l2 connectivity inside every network b) test l2 connectivity between networks

PASS
Name Documentation Message Status

Gre

Gre Encapsulation

Name Documentation Message Status
VPP can route IPv4 traffic from GRE tunnel Create GRE configuration on 2 DUTs, send IP traffic and

check IP is not changed.

PASS
VPP can encapsulate IPv4 traffic in GRE Create GRE configuration on DUT, send IP traffic and

check IP is correctly encapsulate into GRE.

PASS
VPP can decapsulate IPv4 traffic in GRE Create GRE configuration on DUT, send GRE encapsulated

traffic and check IP is received without GRE headers.

PASS
Name Documentation Message Status

Ipv4

Ipv4 Arp Untagged

Name Documentation Message Status
VPP sends ARP requests for unknown destinations Setup IP addresses and route.

Send ICMP packet and check if VPP sends ARP request.

PASS

Ipv4 Iacl Untagged

Name Documentation Message Status
VPP drops packets based on IPv4 source addresses Create classify table on VPP, add source IP address

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 destination addresses Create classify table on VPP, add destination IP address

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 src-addr and dst-addr Create classify table on VPP, add source and destination

IP address of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 protocol (TCP) Create classify table on VPP, add mask for TCP port

into table and setup 'deny' traffic and check if TCP traffic is dropped.

PASS
VPP drops packets based on IPv4 protocol (UDP) Create classify table on VPP, add mask for UDP port

into table and setup 'deny' traffic and check if UDP traffic is dropped.

PASS
VPP drops packets based on IPv4 TCP src ports Create classify table on VPP, add source TCP port

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 TCP dst ports Create classify table on VPP, add destination TCP port

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 TCP src + dst ports Create classify table on VPP, add source and destination

TCP port of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 UDP src ports Create classify table on VPP, add source UDP port

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 UDP dst ports Create classify table on VPP, add destination UDP port

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv4 UDP src + dst ports Create classify table on VPP, add source and destination

UDP port of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS

Ipv4 Untagged

Name Documentation Message Status
VPP replies to ICMPv4 echo request PASS
TG can route to DUT egress interface PASS
TG can route to DUT2 through DUT1 PASS
TG can route to DUT2 egress interface through DUT1 PASS
TG can route to TG through DUT1 and DUT2 PASS
VPP can process ICMP echo request from min to 1500B packet size with 1B increment PASS
VPP responds to ARP request PASS
Name Documentation Message Status

Ipv6

Ipv6 Iacl Untagged

Name Documentation Message Status
VPP drops packets based on IPv6 source addresses Create classify table on VPP, add source IP address

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 destination addresses Create classify table on VPP, add destination IP address

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 src-addr and dst-addr Create classify table on VPP, add source and destination

IP address of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 protocol (TCP) Create classify table on VPP, add mask for TCP port

into table and setup 'deny' traffic and check if TCP traffic is dropped.

PASS
VPP drops packets based on IPv6 protocol (UDP) Create classify table on VPP, add mask for UDP port

into table and setup 'deny' traffic and check if UDP traffic is dropped.

PASS
VPP drops packets based on IPv6 TCP src ports Create classify table on VPP, add source IP address

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 TCP dst ports Create classify table on VPP, add destination TCP port

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 TCP src + dst ports Create classify table on VPP, add source and destination

TCP port of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 UDP src ports Create classify table on VPP, add source UDP port

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 UDP dst ports Create classify table on VPP, add destination UDP port

of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS
VPP drops packets based on IPv6 UDP src + dst ports Create classify table on VPP, add source and destination

UDP port of traffic into table and setup 'deny' traffic and check if traffic is dropped.

PASS

Ipv6 Untagged

Name Documentation Message Status
VPP replies to ICMPv6 echo request PASS
VPP can process ICMPv6 echo request from min to 1500B packet size with 1B increment PASS
TG can route to first DUT egress interface PASS
TG can route to second DUT through first DUT PASS
TG can route to second DUT egress interface through first DUT PASS
TG can route to TG through first and second DUT PASS
VPP replies to IPv6 Neighbor Solicitation PASS
Name Documentation Message Status

L2 Xconnect

L2 Xconnect Untagged

Name Documentation Message Status
Vpp forwards ICMPv4 packets via L2 xconnect in circular topology Setup single link path with X-connect

and send ICMPv4 packet.

PASS
Vpp forwards ICMPv6 packets via L2 xconnect in circular topology Setup single link path with X-connect

and send ICMPv6 packet.

PASS
VPP forwards ICMPv4 packets through VM via L2 x-connect Setup double link path with X-connect via Vhost user

and send ICMPv4 packet.

PASS
VPP forwards ICMPv6 packets through VM via L2 x-connect Setup double link path with X-connect via Vhost user

and send ICMPv6 packet.

PASS
Name Documentation Message Status

Lisp

Lisp Api Untagged

Name Documentation Message Status
VPP can add and delete locator_set Test lisp locator_set API

Set locator_set and locator on the VPP node, check the configured data and then remove it. Check if all locator_set and locators was unset from the VPP node.

PASS
VPP can add, reset and delete locator_set Test lisp locator_set API

Set locator_set and locator on the VPP node, then reset locator_set and set it again. Check the configured data and then remove it. Check if all locator_set and locators was unset from the VPP node.

PASS
Vpp can add and delete eid address Test lisp eid API

Set lisp eid IP address on the VPP node, check the configured data and then remove it. Check if all eid IP address was unset from the VPP node.

PASS
Vpp can add and delete lisp map resolver address Test lisp map resolver address API

Set lisp map resolver address on the VPP node, check the configured data and then remove it. Check if all map resolver address was unset from the VPP node.

PASS

Lisp Dataplane Untagged

Name Documentation Message Status
VPP can pass IPv4 bidirectionally through LISP Test IP4 Lisp remote static mapping.

Set IP4 lisp topology and check if packet passes through Lisp topology.

PASS
VPP can pass IPv6 bidirectionally through LISP Test IP6 Lisp remote static mapping.

Set IP6 lisp topology and check if packet passes through Lisp topology.

PASS
VPP can pass IPv4 over IPv6 bidirectionally through LISP Test IP4 over IP6 in Lisp remote static mapping.

Set IP6 topology and check if the IP4 packet passes through IP6 Lisp topology.

PASS
VPP can pass IPv6 over IPv4 bidirectionally through LISP Test IP6 over IP4 in Lisp remote static mapping.

Set IP4 topology and check if the IP6 packet passes through IP4 Lisp topology.

PASS
Name Documentation Message Status

Tagging

Qinq L2 Xconnect

Name Documentation Message Status
VPP can push and pop two VLAN tags to traffic transferring through xconnect Push two tags on DUT 1 to traffic sent from TG,

pop two tags on DUT 2 from this traffic and receive untagged traffic on TG.

PASS
Name Documentation Message Status

Vxlan

Vxlan Bd Dot1Q

Name Documentation Message Status
VPP can encapsulate L2 in VXLAN over IPv4 over Dot1Q PASS

Vxlan Bd Untagged

Name Documentation Message Status
VPP can pass IPv4 bidirectionally through VXLAN using bridge domain Create VXLAN interface on both VPP nodes. Create one

bridge domain (learning enabled) on both VPP nodes, add VXLAN interface and interface toward TG to bridge domains and check traffic bidirectionally.

PASS
Vpp forwards ICMPv4 packets through VXLAN tunnels in the same split-horizon group of one L2 bridge domain Create two VXLAN interfaces on both VPP nodes. Create one

bridge domain (learning enabled) on the first VPP node, add VXLAN interfaces to the same split-horizon group of the bridge domain where interfaces toward TG are added to. Create two bridge domains (learning enabled) on the second VPP node and add one VXLAN interface and one interface toward TG to each of them. Check traffic bidirectionally.

PASS
Vpp forwards ICMPv4 packets through VXLAN tunnels in different split-horizon groups of one L2 bridge domain Create two VXLAN interfaces on both VPP nodes. Create one

bridge domain (learning enabled) on the first VPP node, add VXLAN interfaces to different split-horizon groups of the bridge domain where interfaces toward TG are added to. Create two bridge domains (learning enabled) on the second VPP node and add one VXLAN interface and one interface toward TG to each of them. Check traffic bidirectionally.

PASS
VPP can pass IPv6 bidirectionally through VXLAN using bridge domain Create VXLAN interface on both VPP nodes. Create one

bridge domain (learning enabled) on both VPP nodes, add VXLAN interface and interface toward TG to bridge domains and check traffic bidirectionally.

PASS
Vpp forwards ICMPv6 packets through VXLAN tunnels in the same split-horizon group of one L2 bridge domain Create two VXLAN interfaces on both VPP nodes. Create one

bridge domain (learning enabled) on the first VPP node, add VXLAN interfaces to the same split-horizon group of the bridge domain where interfaces toward TG are added to. Create two bridge domains (learning enabled) on the second VPP node and add one VXLAN interface and one interface toward TG to each of them. Check traffic bidirectionally.

PASS
Vpp forwards ICMPv6 packets through VXLAN tunnels in different split-horizon groups of one L2 bridge domain Create two VXLAN interfaces on both VPP nodes. Create one

bridge domain (learning enabled) on the first VPP node, add VXLAN interfaces to different split-horizon groups of the bridge domain where interfaces toward TG are added to. Create two bridge domains (learning enabled) on the second VPP node and add one VXLAN interface and one interface toward TG to each of them. Check traffic bidirectionally.

PASS

Vxlan Xconnect Untagged

Name Documentation Message Status
VPP can pass IPv4 bidirectionally through VXLAN PASS
Name Documentation Message Status
Name Documentation Message Status
Name Documentation Message Status

Performance tests description

   description of topologies tested
   description of each test case, test case type
       taken from documentation from .robot files(?)
       reference specs of tested functionality i.e. IETF RFC
   list of executed test cases and summary results
       long tests - graphs with discovered NDR and PDR rates
       short tests - list with PASS/FAIL/ISSUE

Performance tests results

Set of graphs from vpp-csit-verify-hw-perf-1606-long job and trending from semiweekly. For more granular view, we can also link to wiki page with console output in ASCII and also links to robot log.html and report.html, if we find a way to host them somewhere.

Functional tests environment

   Config Data: VPP (DUT)
       VPP Running Configuration
       VPP Startup Configuration
       VPP Version
       VPP Compile Parameters
       VPP Build Script used - pointer if default

   Config Data: VM (DUT)
       VM OS Information

   Config Data: Traffic Generator - Scapy
       TG Version

   Test Data: VPP (DUT)
       VPP telemetry cli Outputs

   Test Data: Traffic Generator - Scapy

Performance tests environment

   Compute Node Environment for VPP (DUT)
       Host hardware: cpu, memory, nics
       Host BIOS Information
       Host OS Linux distro ver
       Host OS Linux kernel ver
       Host OS Linux additional patch references
       Host OS Qemu ver
   Config Data: VPP (DUT)
       NUMA Node Location of VPP Interfaces
           NIC types
       DPDK version
       VPP Version
       VPP Compile Parameters
       VPP Build Script used - pointer if default
       VPP Startup Configuration
       VPP Running Configuration

   Compute Node Environment for Traffic Generator (TG)
       Host BIOS Information
       Host OS Linux distro ver
       Host OS Linux kernel ver
       Host OS Linux additional patch references

   Config Data: Traffic Generator - T-Rex
       TG Version
           DPDK version - if applicable
       TG Compile Parameters
       TG Build Script used - pointer if default
       TG Startup Configuration
       TG Running Configuration
       TG common API calls?

   Test Data: VPP (DUT)
       VPP Telemetry Outputs collected at NDR and PDR

   Test Data: Compute Host (DUT)
       CPU Perf Counters collected at NDR - not collected today

   Test Data: Traffic Generator - T-Rex
       TG (T-Rex) Outputs During & After Test at 64B NDR
       TG (T-Rex) Outputs During & After Test at IMIX NDR
       TG (T-Rex) Outputs During & After Test at 1518 NDR
       TG (T-Rex) Outputs During & After Test at 9000 NDR

CSIT release notes

   jira release notes
   complete dump of test cases from stable/1606 branch
       functional
       performance
Retrieved from "https://wiki.fd.io/index.php?title=CSIT/VPP-16.06_Test_Report_Sandbox&oldid=1813"