CSIT/VPP-16.06 Test Report Draft
FD.io CSIT Test Report for VPP-16.06
Contents
Introduction
This report aims to provide a comprehensive and self-explanatory summary of all CSIT test cases that have been executed against FD.io VPP-16.06 code release, driven by the automated test infrastructure developed within the FD.io CSIT project (Continuous System and Integration Testing).
Executed CSIT test cases have been grouped into the following test suites:
- <test_suite_name>: <test_case_name>, [<test_case_name>]
- bridge_domain: bridge_domain_untagged
- cop: cop_whitelist_blacklist, cop_whitelist_blacklist_IPv6
- dhcp: dhcp_client
- fds_related_tests: provider_network, tenant_network
- gre: gre_encapsulation
- honeycomb: interface_management, vxlan, bridge_domain, tap, interface_vhost_user, sub_interface, persistence, vxlan_gpe.
- ipv4: ipv4_arp_untagged, ipv4_iacl_untagged, ipv4_untagged
- ipv6: ipv6_iacl_untagged, ipv6_untagged
- l2_xconnect: l2_xconnect_untagged
- lisp: lisp_api_untagged, lisp_dataplane_untagged
- performance: Long_Bridge_Domain_Intel-X520-DA2, Long_IPv4_Cop_Intel-X520-DA2, Long_IPv4_Intel-X520-DA2, Long_IPv6_Cop_Intel-X520-DA2, Long_IPv6_Intel-X520-DA2, Long_Xconnect_Dot1q_Intel-X520-DA2, Long_Xconnect_Intel-X520-DA2, Short_Bridge_Domain_Intel-X520-DA2, Short_IPv4_Cop_Intel-X520-DA2, Short_IPv4_Intel-X520-DA2, Short_IPv6_Cop_Intel-X520-DA2, Short_IPv6_Intel-X520-DA2, Short_Xconnect_Dot1q_Intel-X520-DA2, Short_Xconnect_Intel-X520-DA2
- tagging: qinq_l2_xconnect
- vxlan: vxlan_bd_dot1q, vxlan_bd_untagged, vxlan_xconnect_untagged
CSIT source code for test cases listed above is available in CSIT git repository (https://gerrit.fd.io/r/csit) in directory ./tests/suites/<name_of_the_test_suite>.
Following sections provide description of CSIT test cases executed against VPP-16.06 release (vpp branch stable/1606), followed by summary test results and links to more detailed test results data. LF FD.io test environment and VPP DUT configuration specifics are provided later in this report to aid anyone interested in reproducing the complete LF FD.io CSIT testing environment, in either virtual or physical test beds.
Functional tests description
per topology
list of executed test cases and their results PASS/FAIL/ISSUE
description of topologies
description of each test case
reference spec of tested functionality i.e. IETF RFC
Performance tests description
per topology
list of executed test cases and their results
long tests - graphs with discovered NDR and PDR rates
short tests - list with PASS/FAIL/ISSUE
description of topologies
description of each test case, test case type
Functional tests results
Performance tests results
Configuration of DUT VPP technology stack
compute HW OS VPP compilation settings VPP startup configs
Performance tests environment
Config Data: VPP (DUT)
NUMA Node Location of VPP Interfaces
VPP Running Configuration
VPP Startup Configuration
VPP Version
VPP Compile Parameters
VPP Build Script used - pointer if default
Config Data: Compute Host (DUT)
Host BIOS Information
Host OS Information
Config Data: Traffic Generator - T-Rex
TG Version
TG settings
Test Data: VPP (DUT)
VPP Debug_cli Outputs at NDR
Test Data: Compute Host (DUT)
CPU Perf Counters at NDR - not collected today
Test Data: Traffic Generator - T-Rex
TG (T-Rex) Outputs During & After Test at 64B NDR
TG (T-Rex) Outputs During & After Test at IMIX NDR
TG (T-Rex) Outputs During & After Test at 1518 NDR
TG (T-Rex) Outputs During & After Test at 9000 NDR
Functional tests environment
Config Data: VPP (DUT)
VPP Running Configuration
VPP Startup Configuration
VPP Version
VPP Compile Parameters
VPP Build Script used - pointer if default
Config Data: VM (DUT)
VM OS Information
Config Data: Traffic Generator - Scapy
TG Version
Test Data: VPP (DUT)
VPP Debug_cli Outputs at NDR
Test Data: Traffic Generator - Scapy
CSIT release notes
jira release notes
dump of test cases from csit-verified branch
functional
performance
