Difference between revisions of "CSIT/PerfTestPlan"

From fd.io
Jump to: navigation, search
 
(70 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
==FD.io IT systems integration==
+
'''WORK IN PROGRESS - PLEASE TREAT IT AS SUCH'''
  
−
Three Jenkins jobs for CSIT performance test cases are in place today:
+
__TOC__
  
−
# Job name: '''csit-vpp-functional-hw-tb2'''
+
=CSIT development plan - background=
−
#* Description: This job is testing csit project against performance test cases
+
−
#** TODO items
+
−
#*** name - rename to "csit-vpp-performance-hw" or better "csit-vpp-perf-hw"
+
−
#*** scope - currently only tb2, need to add other testbeds: tb1, tb3
+
−
#*** scope - tbX reservation script and tbX scheduler code already done and tested, needs to be used
+
−
#* Trigger: Comment "hw-perf-check" in proposed patch in CSIT project
+
−
#** TODO items
+
−
#*** rename to "verify-perf-all"
+
−
#*** consider adding later on "verify-perf-short", "verify-perf-long"
+
−
#* Status: WORKING
+
−
# Job name: '''vpp-verify-performance-long'''
+
−
#* Description: This job is testing vpp proposed patch against long performance test cases
+
−
#* Trigger: Comment "verify-perf-long" in proposed patch in VPP project
+
−
#* Status: WORKING
+
−
# Job name: '''vpp-verify-performance-short'''
+
−
#* Description: This job is testing vpp proposed patch against short performance test cases
+
−
#* Trigger: Comment "verify-perf-short" in proposed patch in VPP project
+
−
#* Status: WORKING
+
  
−
==CSIT performance test cases - working==
+
CSIT test development priorities are aligned with VPP releases timeline, with relative dev work prioritization for each VPP release driven by perceived importance of test verification for VPP functionality and performance. Specific CSIT dev work priorities are driven by rough consensus within the project and contributors and committers availability and willingness to do work.
  
−
# Working performance test cases - 18 working in FD.io physical testbeds today.
+
Following priority tagging scheme is used in this CSIT wiki and FD.io jira for CSIT:
−
# Last manual run results reports:
+
−
#* https://jenkins.fd.io/view/csit/job/csit-vpp-functional-hw-tb2/
+
−
#* https://jenkins.fd.io/view/vpp/job/vpp-verify-performance-long/
+
−
#* https://jenkins.fd.io/view/vpp/job/vpp-verify-performance-short/
+
−
# List of working performance test cases
+
−
## ~/csit$  grep "^| [a-zA-Z1-9]" -nr tests | grep -vi " | "  | sort | grep performance
+
−
## tests/suites/performance/long_bridge_domain.robot:24:| Find NDR by using linear search and 64B frames through bridge domain in 3-node topology
+
−
##* // TODO items: rename to `Find NDR by using RFC2544 linear search and 64B frames through bridge domain in 3-node topology`
+
−
## tests/suites/performance/long_bridge_domain.robot:34:| Find NDR by using linear search and 1518B frames through bridge domain in 3-node topology
+
−
## tests/suites/performance/long_bridge_domain.robot:44:| Find NDR by using linear search and 9000B frames through bridge domain in 3-node topology
+
−
## tests/suites/performance/long_ipv4.robot:28:| Find NDR by using linear search and 64B frames through IPv4 forwarding in 3-node topology
+
−
## tests/suites/performance/long_ipv4.robot:38:| Find NDR by using linear search and 1518B frames through IPv4 forwarding in 3-node topology
+
−
## tests/suites/performance/long_ipv4.robot:48:| Find NDR by using linear search and 9000B frames through IPv4 forwarding in 3-node topology
+
−
## tests/suites/performance/long_xconnect.robot:25:| Find NDR by using linear search and 64B frames through L2 cross connect in 3-node topology
+
−
## tests/suites/performance/long_xconnect.robot:35:| Find NDR by using linear search and 1518B frames through L2 cross connect in 3-node topology
+
−
## tests/suites/performance/long_xconnect.robot:45:| Find NDR by using linear search and 9000B frames through L2 cross connect in 3-node topology
+
−
## tests/suites/performance/short_bridge_domain.robot:23:| 1core VPP passes 64B frames through bridge domain at 3.5mpps in 3-node topology
+
−
## tests/suites/performance/short_bridge_domain.robot:30:| 1core VPP passes 1518B frames through bridge domain at 10gbps in 3-node topology
+
−
## tests/suites/performance/short_bridge_domain.robot:37:| 1core VPP passes 9000B frames through bridge domain at 10gbps in 3-node topology
+
−
## tests/suites/performance/short_ipv4.robot:27:| 1core VPP passes 64B frames through IPv4 forwarding at 3.5mpps in 3-node topology
+
−
##* // TODO items: rename to `1core VPP passes 64B frames through IPv4 forwarding at 2x 3.5Mpps in 3-node topology`
+
−
##* // TODO items: currently testing all functional combinations takes 10-to-15 minutes; going forward would need to split the functional combinations
+
−
## tests/suites/performance/short_ipv4.robot:34:| 1core VPP passes 1518B frames through IPv4 forwarding at 10gbps in 3-node topology
+
−
## tests/suites/performance/short_ipv4.robot:41:| 1core VPP passes 9000B frames through IPv4 forwarding at 10gbps in 3-node topology
+
−
## tests/suites/performance/short_xconnect.robot:24:| 1core VPP passes 64B frames through L2 cross connect at 3.5mpps in 3-node topology
+
−
## tests/suites/performance/short_xconnect.robot:31:| 1core VPP passes 1518B frames through L2 cross connect at 10gbps in 3-node topology
+
−
## tests/suites/performance/short_xconnect.robot:38:| 1core VPP passes 9000B frames through L2 cross connect at 10gbps in 3-node topology
+
  
−
==CSIT performance test cases - plan and status==
+
* [$VppRlsNo-$RelPrio]
 +
** VppRlsNo - aligned VPP release number - 16.09, 16.12, 17.03.
 +
** RelPrio - CSIT release priority - P0 for Must Have, P1 for Should Have, P2 for Could Have
 +
* CSIT dev work tagging examples:
 +
** [1609-P0] - CSIT Must Have for vpp-v1609
 +
** [1609-P1] - CSIT Should Have for vpp-v1609
 +
** [1609-P2] - CSIT Could Have for vpp-v1609
  
−
# [DONE, mmiklus] DropRateSearch
+
CSIT dev work plan execution tracked on [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=1&projectKey=CSIT CSIT jira], with priority tags encoded as Jira labels for easy of tracking and reporting.
−
#* [DONE, pmikus] binary search [https://gerrit.fd.io/r/#/c/564/ gerrit.fd.io change]
+
 
−
#* [DONE, pmikus] BESTofN/WORSTofN [https://gerrit.fd.io/r/#/c/603/ gerrit.fd.io change]
+
=CSIT performance tests - dev plan=
−
#* [IN REVIEW, pmikus] combined search [https://gerrit.fd.io/r/#/c/671/ gerrit.fd.io change]
+
 
−
# [WIP, mmiklus] extend topology compute library with NIC type filtering
+
CSIT performance test cases devevelopment work is grouped and tracked across the following VPP functional areas:
−
# [WIP, pmikus] multicore tests [https://gerrit.fd.io/r/#/c/710/ gerrit.fd.io change]
+
 
−
# [TBD] IPv6
+
# NIC devices and drivers
−
# [TBD] Large-scale FIB tests
+
# IPv4 data plane
−
#* IPv4
+
# IPv4 control plane
−
#* IPv6
+
# IPv4 encapsulations
−
#* mac
+
# IPv4 telemetry
−
#* Various prefix lengths
+
# IPv6 data plane
−
# [TBD] Encapsulations
+
# IPv6 control plane
−
#* VxLAN
+
# IPv6 encapsulations
−
#* VLAN
+
# IPv6 telemetry
−
#* GRE
+
# Ethernet L2 data plane
−
#* IPSEC
+
# Ethernet L2 control plane
−
# [TBD] Create topology files for tb1, tb3
+
# Ethernet L2 encapsulations
−
# [TBD] T-REX VXLAN traffic profile & CSIT tests
+
# Ethernet L2 telemetry
−
# IxNetwork driver
+
# MPLS data plane
−
#* [TBD] driver design
+
# NSH data plane
−
#* [TBD] driver implementation
+
 
−
#** setup, teardown
+
CSIT team is tracking completed CSIT dev work per above area (completed = test cases running in FD.io), as well as top-level test coverage metric [%] of VPP functions. VPP code development dependencies are marked with [Blocked-by VPP-XXX]. Performance tests require the functional tests to be completed - see sub-sections "Functional tests preconditions".
−
#** traffic stream configuration
+
 
−
#** statistics retrieval
+
CSIT performance tests are split into the two main groups:
−
#* [TBD] latency measurements
+
* NDR(PDR) throughput discovery - tests searching for NDR non-drop rate throughput or PDR partial-drop rate throughput for specified configuration. Also referred to as "long" tests.
−
#** feasibility
+
* refNDR throughput verification - tests verifying code patches against the reference NDR throughput. Also referred to as "short" tests.
−
#** implementation
+
 
−
#* [TBD] integration with CSIT tests
+
==Infrastructure - NIC devices and drivers==
−
# Moongen driver
+
 
−
#* [TBD] verify requirements and available HW against latest MoonGen codebase
+
'''1609-P0'''
−
#* [TBD] integration
+
 
−
# T-REX
+
<jira jql='project = CSIT AND labels = perf AND labels = infra AND labels = 1609-p0'></jira>
−
#* [TBD] latency (T-REX support to be confirmed)
+
 
−
# Other work items
+
'''1609-P1'''
−
#* [TBD] integrate [https://pypi.python.org/pypi/sysinfosuite Python sysinfosuite] to CSIT
+
 
−
#* [TBD] unified performance trend dashboard
+
<jira jql='project = CSIT AND labels = perf AND labels = infra AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = infra AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1606-DONE] NIC Intel X520 2p10GE - DPDK niantic driver, 2 ports within the same NIC.
 +
# [1609-P0] NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
 +
# [1609-DONE] VPP vhost-user - VPP vhost-user driver, virtio in VM.
 +
# [1609-P1] NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
 +
# [1609-DONE] DPDK vhost-user - DPDK vhost-user driver, virtio in VM.
 +
# [1612-P0] vhost-user - XXX vhost-user driver, virtio in VM, box-full test with NxVMs.
 +
# [1612-P2] continue to expand VPP driver coverage - more NIC-DPDK drivers, other TBD.
 +
 
 +
==IPv4 data plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4dp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4dp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4dp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
'''Baseline performance'''
 +
 
 +
# [1609-P0] IPv4 routing RFC791 - baseline - NIC-to-NIC(XL710).
 +
# [1609-DONE] IPv4 routing RFC791 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
 +
# [1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
 +
# [1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
 +
# [1609-P1] extranet VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
 +
# [1609-P1] extranet VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
 +
# [1606-DONE] IPv4 routing RFC791 - baseline - NIC-to-NIC(X520).
 +
 
 +
'''FIB performance'''
 +
 
 +
Large-scale FIB tests
 +
# [1609-DONE] same prefix length.
 +
# [1609-P1] varying prefix lengths.
 +
 
 +
'''Security performance'''
 +
 
 +
# [1606-DONE] security - iacl with n-tuple classification, ip4.
 +
# [1606-DONE] security - cop white-/black-list ip4 src addr filtering.
 +
 
 +
'''QoS performance'''
 +
 
 +
# [1609-REVIEW] QoS - ingress policing - rate metering, marking and limiting - 2r3c-(cb,ca) policers, RFC2698.
 +
# [1609-P1] QoS - ingress policing - rate metering, marking and limiting - 1r2c-(cb,ca) policers, RFC2697.
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P2] security - source IPv4 Reverse Path Forwarding (RPF).  
 +
    # [1609-P1] routing multipath (ECMP, UECMP) - baseline.  
 +
    # [1609-P1] [Blocked-by VPP-xxx] carrier grade nat 44, CGN44, RFC6598, RFC6888.
 +
    # [1609-P2] [Blocked-by VPP-xxx] security - stateful firewall.
 +
 
 +
==IPv4 control plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4cp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4cp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4cp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] Proxy ARP.  
 +
    # [1609-P0] IKEv2.  
 +
    # [1609-P1] DHCPv4 proxy.
 +
    # [1606-DONE] ARP.
 +
    # [1606-DONE] ICMPv4.  
 +
    # [1606-DONE] DHCPv4 client.  
 +
 
 +
==IPv4 encapsulations==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4enc AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4enc AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4enc AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
'''Baseline performance'''
 +
 
 +
# [1609-P1] ip4-gre-ip4 - baseline main fib.
 +
# [1609-DONE] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp.
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] ip4-ipsec-ip4 - baseline main fib.
 +
    # [1609-P0] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), virt2lisp.
 +
    # [1609-P0] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
 +
    # [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp.
 +
    # [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
 +
    # [1606-DONE] ip4-gre-ip4 - baseline main fib.
 +
 
 +
==IPv4 telemetry==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4tel AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4tel AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip4tel AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] IPv4 IPFIX - stats collection and export - baseline.  
 +
    # [1609-P0] IPv4 IPFIX - stats collection and export - functional scale.  
 +
    # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline.
 +
    # [1609-P1] Lawful intercept  - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.
 +
 
 +
==IPv6 data plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6dp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6dp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6dp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
'''Baseline performance'''
 +
 
 +
# [1609-P0] IPv6 routing RFC2460 - baseline - NIC-to-NIC(XL710).
 +
# [1609-P0] IPv6 routing RFC2460 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
 +
# [1609-P1] VPNv6 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
 +
# [1609-P1] VPNv6 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
 +
# [1609-P2] extranet VPN routed forwarding (VRF) - baseline.
 +
# [1606-DONE] IPv6 routing RFC2460 - baseline - NIC-to-NIC(X520).
 +
 
 +
'''FIB performance'''
 +
 
 +
Large-scale FIB tests
 +
# [1609-DONE] same prefix length.
 +
# [1609-P1] varying prefix lengths.
 +
 
 +
'''Security performance'''
 +
 
 +
# [1609-DONE] security - iacl with n-tuple classification, ip6.
 +
# [1606-DONE] security - cop white-/black-list ip6 src addr filtering.
 +
 
 +
'''QoS performance'''
 +
 
 +
# [1609-P1] QoS - ingress policing - rate metering, marking and rate-limiting -1r2c-(cb,ca), 2r3c-(cb,ca) policers, RFC2475, RFC2697, RFC2698.
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] security - source IPv6 Reverse Path Forwarding (RPF).
 +
    # [1609-P1] routing multipath (ECMP, UECMP) - baseline.
 +
    # [1609-P2] [Blocked-by VPP-xxx] security - stateful firewall.
 +
 
 +
==IPv6 control plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6cp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6cp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6cp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] router advertisement.
 +
    # [1609-P1] DHCPv6 proxy.
 +
    # [1609-P2] [Blocked-by VPP-xxx] ip6-ioam, in-band OAM.
 +
    # [1606-DONE] neighbor discovery, RFC4861.
 +
    # [1606-DONE] ICMPv6.
 +
 
 +
==IPv6 encapsulations==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6enc AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6enc AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6enc AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-DONE] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp.
 +
# [1609-DONE] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp.
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] ip4-o-ip6 softwire.
 +
    # [1609-P0] ip6-ipsec-ip6.
 +
    # [1609-P0] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), virt2lisp.
 +
    # [1609-P0] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp.
 +
    # [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), virt2lisp.
 +
    # [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp.
 +
    # [1609-P0] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp.
 +
    # [1609-P0] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
 +
    # [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp.
 +
    # [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
 +
    # [1609-P1] [Blocked-by VPP-xxx] ip6-segment-routing.
 +
 
 +
==IPv6 telemetry==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6tel AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6tel AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = ip6tel AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] IPv6 IPFIX - stats collection and export - baseline.
 +
    # [1609-P0] IPv6 IPFIX - stats collection and export - functional scale.
 +
    # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline.
 +
    # [1609-P1] Lawful intercept  - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.
 +
 
 +
==Ethernet L2 data plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2dp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2dp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2dp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
'''Baseline performance'''
 +
 
 +
# [1609-P0] L2 Cross-connect - baseline - NIC-to-NIC(XL710).
 +
# [1609-P0] Bridge domain - baseline - NIC-to-NIC(XL710).
 +
# [1609-DONE] L2 Cross-connect - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
 +
# [1609-DONE] Bridge domain - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
 +
# [1606-DONE] L2 Cross-connect - baseline - NIC-to-NIC(X520).
 +
# [1606-DONE] Bridge domain - baseline - NIC-to-NIC(X520).
 +
 
 +
'''FIB performance'''
 +
 
 +
Large-scale FIB tests
 +
 
 +
# [1609-P1] [Blocked-by VPP-201] MAC addresses.
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] Bridge domain - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2.
 +
    # [1609-P1] Integrated Routing-Bridging (IRB) - baseline.
 +
    # [1606-DONE] L2 Cross-connect - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2.
 +
 
 +
==Ethernet L2 control plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2cp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2cp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2cp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
# [1606-DONE] MAC learning.
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P2] [Blocked-by VPP-xxx] CDP/LLDP.
 +
    # [1606-DONE] MAC split-horizon.
 +
 
 +
==Ethernet L2 encapsulations==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2enc AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2enc AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2enc AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-P0] l2-eth-lispgpe-ip6 - L2BD.
 +
# [1609-P0] l2-eth-lispgpe-ip4 - L2BD.
 +
# [1609-P1] l2-vxlan-ipv4 - with bridge-domain.
 +
# [1609-DONE] l2-vxlan-ipv4 - with L2 cross-connect.
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P0] l2-eth-lispgpe-ip6 - L2BD, L2XC.
 +
    # [1609-P0] l2-eth-lispgpe-ip4 - L2BD, L2XC.
 +
    # [1609-P0] l2eth-gre-ip4-ipsec-ip4-eth - L2BD, L2XC.
 +
    # [1609-P2] l2-eth-l2tpv3-ip4 - L2BD, L2XC.
 +
    # [1609-P2] l2-eth-l2tpv3-ip6 - L2BD, L2XC.
 +
    # [1609-P2] l2-eth-gre-ip4 - L2BD, L2XC.
 +
    # [1609-P2] l2-eth-gre-ip6 - L2BD, L2XC.
 +
 
 +
==L2 telemetry==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2tel AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2tel AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = l2tel AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P1] L2 FIB - stats collection and export - baseline.
 +
    # [1609-P2] L2 FIB - stats collection and export - functional scale.
 +
    # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline.
 +
    # [1609-P1] Lawful intercept  - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.
 +
 
 +
==MPLS data plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = mplsdp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = mplsdp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = mplsdp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P2] vpnv4-mpls-gre-ip4.
 +
    # [1609-P2] mpls-o-eth.
 +
 
 +
==NSH data plane==
 +
 
 +
'''1609-P0'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = nshdp AND labels = 1609-p0'></jira>
 +
 
 +
'''1609-P1'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = nshdp AND labels = 1609-p1'></jira>
 +
 
 +
'''1609-P2'''
 +
 
 +
<jira jql='project = CSIT AND labels = perf AND labels = nshdp AND labels = 1609-p2'></jira>
 +
 
 +
Indicative coverage = <to_be_added>
 +
 
 +
# [1609-Px] <to_be_added>
 +
 
 +
'''Functional tests preconditions'''
 +
 
 +
Below copied from CSIT functional tests plan - provided for reference only:
 +
 
 +
    # [1609-P1] ip4-nsh-gre-ip4.
 +
    # [1609-P1] ip4-nsh-vxlan-gpe-ip4.
 +
 
 +
END OF NOTE

Latest revision as of 11:45, 20 September 2016

WORK IN PROGRESS - PLEASE TREAT IT AS SUCH

CSIT development plan - background

CSIT test development priorities are aligned with VPP releases timeline, with relative dev work prioritization for each VPP release driven by perceived importance of test verification for VPP functionality and performance. Specific CSIT dev work priorities are driven by rough consensus within the project and contributors and committers availability and willingness to do work.

Following priority tagging scheme is used in this CSIT wiki and FD.io jira for CSIT:

  • [$VppRlsNo-$RelPrio]
    • VppRlsNo - aligned VPP release number - 16.09, 16.12, 17.03.
    • RelPrio - CSIT release priority - P0 for Must Have, P1 for Should Have, P2 for Could Have
  • CSIT dev work tagging examples:
    • [1609-P0] - CSIT Must Have for vpp-v1609
    • [1609-P1] - CSIT Should Have for vpp-v1609
    • [1609-P2] - CSIT Could Have for vpp-v1609

CSIT dev work plan execution tracked on CSIT jira, with priority tags encoded as Jira labels for easy of tracking and reporting.

CSIT performance tests - dev plan

CSIT performance test cases devevelopment work is grouped and tracked across the following VPP functional areas:

  1. NIC devices and drivers
  2. IPv4 data plane
  3. IPv4 control plane
  4. IPv4 encapsulations
  5. IPv4 telemetry
  6. IPv6 data plane
  7. IPv6 control plane
  8. IPv6 encapsulations
  9. IPv6 telemetry
  10. Ethernet L2 data plane
  11. Ethernet L2 control plane
  12. Ethernet L2 encapsulations
  13. Ethernet L2 telemetry
  14. MPLS data plane
  15. NSH data plane

CSIT team is tracking completed CSIT dev work per above area (completed = test cases running in FD.io), as well as top-level test coverage metric [%] of VPP functions. VPP code development dependencies are marked with [Blocked-by VPP-XXX]. Performance tests require the functional tests to be completed - see sub-sections "Functional tests preconditions".

CSIT performance tests are split into the two main groups:

  • NDR(PDR) throughput discovery - tests searching for NDR non-drop rate throughput or PDR partial-drop rate throughput for specified configuration. Also referred to as "long" tests.
  • refNDR throughput verification - tests verifying code patches against the reference NDR throughput. Also referred to as "short" tests.

Infrastructure - NIC devices and drivers

1609-P0

CSIT-396 Job csit-vpp-verify-perf-master-nightly-all #1044 failed Done
CSIT-346 Enable IPv6 latency in Trex driver Done
CSIT-342 Update T-rex version to v2.07 Done
CSIT-315 DPDK vhost-user - DPDK vhost-user driver, virtio in VM Done
CSIT-313 NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC Done
CSIT-312 VPP vhost-user - VPP vhost-user driver, virtio in VM Done
CSIT-177 T-rex pkill issue during initialization Done
CSIT-106 Create topology files for LF TB1, TB2, TB3 with XL710 NICs Done
view in JIRA

1609-P1

CSIT-404 Adjust DPDK testpmd startup parameters Done
CSIT-310 NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC Done
view in JIRA

1609-P2

CSIT-317 continue to expand VPP driver coverage - more NIC-DPDK drivers, other TBD Done
CSIT-314 NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC Done
CSIT-178 Interactive console vpp_api_test is timeouting during dump_stats or add_route API calls with large fib tables Done
view in JIRA

Indicative coverage = <to_be_added>

  1. [1606-DONE] NIC Intel X520 2p10GE - DPDK niantic driver, 2 ports within the same NIC.
  2. [1609-P0] NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
  3. [1609-DONE] VPP vhost-user - VPP vhost-user driver, virtio in VM.
  4. [1609-P1] NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
  5. [1609-DONE] DPDK vhost-user - DPDK vhost-user driver, virtio in VM.
  6. [1612-P0] vhost-user - XXX vhost-user driver, virtio in VM, box-full test with NxVMs.
  7. [1612-P2] continue to expand VPP driver coverage - more NIC-DPDK drivers, other TBD.

IPv4 data plane

1609-P0

CSIT-366 IPv4 DP - IPv4 routing RFC791 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio) Done
CSIT-275 IPv4 DP - QoS - ingress policing - rate metering, marking and limiting - 2r3c-(cb,ca) policer, RFC2698 Done
CSIT-169 IPv4 DP - scale - same prefix length Done
CSIT-119 IPv4 DP - security - iacl with n-tuple classification, ip4 Done
CSIT-105 IPv4 DP - security - cop white-/black-list ip4 src addr filtering Done
view in JIRA

1609-P1

CSIT-327 ip4dp xl710 - IPv4 routed forwarding baseline - xl710 nic2nic Done
view in JIRA

1609-P2

CSIT-329 IPv4 DP - VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710) Done
CSIT-328 IPv4 DP - VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520) Done
CSIT-274 IPv4 DP - QoS - ingress policing - rate metering, marking and limiting - 1r2c-(ca,cb) policer, RFC2697 Done
CSIT-273 IPv4 DP - scale - varying prefix lengths Done
CSIT-272 IPv4 DP - Continue to expand VPP driver coverage - DPDK, other TBD (Netmap?, ) Done
view in JIRA

Indicative coverage = <to_be_added>

Baseline performance

  1. [1609-P0] IPv4 routing RFC791 - baseline - NIC-to-NIC(XL710).
  2. [1609-DONE] IPv4 routing RFC791 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
  3. [1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
  4. [1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
  5. [1609-P1] extranet VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
  6. [1609-P1] extranet VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
  7. [1606-DONE] IPv4 routing RFC791 - baseline - NIC-to-NIC(X520).

FIB performance

Large-scale FIB tests

  1. [1609-DONE] same prefix length.
  2. [1609-P1] varying prefix lengths.

Security performance

  1. [1606-DONE] security - iacl with n-tuple classification, ip4.
  2. [1606-DONE] security - cop white-/black-list ip4 src addr filtering.

QoS performance

  1. [1609-REVIEW] QoS - ingress policing - rate metering, marking and limiting - 2r3c-(cb,ca) policers, RFC2698.
  2. [1609-P1] QoS - ingress policing - rate metering, marking and limiting - 1r2c-(cb,ca) policers, RFC2697.

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P2] security - source IPv4 Reverse Path Forwarding (RPF). 
   # [1609-P1] routing multipath (ECMP, UECMP) - baseline. 
   # [1609-P1] [Blocked-by VPP-xxx] carrier grade nat 44, CGN44, RFC6598, RFC6888. 
   # [1609-P2] [Blocked-by VPP-xxx] security - stateful firewall.

IPv4 control plane

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] Proxy ARP. 
   # [1609-P0] IKEv2. 
   # [1609-P1] DHCPv4 proxy. 
   # [1606-DONE] ARP. 
   # [1606-DONE] ICMPv4. 
   # [1606-DONE] DHCPv4 client.

IPv4 encapsulations

1609-P0

CSIT-200 enc - LISP Tests Done
view in JIRA

1609-P1

CSIT-288 ip4enc - ip4-gre-ip4 - with termination into main ip4 fib Done
view in JIRA

1609-P2

CSIT-289 IPv4 enc - ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp Done
view in JIRA

Indicative coverage = <to_be_added>

Baseline performance

  1. [1609-P1] ip4-gre-ip4 - baseline main fib.
  2. [1609-DONE] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp.

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] ip4-ipsec-ip4 - baseline main fib.
   # [1609-P0] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), virt2lisp.
   # [1609-P0] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
   # [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp.
   # [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
   # [1606-DONE] ip4-gre-ip4 - baseline main fib.

IPv4 telemetry

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] IPv4 IPFIX - stats collection and export - baseline. 
   # [1609-P0] IPv4 IPFIX - stats collection and export - functional scale. 
   # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline. 
   # [1609-P1] Lawful intercept  - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.

IPv6 data plane

1609-P0

CSIT-213 IPv6 DP - security - iacl with n-tuple classification, ip6 Done
CSIT-179 IPv6 DP - scale - same prefix length Done
CSIT-109 IPv6 DP - security - cop white-/black-list ip6 src addr filtering Done
view in JIRA

1609-P1

CSIT-318 ip6dp xl710 - IPv6 routed forwarding baseline - xl710 nic2nic Done
view in JIRA

1609-P2

CSIT-319 IPv6 DP - VPNv6 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710) Done
CSIT-303 IPv6 DP - QoS - ingress policing - rate metering, marking and rate-limiting Done
CSIT-302 IPv6 DP - scale - varying prefix lengths Done
CSIT-301 IPv6 DP - VPN routed forwarding (VRF) - baseline Done
CSIT-300 IPv6 DP - continue to expand VPP driver coverage - DPDK, other TBD (Netmap?, ) Done
view in JIRA

Indicative coverage = <to_be_added>

Baseline performance

  1. [1609-P0] IPv6 routing RFC2460 - baseline - NIC-to-NIC(XL710).
  2. [1609-P0] IPv6 routing RFC2460 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
  3. [1609-P1] VPNv6 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
  4. [1609-P1] VPNv6 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
  5. [1609-P2] extranet VPN routed forwarding (VRF) - baseline.
  6. [1606-DONE] IPv6 routing RFC2460 - baseline - NIC-to-NIC(X520).

FIB performance

Large-scale FIB tests

  1. [1609-DONE] same prefix length.
  2. [1609-P1] varying prefix lengths.

Security performance

  1. [1609-DONE] security - iacl with n-tuple classification, ip6.
  2. [1606-DONE] security - cop white-/black-list ip6 src addr filtering.

QoS performance

  1. [1609-P1] QoS - ingress policing - rate metering, marking and rate-limiting -1r2c-(cb,ca), 2r3c-(cb,ca) policers, RFC2475, RFC2697, RFC2698.

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] security - source IPv6 Reverse Path Forwarding (RPF). 
   # [1609-P1] routing multipath (ECMP, UECMP) - baseline.
   # [1609-P2] [Blocked-by VPP-xxx] security - stateful firewall.

IPv6 control plane

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] router advertisement.
   # [1609-P1] DHCPv6 proxy.
   # [1609-P2] [Blocked-by VPP-xxx] ip6-ioam, in-band OAM.
   # [1606-DONE] neighbor discovery, RFC4861.
   # [1606-DONE] ICMPv6.

IPv6 encapsulations

1609-P0

CSIT-200 enc - LISP Tests Done
view in JIRA

1609-P1

view in JIRA

1609-P2

CSIT-320 IPv6 enc - ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp Done
view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-DONE] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp.
  2. [1609-DONE] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp.

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] ip4-o-ip6 softwire.
   # [1609-P0] ip6-ipsec-ip6.
   # [1609-P0] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), virt2lisp.
   # [1609-P0] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp.
   # [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), virt2lisp.
   # [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp.
   # [1609-P0] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp.
   # [1609-P0] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
   # [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp.
   # [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. ?
   # [1609-P1] [Blocked-by VPP-xxx] ip6-segment-routing.

IPv6 telemetry

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] IPv6 IPFIX - stats collection and export - baseline.
   # [1609-P0] IPv6 IPFIX - stats collection and export - functional scale.
   # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline.
   # [1609-P1] Lawful intercept  - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.

Ethernet L2 data plane

1609-P0

CSIT-324 L2dp - Bridge domain - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio) Done
CSIT-323 L2dp - L2 Cross-connect - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio) Done
CSIT-321 L2dp - L2 Cross-connect - baseline - NIC-to-NIC(XL710 Done
view in JIRA

1609-P1

CSIT-322 l2dp xl710 - L2 bridge domain baseline - xl710 nic2nic Done
view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

Baseline performance

  1. [1609-P0] L2 Cross-connect - baseline - NIC-to-NIC(XL710).
  2. [1609-P0] Bridge domain - baseline - NIC-to-NIC(XL710).
  3. [1609-DONE] L2 Cross-connect - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
  4. [1609-DONE] Bridge domain - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
  5. [1606-DONE] L2 Cross-connect - baseline - NIC-to-NIC(X520).
  6. [1606-DONE] Bridge domain - baseline - NIC-to-NIC(X520).

FIB performance

Large-scale FIB tests

  1. [1609-P1] [Blocked-by VPP-201] MAC addresses.

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] Bridge domain - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2.
   # [1609-P1] Integrated Routing-Bridging (IRB) - baseline.
   # [1606-DONE] L2 Cross-connect - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2.

Ethernet L2 control plane

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>
  2. [1606-DONE] MAC learning.

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P2] [Blocked-by VPP-xxx] CDP/LLDP.
   # [1606-DONE] MAC split-horizon.

Ethernet L2 encapsulations

1609-P0

CSIT-218 L2enc - l2-vxlan-ipv4 - with L2 cross-connect Done
view in JIRA

1609-P1

CSIT-326 l2enc - l2-eth-lispgpe-ip4 with termination into L2BD Done
CSIT-325 l2enc - l2-eth-lispgpe-ip6 with termination into L2BD Done
view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-P0] l2-eth-lispgpe-ip6 - L2BD.
  2. [1609-P0] l2-eth-lispgpe-ip4 - L2BD.
  3. [1609-P1] l2-vxlan-ipv4 - with bridge-domain.
  4. [1609-DONE] l2-vxlan-ipv4 - with L2 cross-connect.

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P0] l2-eth-lispgpe-ip6 - L2BD, L2XC.
   # [1609-P0] l2-eth-lispgpe-ip4 - L2BD, L2XC.
   # [1609-P0] l2eth-gre-ip4-ipsec-ip4-eth - L2BD, L2XC.
   # [1609-P2] l2-eth-l2tpv3-ip4 - L2BD, L2XC.
   # [1609-P2] l2-eth-l2tpv3-ip6 - L2BD, L2XC.
   # [1609-P2] l2-eth-gre-ip4 - L2BD, L2XC.
   # [1609-P2] l2-eth-gre-ip6 - L2BD, L2XC.

L2 telemetry

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P1] L2 FIB - stats collection and export - baseline.
   # [1609-P2] L2 FIB - stats collection and export - functional scale.
   # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline.
   # [1609-P1] Lawful intercept  - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.

MPLS data plane

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P2] vpnv4-mpls-gre-ip4.
   # [1609-P2] mpls-o-eth.

NSH data plane

1609-P0

view in JIRA

1609-P1

view in JIRA

1609-P2

view in JIRA

Indicative coverage = <to_be_added>

  1. [1609-Px] <to_be_added>

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only: 

   # [1609-P1] ip4-nsh-gre-ip4.
   # [1609-P1] ip4-nsh-vxlan-gpe-ip4.

END OF NOTE

Retrieved from "https://wiki.fd.io/index.php?title=CSIT/PerfTestPlan&oldid=3097"