WORK IN PROGRESS - PLEASE TREAT IT AS SUCH

CSIT development plan - background

CSIT test development priorities are aligned with VPP releases timeline, with relative dev work prioritization for each VPP release driven by perceived importance of test verification for VPP functionality and performance. Specific CSIT dev work priorities are driven by rough consensus within the project and contributors and committers availability and willingness to do work.

Following priority tagging scheme is used in this CSIT wiki and FD.io jira for CSIT:

[$VppRlsNo-$RelPrio]
- VppRlsNo - aligned VPP release number - 16.09, 16.12, 17.03.
- RelPrio - CSIT release priority - P0 for Must Have, P1 for Should Have, P2 for Could Have
CSIT dev work tagging examples:
- [1609-P0] - CSIT Must Have for vpp-v1609
- [1609-P1] - CSIT Should Have for vpp-v1609
- [1609-P2] - CSIT Could Have for vpp-v1609

CSIT dev work plan execution tracked on CSIT jira, with priority tags encoded as Jira labels for easy of tracking and reporting.

CSIT performance tests - dev plan

CSIT performance test cases devevelopment work is grouped and tracked across the following VPP functional areas:

NIC devices and drivers
IPv4 data plane
IPv4 control plane
IPv4 encapsulations
IPv4 telemetry
IPv6 data plane
IPv6 control plane
IPv6 encapsulations
IPv6 telemetry
Ethernet L2 data plane
Ethernet L2 control plane
Ethernet L2 encapsulations
Ethernet L2 telemetry
MPLS data plane
NSH data plane

CSIT team is tracking completed CSIT dev work per above area (completed = test cases running in FD.io), as well as top-level test coverage metric [%] of VPP functions. VPP code development dependencies are marked with [Blocked-by VPP-XXX]. Performance tests require the functional tests to be completed - see sub-sections "Functional tests preconditions".

CSIT performance tests are split into the two main groups:

NDR(PDR) throughput discovery - tests searching for NDR non-drop rate throughput or PDR partial-drop rate throughput for specified configuration. Also referred to as "long" tests.
refNDR throughput verification - tests verifying code patches against the reference NDR throughput. Also referred to as "short" tests.

Infrastructure - NIC devices and drivers

1609-P0

Bug: jira.translation.issuetype.bug.name.desc CSIT-396 Job csit-vpp-verify-perf-master-nightly-all #1044 failed Done
Improvement: An improvement or enhancement to an existing feature or task. CSIT-346 Enable IPv6 latency in Trex driver Done
Task: A task that needs to be done. CSIT-342 Update T-rex version to v2.07 Done
Sub-task: The sub-task of the issue CSIT-315 DPDK vhost-user - DPDK vhost-user driver, virtio in VM Done
New Feature: A new feature of the product, which has yet to be developed. CSIT-313 NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC Done
CSIT-312 VPP vhost-user - VPP vhost-user driver, virtio in VM Done
Bug: jira.translation.issuetype.bug.name.desc CSIT-177 T-rex pkill issue during initialization Done
Task: A task that needs to be done. CSIT-106 Create topology files for LF TB1, TB2, TB3 with XL710 NICs Done
view in JIRA

1609-P1

Improvement: An improvement or enhancement to an existing feature or task. CSIT-404 Adjust DPDK testpmd startup parameters Done
New Feature: A new feature of the product, which has yet to be developed. CSIT-310 NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC Done
view in JIRA

1609-P2

New Feature: A new feature of the product, which has yet to be developed. CSIT-317 continue to expand VPP driver coverage - more NIC-DPDK drivers, other TBD Done
CSIT-314 NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC Done
Bug: jira.translation.issuetype.bug.name.desc CSIT-178 Interactive console vpp_api_test is timeouting during dump_stats or add_route API calls with large fib tables Done
view in JIRA

Indicative coverage = <to_be_added>

[1606-DONE] NIC Intel X520 2p10GE - DPDK niantic driver, 2 ports within the same NIC.
[1609-P0] NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
[1609-DONE] VPP vhost-user - VPP vhost-user driver, virtio in VM.
[1609-P1] NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
[1609-DONE] DPDK vhost-user - DPDK vhost-user driver, virtio in VM.
[1612-P0] vhost-user - XXX vhost-user driver, virtio in VM, box-full test with NxVMs.
[1612-P2] continue to expand VPP driver coverage - more NIC-DPDK drivers, other TBD.

IPv4 data plane

1609-P0

Sub-task: The sub-task of the issue CSIT-366 IPv4 DP - IPv4 routing RFC791 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio) Done
New Feature: A new feature of the product, which has yet to be developed. CSIT-275 IPv4 DP - QoS - ingress policing - rate metering, marking and limiting - 2r3c-(cb,ca) policer, RFC2698 Done
CSIT-169 IPv4 DP - scale - same prefix length Done
CSIT-119 IPv4 DP - security - iacl with n-tuple classification, ip4 Done
Task: A task that needs to be done. CSIT-105 IPv4 DP - security - cop white-/black-list ip4 src addr filtering Done
view in JIRA

1609-P1

Sub-task: The sub-task of the issue CSIT-327 ip4dp xl710 - IPv4 routed forwarding baseline - xl710 nic2nic Done
view in JIRA

1609-P2

Sub-task: The sub-task of the issue CSIT-329 IPv4 DP - VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710) Done
New Feature: A new feature of the product, which has yet to be developed. CSIT-328 IPv4 DP - VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520) Done
CSIT-274 IPv4 DP - QoS - ingress policing - rate metering, marking and limiting - 1r2c-(ca,cb) policer, RFC2697 Done
CSIT-273 IPv4 DP - scale - varying prefix lengths Done
New Feature: A new feature of the product, which has yet to be developed. CSIT-272 IPv4 DP - Continue to expand VPP driver coverage - DPDK, other TBD (Netmap?, ) Done
view in JIRA

Indicative coverage = <to_be_added>

Baseline performance

[1609-P0] IPv4 routing RFC791 - baseline - NIC-to-NIC(XL710).
[1609-DONE] IPv4 routing RFC791 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
[1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
[1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
[1609-P1] extranet VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
[1609-P1] extranet VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
[1606-DONE] IPv4 routing RFC791 - baseline - NIC-to-NIC(X520).

FIB performance

Large-scale FIB tests

[1609-DONE] same prefix length.
[1609-P1] varying prefix lengths.

Security performance

[1606-DONE] security - iacl with n-tuple classification, ip4.
[1606-DONE] security - cop white-/black-list ip4 src addr filtering.

QoS performance

[1609-REVIEW] QoS - ingress policing - rate metering, marking and limiting - 2r3c-(cb,ca) policers, RFC2698.
[1609-P1] QoS - ingress policing - rate metering, marking and limiting - 1r2c-(cb,ca) policers, RFC2697.

Functional tests preconditions

Below copied from CSIT functional tests plan - provided for reference only:

   # [1609-P2] security - source IPv4 Reverse Path Forwarding (RPF). 
   # [1609-P1] routing multipath (ECMP, UECMP) - baseline. 
   # [1609-P1] [Blocked-by VPP-xxx] carrier grade nat 44, CGN44, RFC6598, RFC6888. 
   # [1609-P2] [Blocked-by VPP-xxx] security - stateful firewall.

IPv4 control plane

1609-P0

CSIT/PerfTestPlan

Contents

CSIT development plan - background

CSIT performance tests - dev plan

Infrastructure - NIC devices and drivers

IPv4 data plane

IPv4 control plane

IPv4 encapsulations

IPv4 telemetry

IPv6 data plane

IPv6 control plane

IPv6 encapsulations

IPv6 telemetry

Ethernet L2 data plane

Ethernet L2 control plane

Ethernet L2 encapsulations

L2 telemetry

MPLS data plane

NSH data plane

Navigation menu

Search