Difference between revisions of "CSIT/PerfTestPlan"
Mackonstan (Talk | contribs) |
Hungtuanpham (Talk | contribs) |
||
Line 49: | Line 49: | ||
# [1606-DONE] NIC Intel X520 2p10GE - DPDK niantic driver, 2 ports within the same NIC. | # [1606-DONE] NIC Intel X520 2p10GE - DPDK niantic driver, 2 ports within the same NIC. | ||
# [1609-P0] NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC. | # [1609-P0] NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC. | ||
− | # [1609-P0] VPP vhost-user - VPP vhost-user driver, virtio in VM. | + | # [1609-P0] VPP vhost-user - VPP vhost-user driver, virtio in VM. [https://jira.fd.io/browse/CSIT-312 CSIT-312] |
# [1609-P1] NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC. | # [1609-P1] NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC. | ||
# [1612-P0] DPDK vhost-user - DPDK vhost-user driver, virtio in VM. | # [1612-P0] DPDK vhost-user - DPDK vhost-user driver, virtio in VM. |
Revision as of 19:26, 27 July 2016
WORK IN PROGRESS - PLEASE TREAT IT AS SUCH
Contents
- 1 CSIT development plan - background
- 2 CSIT performance tests - dev plan
- 2.1 NIC devices and drivers
- 2.2 IPv4 data plane
- 2.3 IPv4 control plane
- 2.4 IPv4 encapsulations
- 2.5 IPv4 telemetry
- 2.6 IPv6 data plane
- 2.7 IPv6 control plane
- 2.8 IPv6 encapsulations
- 2.9 IPv6 telemetry
- 2.10 Ethernet L2 data plane
- 2.11 Ethernet L2 control plane
- 2.12 Ethernet L2 encapsulations
- 2.13 L2 telemetry
- 2.14 MPLS data plane
- 2.15 NSH data plane
CSIT development plan - background
CSIT test development priorities are aligned with VPP releases timeline, with relative dev work prioritization for each VPP release driven by perceived importance of test verification for VPP functionality and performance. Specific CSIT dev work priorities are driven by rough consensus within the project and contributors and committers availability and willingness to do work.
Following priority tagging scheme is used in this CSIT wiki and FD.io jira for CSIT:
- [$VppRlsNo-$RelPrio]
- VppRlsNo - aligned VPP release number - 16.09, 16.12, 17.03.
- RelPrio - CSIT release priority - P0 for Must Have, P1 for Should Have, P2 for Could Have
- CSIT dev work tagging examples:
- [1609-P0] - CSIT Must Have for vpp-v1609
- [1609-P1] - CSIT Should Have for vpp-v1609
- [1609-P2] - CSIT Could Have for vpp-v1609
CSIT dev work plan execution tracked on CSIT jira, with priority tags encoded as Jira labels for easy of tracking and reporting.
CSIT performance tests - dev plan
CSIT performance test cases devevelopment work is grouped and tracked across the following VPP functional areas:
- NIC devices and drivers
- IPv4 data plane
- IPv4 control plane
- IPv4 encapsulations
- IPv4 telemetry
- IPv6 data plane
- IPv6 control plane
- IPv6 encapsulations
- IPv6 telemetry
- Ethernet L2 data plane
- Ethernet L2 control plane
- Ethernet L2 encapsulations
- Ethernet L2 telemetry
- MPLS data plane
- NSH data plane
CSIT team is tracking completed CSIT dev work per above area (completed = test cases running in FD.io), as well as top-level test coverage metric [%] of VPP functions. VPP code development dependencies are marked with [Blocked-by VPP-XXX]. Performance tests require the functional tests to be completed - see sub-sections "Functional tests preconditions".
CSIT performance tests are split into the two main groups:
- NDR(PDR) throughput discovery - tests searching for NDR non-drop rate throughput or PDR partial-drop rate throughput for specified configuration. Also referred to as "long" tests.
- refNDR throughput verification - tests verifying code patches against the reference NDR throughput. Also referred to as "short" tests.
NIC devices and drivers
- [1606-DONE] NIC Intel X520 2p10GE - DPDK niantic driver, 2 ports within the same NIC.
- [1609-P0] NIC Intel XL710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
- [1609-P0] VPP vhost-user - VPP vhost-user driver, virtio in VM. CSIT-312
- [1609-P1] NIC Intel X710 2p40GE - DPDK i40e driver, 2 ports within the same NIC.
- [1612-P0] DPDK vhost-user - DPDK vhost-user driver, virtio in VM.
- [1612-P0] vhost-user - XXX vhost-user driver, virtio in VM, box-full test with NxVMs.
- [1612-P2] continue to expand VPP driver coverage - more NIC-DPDK drivers, other TBD.
IPv4 data plane
JIRA board view, JIRA list view - ip4dp label for CSIT work tracking.
Indicative coverage = <to_be_added>
Baseline performance
- [1609-P0] IPv4 routing RFC791 - baseline - NIC-to-NIC(XL710).
- [1609-P0] IPv4 routing RFC791 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio), CSIT-272.
- [1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(X520).
- [1609-P1] VPNv4 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
- [1606-DONE] IPv4 routing RFC791 - baseline - NIC-to-NIC(X520).
FIB performance
Large-scale FIB tests
- [1609-P0] IPv4 same prefix length, Gerrit-#1613, CSIT-169.
- [1609-P1] IPv4 varying prefix lengths, CSIT-273.
Security performance
- [1606-DONE] security - iacl with n-tuple classification, ip4. NDR, refNDR.
- [1606-DONE] security - cop white-/black-list ip4 src addr filtering. NDR, refNDR.
QoS performance
- [1609-P0] QoS - ingress policing - rate metering, marking and limiting - 2r3c-color-blind policer. CSIT-274
- [1609-P1] QoS - ingress policing - rate metering, marking and limiting - 1r2c-color-blind policer. CSIT-275
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] VPN routed forwarding (VRF) - baseline. CSIT-276 # [1609-P0] extranet VPN routed forwarding (VRF) - baseline. CSIT-277 # [1609-P0] security - source IPv4 Reverse Path Forwarding (RPF). CSIT-278 # [1609-P1] routing multipath (ECMP, UECMP) - baseline. CSIT-279 # [1609-P1] [Blocked-by VPP-xxx] carrier grade nat 44, CGN44, RFC6598, RFC6888. CSIT-280 # [1609-P2] [Blocked-by VPP-xxx] security - stateful firewall. CSIT-281
IPv4 control plane
JIRA board view, [ https://jira.fd.io/issues/?filter=10644 JIRA list view] - ip4cp label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] Proxy ARP. CSIT-282 # [1609-P0] IKEv2. CSIT-283 # [1609-P1] DHCPv4 proxy. CSIT-284 # [1606-DONE] ARP. CSIT-285 # [1606-DONE] ICMPv4. CSIT-286 # [1606-DONE] DHCPv4 client. CSIT-287
IPv4 encapsulations
JIRA board view, JIRA list view - ip4enc label for CSIT work tracking.
Indicative coverage = <to_be_added>
Baseline performance
- [1609-P1] ip4-gre-ip4 - baseline main fib. CSIT-288
- [1609-P1] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp. CSIT-289
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] ip4-ipsec-ip4 - baseline main fib. CSIT-290 # [1609-P0] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-291 # [1609-P0] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-292 # [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-293 # [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-294 # [1606-DONE] ip4-gre-ip4 - baseline main fib. CSIT-295
IPv4 telemetry
JIRA board view, JIRA list view - ip4tel label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] IPv4 IPFIX - stats collection and export - baseline. CSIT-296 # [1609-P0] IPv4 IPFIX - stats collection and export - functional scale. CSIT-297 # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline. CSIT-298 # [1609-P1] Lawful intercept - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline. CSIT-299
IPv6 data plane
JIRA board view, JIRA list view - ip6dp label for CSIT work tracking.
Indicative coverage = <to_be_added>
Baseline performance
- [1609-P0] IPv6 routing RFC2460 - baseline - NIC-to-NIC(XL710).
- [1609-P0] IPv6 routing RFC2460 - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio), [CSIT-300].
- [1609-P1] VPNv6 routed forwarding (VRF) - baseline - NIC-to-NIC(X520), [CSIT-301].
- [1609-P1] VPNv6 routed forwarding (VRF) - baseline - NIC-to-NIC(XL710).
- [1606-DONE] IPv6 routing RFC2460 - baseline - NIC-to-NIC(X520).
FIB performance
Large-scale FIB tests
- [1609-REVIEW] IPv6 same prefix length, Gerrit #1664 Jira CSIT-179.
- [1609-P1] IPv6 varying prefix lengths, CSIT-302.
Security performance
- [1609-DONE] security - iacl with n-tuple classification, ip6. NDR, refNDR.
- [1606-DONE] security - cop white-/black-list ip6 src addr filtering. NDR, refNDR.
QoS performance
- [1609-P1] QoS - ingress policing - rate metering, marking and rate-limiting, CSIT-303.
Functional tests preconditions
- [1606-DONE] routing [rfc2460] - baseline.
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] VPN routed forwarding (VRF) - baseline. CSIT-304 # [1609-P0] extranet VPN routed forwarding (VRF) - baseline. CSIT-305 # [1609-P0] security - source IPv6 Reverse Path Forwarding (RPF). CSIT-306 # [1609-P0] qos - ingress policing - 1r2c-(cb,ca), 2r3c-(cb,ca) policers, RFC2475, RFC2697, RFC2698. CSIT-307 # [1609-P1] routing multipath (ECMP, UECMP) - baseline. CSIT-308 # [1609-P2] [Blocked-by VPP-xxx] security - stateful firewall. CSIT-309
IPv6 control plane
JIRA board view, JIRA list view - ip6cp label for CSIT work tracki
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] router advertisement. # [1609-P1] DHCPv6 proxy. # [1609-P2] [Blocked-by VPP-xxx] ip6-ioam, in-band OAM. # [1606-DONE] neighbor discovery, RFC4861. # [1606-DONE] ICMPv6.
IPv6 encapsulations
JIRA board view, JIRA list view - ip6enc label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-P1] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp.
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] ip4-o-ip6 softwire. # [1609-P0] ip6-ipsec-ip6. # [1609-P0] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. # [1609-P0] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. # [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. # [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. # [1609-P0] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. # [1609-P0] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. # [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. # [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. # [1609-P1] [Blocked-by VPP-xxx] ip6-segment-routing.
IPv6 telemetry
JIRA board view, JIRA list view - ip6tel label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] IPv6 IPFIX - stats collection and export - baseline. # [1609-P0] IPv6 IPFIX - stats collection and export - functional scale. # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline. # [1609-P1] Lawful intercept - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.
Ethernet L2 data plane
JIRA board view, JIRA list view - l2dp label for CSIT work tracking.
Indicative coverage = <to_be_added>
Baseline performance
- [1609-P0] L2 Cross-connect - baseline - NIC-to-NIC(XL710).
- [1609-P0] Bridge domain - baseline - NIC-to-NIC(XL710).
- [1609-P0] L2 Cross-connect - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
- [1609-P0] Bridge domain - baseline vhost-user - NIC(X520)-to-VM(l3fwd,virtio).
- [1606-DONE] L2 Cross-connect - baseline - NIC-to-NIC(X520).
- [1606-DONE] Bridge domain - baseline - NIC-to-NIC(X520).
FIB performance
Large-scale FIB tests
- [1609-P0] [Blocked-by VPP-201] MAC addresses - Gerrit #1862
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] Bridge domain - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2. # [1609-P1] Integrated Routing-Bridging (IRB) - baseline. # [1606-DONE] L2 Cross-connect - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2.
Ethernet L2 control plane
JIRA board view, JIRA list view - l2cp label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
- [1606-DONE] MAC learning.
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P2] [Blocked-by VPP-xxx] CDP/LLDP. # [1606-DONE] MAC split-horizon.
Ethernet L2 encapsulations
JIRA board view, JIRA list view - l2enc label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-P0] l2-eth-lispgpe-ip6 - L2BD.
- [1609-P0] l2-eth-lispgpe-ip4 - L2BD.
- [1606-DONE] l2-vxlan-ipv4 - with bridge-domain.
- [1606-DONE] l2-vxlan-ipv4 - with L2 cross-connect.
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P0] l2-eth-lispgpe-ip6 - L2BD, L2XC. # [1609-P0] l2-eth-lispgpe-ip4 - L2BD, L2XC. # [1609-P0] l2eth-gre-ip4-ipsec-ip4-eth - L2BD, L2XC. # [1609-P2] l2-eth-l2tpv3-ip4 - L2BD, L2XC. # [1609-P2] l2-eth-l2tpv3-ip6 - L2BD, L2XC. # [1609-P2] l2-eth-gre-ip4 - L2BD, L2XC. # [1609-P2] l2-eth-gre-ip6 - L2BD, L2XC. # [1606-DONE] l2-vxlan-ipv4 - with bridge-domain. # [1606-DONE] l2-vxlan-ipv4 - with L2 cross-connect.
L2 telemetry
JIRA board view, JIRA list view - l2tel label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P1] L2 FIB - stats collection and export - baseline. # [1609-P2] L2 FIB - stats collection and export - functional scale. # [1609-P0] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline. # [1609-P1] Lawful intercept - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline.
MPLS data plane
[ JIRA board view], [ JIRA list view] - mplsdp label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P2] vpnv4-mpls-gre-ip4. # [1609-P2] mpls-o-eth.
NSH data plane
[ JIRA board view], [ JIRA list view] - nshdp label for CSIT work tracking.
Indicative coverage = <to_be_added>
- [1609-Px] <to_be_added>
Functional tests preconditions
Below copied from CSIT functional tests plan - provided for reference only:
# [1609-P1] ip4-nsh-gre-ip4. # [1609-P1] ip4-nsh-vxlan-gpe-ip4.
END OF NOTE