Difference between revisions of "CSIT/PerfTestPlan"

Revision as of 07:22, 15 April 2016

FD.io IT systems integration

Three Jenkins jobs for CSIT performance test cases are in place today:

1. Job name: csit-vpp-functional-hw-tb2
   • Description: This job is testing csit project against performance test cases
     • TODO items
       • name - rename to "csit-vpp-performance-hw" or better "csit-vpp-perf-hw"
         • [IN REVIEW, pmikus] gerrit.fd.io change
       • scope - currently only tb2, need to add other testbeds: tb1, tb3
       • scope - tbX reservation script and tbX scheduler code already done and tested, needs to be used
   • Trigger: Comment "hw-perf-check" in proposed patch in CSIT project
     • TODO items
       • rename to "verify-perf-all"
         • [IN REVIEW, pmikus] gerrit.fd.io change
       • consider adding later on "verify-perf-short", "verify-perf-long"
   • Status: WORKING
2. Job name: vpp-verify-performance-long
   • Description: This job is testing vpp proposed patch against long performance test cases
   • Trigger: Comment "verify-perf-long" in proposed patch in VPP project
   • Status: WORKING
3. Job name: vpp-verify-performance-short
   • Description: This job is testing vpp proposed patch against short performance test cases
   • Trigger: Comment "verify-perf-short" in proposed patch in VPP project
   • Status: WORKING

CSIT performance test cases - working

1. Working performance test cases - 18 working in FD.io physical testbeds today.
2. Last manual run results reports:
   • https://jenkins.fd.io/view/csit/job/csit-vpp-functional-hw-tb2/
   • https://jenkins.fd.io/view/vpp/job/vpp-verify-performance-long/
   • https://jenkins.fd.io/view/vpp/job/vpp-verify-performance-short/
3. List of working performance test cases
   1. ~/csit$ grep "^| [a-zA-Z1-9]" -nr tests | grep -vi " | " | sort | grep performance
   2. tests/suites/performance/long_bridge_domain.robot:24:| Find NDR by using linear search and 64B frames through bridge domain in 3-node topology
      • // TODO items: rename to `Find NDR by using RFC2544 linear search and 64B frames through bridge domain in 3-node topology`
   3. tests/suites/performance/long_bridge_domain.robot:34:| Find NDR by using linear search and 1518B frames through bridge domain in 3-node topology
   4. tests/suites/performance/long_bridge_domain.robot:44:| Find NDR by using linear search and 9000B frames through bridge domain in 3-node topology
   5. tests/suites/performance/long_ipv4.robot:28:| Find NDR by using linear search and 64B frames through IPv4 forwarding in 3-node topology
   6. tests/suites/performance/long_ipv4.robot:38:| Find NDR by using linear search and 1518B frames through IPv4 forwarding in 3-node topology
   7. tests/suites/performance/long_ipv4.robot:48:| Find NDR by using linear search and 9000B frames through IPv4 forwarding in 3-node topology
   8. tests/suites/performance/long_xconnect.robot:25:| Find NDR by using linear search and 64B frames through L2 cross connect in 3-node topology
   9. tests/suites/performance/long_xconnect.robot:35:| Find NDR by using linear search and 1518B frames through L2 cross connect in 3-node topology
   10. tests/suites/performance/long_xconnect.robot:45:| Find NDR by using linear search and 9000B frames through L2 cross connect in 3-node topology
   11. tests/suites/performance/short_bridge_domain.robot:23:| 1core VPP passes 64B frames through bridge domain at 3.5mpps in 3-node topology
   12. tests/suites/performance/short_bridge_domain.robot:30:| 1core VPP passes 1518B frames through bridge domain at 10gbps in 3-node topology
   13. tests/suites/performance/short_bridge_domain.robot:37:| 1core VPP passes 9000B frames through bridge domain at 10gbps in 3-node topology
   14. tests/suites/performance/short_ipv4.robot:27:| 1core VPP passes 64B frames through IPv4 forwarding at 3.5mpps in 3-node topology
       • // TODO items: rename to `1core VPP passes 64B frames through IPv4 forwarding at 2x 3.5Mpps in 3-node topology`
       • // TODO items: currently testing all functional combinations takes 10-to-15 minutes; going forward would need to split the functional combinations
   15. tests/suites/performance/short_ipv4.robot:34:| 1core VPP passes 1518B frames through IPv4 forwarding at 10gbps in 3-node topology
   16. tests/suites/performance/short_ipv4.robot:41:| 1core VPP passes 9000B frames through IPv4 forwarding at 10gbps in 3-node topology
   17. tests/suites/performance/short_xconnect.robot:24:| 1core VPP passes 64B frames through L2 cross connect at 3.5mpps in 3-node topology
   18. tests/suites/performance/short_xconnect.robot:31:| 1core VPP passes 1518B frames through L2 cross connect at 10gbps in 3-node topology
   19. tests/suites/performance/short_xconnect.robot:38:| 1core VPP passes 9000B frames through L2 cross connect at 10gbps in 3-node topology

CSIT performance test cases - plan and status

1. [DONE, mmiklus] DropRateSearch
   • [DONE, pmikus] binary search gerrit.fd.io change
   • [DONE, pmikus] BESTofN/WORSTofN gerrit.fd.io change
   • [DONE, pmikus] combined search gerrit.fd.io change
2. [WIP, mmiklus] extend topology compute library with NIC type filtering
3. [IN REVIEW, pmikus] multicore tests gerrit.fd.io change
4. [TBD] IPv6
5. [TBD] Large-scale FIB tests
   • IPv4
   • IPv6
   • mac
   • Various prefix lengths
6. [TBD] Encapsulations
   • VxLAN
   • VLAN
   • GRE
   • IPSEC
7. [TBD] Create topology files for tb1, tb3
8. [TBD] Host, driver, VPP baseline performance
   • [TBD] User-mode app (L3FWD-xconnect) with DPDK driver per NIC
   • [TBD] User-mode VPP baseline config with the same DPDK driver per NIC
9. [TBD] vNet VPP topologies
   • [TBD] Phy1-VPP-Phy2
   • [TBD] Phy1-VPP-VM-VPP-Phy2
10. [TBD] T-REX VXLAN traffic profile & CSIT tests
11. IxNetwork driver
    • [TBD] driver design
    • [TBD] driver implementation
      • setup, teardown
      • traffic stream configuration
      • statistics retrieval
    • [TBD] latency measurements
      • feasibility
      • implementation
    • [TBD] integration with CSIT tests
12. Moongen driver
    • [TBD] verify requirements and available HW against latest MoonGen codebase
    • [TBD] integration
13. T-REX
    • [TBD] latency (T-REX support to be confirmed)
14. Other work items
    • [TBD] integrate Python sysinfosuite to CSIT
    • [TBD] unified performance trend dashboard