CSIT/PerfTestPlan

From fd.io
< CSIT
Revision as of 13:06, 13 April 2016 by Mackonstan (Talk | contribs)

Jump to: navigation, search

FD.io IT systems integration

Three Jenkins jobs for CSIT performance test cases are in place today:

  1. Job name: csit-vpp-functional-hw-tb2
    • Description: This job is testing csit project against performance test cases
      • TODO items
        • name - rename to "csit-vpp-performance-hw" or better "csit-vpp-perf-hw"
        • scope - currently only tb2, need to add other testbeds: tb1, tb3
        • scope - tbX reservation script and tbX scheduler code already done and tested, needs to be used
    • Trigger: Comment "hw-perf-check" in proposed patch in CSIT project
      • TODO items
        • rename to "verify-perf-all"
        • consider adding later on "verify-perf-short", "verify-perf-long"
    • Status: WORKING
  2. Job name: vpp-verify-performance-long
    • Description: This job is testing vpp proposed patch against long performance test cases
    • Trigger: Comment "verify-perf-long" in proposed patch in VPP project
    • Status: WORKING
  3. Job name: vpp-verify-performance-short
    • Description: This job is testing vpp proposed patch against short performance test cases
    • Trigger: Comment "verify-perf-short" in proposed patch in VPP project
    • Status: WORKING

CSIT performance test cases - working

  1. Working performance test cases - 18 working in FD.io physical testbeds today.
  2. Last manual run results reports:
  3. List of working performance test cases
    1. ~/csit$ grep "^| [a-zA-Z1-9]" -nr tests | grep -vi " | " | sort | grep performance
    2. tests/suites/performance/long_bridge_domain.robot:24:| Find NDR by using linear search and 64B frames through bridge domain in 3-node topology
      • // TODO items: rename to `Find NDR by using RFC2544 linear search and 64B frames through bridge domain in 3-node topology`
    3. tests/suites/performance/long_bridge_domain.robot:34:| Find NDR by using linear search and 1518B frames through bridge domain in 3-node topology
    4. tests/suites/performance/long_bridge_domain.robot:44:| Find NDR by using linear search and 9000B frames through bridge domain in 3-node topology
    5. tests/suites/performance/long_ipv4.robot:28:| Find NDR by using linear search and 64B frames through IPv4 forwarding in 3-node topology
    6. tests/suites/performance/long_ipv4.robot:38:| Find NDR by using linear search and 1518B frames through IPv4 forwarding in 3-node topology
    7. tests/suites/performance/long_ipv4.robot:48:| Find NDR by using linear search and 9000B frames through IPv4 forwarding in 3-node topology
    8. tests/suites/performance/long_xconnect.robot:25:| Find NDR by using linear search and 64B frames through L2 cross connect in 3-node topology
    9. tests/suites/performance/long_xconnect.robot:35:| Find NDR by using linear search and 1518B frames through L2 cross connect in 3-node topology
    10. tests/suites/performance/long_xconnect.robot:45:| Find NDR by using linear search and 9000B frames through L2 cross connect in 3-node topology
    11. tests/suites/performance/short_bridge_domain.robot:23:| 1core VPP passes 64B frames through bridge domain at 3.5mpps in 3-node topology
    12. tests/suites/performance/short_bridge_domain.robot:30:| 1core VPP passes 1518B frames through bridge domain at 10gbps in 3-node topology
    13. tests/suites/performance/short_bridge_domain.robot:37:| 1core VPP passes 9000B frames through bridge domain at 10gbps in 3-node topology
    14. tests/suites/performance/short_ipv4.robot:27:| 1core VPP passes 64B frames through IPv4 forwarding at 3.5mpps in 3-node topology
      • // TODO items: rename to `1core VPP passes 64B frames through IPv4 forwarding at 2x 3.5Mpps in 3-node topology`
      • // TODO items: currently testing all functional combinations takes 10-to-15 minutes; going forward would need to split the functional combinations
    15. tests/suites/performance/short_ipv4.robot:34:| 1core VPP passes 1518B frames through IPv4 forwarding at 10gbps in 3-node topology
    16. tests/suites/performance/short_ipv4.robot:41:| 1core VPP passes 9000B frames through IPv4 forwarding at 10gbps in 3-node topology
    17. tests/suites/performance/short_xconnect.robot:24:| 1core VPP passes 64B frames through L2 cross connect at 3.5mpps in 3-node topology
    18. tests/suites/performance/short_xconnect.robot:31:| 1core VPP passes 1518B frames through L2 cross connect at 10gbps in 3-node topology
    19. tests/suites/performance/short_xconnect.robot:38:| 1core VPP passes 9000B frames through L2 cross connect at 10gbps in 3-node topology

CSIT performance test cases - plan and status

  1. [DONE, mmiklus] DropRateSearch
  2. [WIP, mmiklus] extend topology compute library with NIC type filtering
  3. [WIP, pmikus] multicore tests gerrit.fd.io change
  4. [TBD] IPv6
  5. [TBD] Large-scale FIB tests
    • IPv4
    • IPv6
    • mac
    • Various prefix lengths
  6. [TBD] Encapsulations
    • VxLAN
    • VLAN
    • GRE
    • IPSEC
  7. [TBD] Create topology files for tb1, tb3
  8. [TBD] T-REX VXLAN traffic profile & CSIT tests
  9. IxNetwork driver
    • [TBD] driver design
    • [TBD] driver implementation
      • setup, teardown
      • traffic stream configuration
      • statistics retrieval
    • [TBD] latency measurements
      • feasibility
      • implementation
    • [TBD] integration with CSIT tests
  10. Moongen driver
    • [TBD] verify requirements and available HW against latest MoonGen codebase
    • [TBD] integration
  11. T-REX
    • [TBD] latency (T-REX support to be confirmed)
  12. Other work items


--- end