CSIT/PerfTestPlan

From fd.io
< CSIT
Revision as of 07:08, 7 July 2016 by Pmikus (Talk | contribs)

Jump to: navigation, search

CSIT Jenkins Jobs for Performance Tests

Jenkins Jobs
Task Job Triggers (patch comment VPP code CSIT code LF Testbed Status
Verify VPP code changes as they come - performance short tests vpp-csit-verify-hw-perf-master-short "verify-perf-short" VPP master branch + gerrit change under test csit-verified-tag TB1, TB2, TB3 WORKING
Verify VPP code changes as they come - performance long tests vpp-csit-verify-hw-perf-master-long "verify-perf-long" VPP master branch + gerrit change under test csit-verified-tag TB1, TB2, TB3 WORKING
Verify VPP code changes as they come - all performance tests vpp-csit-verify-hw-perf-master-all "verify-perf-all" VPP master branch + gerrit change under test csit-verified-tag TB1, TB2, TB3 WORKING
Verify CSIT code changes as they come - all performance tests csit-vpp-perf-master-all "verify-perf-all" vpp-master-verified vpp build csit master branch + gerrit change under test TB1, TB2, TB3 WORKING
Verify CSIT code changes as they come - performance long tests csit-vpp-perf-master-long "verify-perf-long" vpp-master-verified vpp build csit master branch + gerrit change under test TB1, TB2, TB3 WORKING
Verify CSIT code against VPP release 16.06 - all performance tests csit-vpp-perf-1606-all "verify-perf-all" vpp-master-verified vpp build csit stable/1606 branch TB1, TB2, TB3 WORKING
Verify CSIT code against VPP release 16.06 - performance long tests csit-vpp-perf-1606-long "verify-perf-long" vpp-master-verified vpp build csit stable/1606 branch TB1, TB2, TB3 WORKING

Performance Test Cases - Working

All 3 LF physical testbeds are all live - CSIT/CSIT LF testbed

CSIT test suite list for VPP v16.06

Test Suites

CSIT test suite list for VPP v16.09

[TO-DO]

Performance Test Cases - Plan and Status

For CSIT project to work and be useful, it is proposed that CSIT code and test cases development should be done in coordination with VPP Release Candidate content and release schedule. CSIT development work needs to be prioritized based on FD.io community needs and feedback, and csit-dev work distributed. To facilitate this, we compiled an initial list of proposed performance test cases that must, should, could be tested. Each csit-dev work item is prefixed with status and/or proposed priority relative to FD.io VPP release:

  • [Pn-Rm] – Proposed coverage of VPP functionality by FD.io CSIT functional and performance test cases.
    • Pn – CSIT development and integration priority; n=0,1,2, 0-for-MUST, 1-for-SHOULD, 2-for-COULD.
    • Rm – VPP major release, R1 stands for the first FD.io VPP release, R2 for the 2nd FD.io VPP release.
  • [DONE-Rm] - Done and working test cases, executed for VPP Rm code.
  • [TBD] - Development work scheduling priority to-be-defined.

Plan execution tracked on CSIT jira

CSIT-DEV plan for VPP v16.06

  1. Long throughput benchmarks - libraries
    1. [DONE-R1] IPv4 NonDropRate search
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] gerrit
      • [DONE-R1] BESTofN/WORSTofN search gerrit
      • [DONE-R1] combined binary+linear search gerrit
    2. [DONE-R1] L2 NonDropRate search [gerrit-ref]
    3. [DONE-R1] IPv6 NonDropRate search [gerrit-ref]
  2. Long throughput benchmarks - test cases
    1. [DONE-R1] IPv4 baseline NonDropRate search
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] [gerrit-ref]
      • [DONE-R1] BESTofN/WORSTofN search [gerrit-ref]
      • [DONE-R1] combined binary+linear search [gerrit-ref]
    2. [DONE-R1] L2 NonDropRate search
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] [gerrit-ref]
      • [DONE-R1] BESTofN/WORSTofN search [gerrit-ref]
      • [DONE-R1] combined binary+linear search [gerrit-ref]
    3. [DONE-R1] IPv6 NonDropRate search [gerrit-ref]
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] [gerrit-ref]
      • [DONE-R1] BESTofN/WORSTofN search [gerrit-ref]
      • [DONE-R1] combined binary+linear search [gerrit-ref]
  3. Short throughput benchmarks - libraries
    1. [DONE-R1] short IPv4 throughput test against reference NDR [gerrit-ref]
    2. [DONE-R1] short L2 throughput test against reference NDR [gerrit-ref]
    3. [DONE-R1] short IPv6 throughput test against reference NDR [gerrit-ref]
  4. Short throughput benchmarks - test cases
    1. [DONE-R1] short IPv4 throughput - routed-forwarding [gerrit-ref]
    2. [DONE-R1] short L2 throughput - bridge-domain, l2-xconnect [gerrit-ref]
    3. [DONE-R1] short IPv6 throughput - routed-forwarding [gerrit-ref]
  5. 3-node physical testbed - libraries
    • [DONE-R1] Create host topology files for tb2
    • [DONE-R1] Extend host topology with NIC type filtering
    • [DONE-R1] Create host topology files for tb1, tb3
  6. Multi-core multi-thread tests - libraries
    • [DONE-R1] multicore tests gerrit
  7. Large-scale FIB tests - libraries
    • [P1-R1] IPv4 same prefix length
    • [P2-R1] IPv4 varying prefix lengths
    • [P1-R1] IPv6 same prefix length
    • [P2-R1] IPv6 varying prefix lengths
    • [P1-R1] MAC addresses
  8. Packet encapsulations - test cases
    1. IP4 encapsulations
    2. [P1-R1] ip4-gre-ip4
    3. [P1-R2] ip4-ipsec-ip4
  9. IP6 encapsulations
    1. [P1-R1] ip4-o-ip6 softwire
      • [P1-R1] lightweight46 [rfc7596]
      • [P2-R1] map-e [rfc7597]
      • [P2-R1] map-t [rfc7599]
    2. [TBD] ip6-segment-routing
    3. [P1-R2] ip6-ipsec-ip6
  10. Security feature
    1. [DONE-R1] Classify IPv4 iACL
    2. [DONE-R1] Classify IPv6 iACL
    3. [DONE-R1] IPv4 COP
    4. [TBD] IPv6 COP
  11. L2 encapsulations
    1. [DONE-R1] l2-vlan-dot1q
    2. [DONE-R1] l2-vlan-dot1ad
    3. [P1-R1] l2-vxlan-ipv4
    4. [TBD] l2-eth-l2tpv3-ip4
    5. [TBD] l2-eth-l2tpv3-ip6
    6. [TBD] l2-eth-gre-ip4
    7. [TBD] l2-eth-gre-ip6
  12. VPP vNet topology baselines - libraries
    1. [TBD] Host stack baseline - Phy1-App-Phy2
      • [TBD] Reference host user-mode app, L3FWD-xconnect, DPDK driver per NIC type
    2. [TBD] Host stack VPP baseline - Phy1-VPP-Phy2
      • [TBD] Host VPP IPv4 baseline config, DPDK driver per NIC type
    3. [TBD] Guest stack baseline - Phy1-pcipt-AppVM-pcipt-Phy2
      • [TBD] Reference guest user-mode app, L3FWD-xconnect, DPDK driver per NIC type
    4. [TBD] Guest stack VPP baseline - Phy1-pcipt-VPPVM-pcipt-Phy2
      • [TBD] VPP IPv4 baseline config, DPDK driver per NIC type
    5. [TBD] Host and guest stack baseline - Phy1-VPP-AppVM-VPP-Phy2
      • [TBD] VPP IPv4 baseline config, vhost-user to AppVM
  13. vNet VPP topologies - libraries
    • [DONE-R1] Phy1-VPP-Phy2
    • [P0-R1] Phy1-VPP-VM-VPP-Phy2
  14. Traffic generator drivers
    1. [DONE-R1] T-Rex
      • [DONE-R1] T-Rex driver
      • [TBD] TRex support for packet delay and packet delay variation (T-REX support to be confirmed)
    2. [TBD] IXIA IxNetwork driver
      • [TBD] driver design
      • [TBD] driver implementation
        • [TBD] test case setup, teardown
        • [TBD] traffic stream configuration
        • [TBD] statistics retrieval
      • [TBD] latency measurements
        • [TBD] feasibility
        • [TBD] implementation
      • [TBD] integration with CSIT tests
    3. [TBD] Moongen driver
      • [TBD] verify requirements and available HW against latest MoonGen codebase
      • [TBD] CSIT integration
  15. Other work items

CSIT-DEV plan for VPP v16.09

Retrieved from "https://wiki.fd.io/index.php?title=CSIT/PerfTestPlan&oldid=2053"