CSIT/PerfTestPlan

From fd.io
< CSIT
Revision as of 08:10, 31 May 2016 by Pmikus (Talk | contribs)

Jump to: navigation, search

CSIT Jenkins Jobs for Performance Tests

  1. task: verify vpp code changes as they come - performance short tests
    • job: vpp-csit-verify-hw-perf-master-short
    • triggers: "verify-perf-short" patch comment
    • vpp code: vpp master branch PLUS gerrit change under test
    • csit code: csit-verified-tag
    • testbed: LF HW compute nodes - hw-tb1,tb2,tb3
    • status: WORKING
  2. task: verify vpp code changes as they come - performance long tests
    • job: vpp-csit-verify-hw-perf-master-long
    • triggers: "verify-perf-long" patch comment
    • vpp code: vpp master branch PLUS gerrit change under test
    • csit code: csit-verified-tag
    • testbed: LF HW compute nodes - hw-tb1,tb2,tb3
    • status: WORKING
  3. task: verify vpp code changes as they come - performance all tests
    • job: vvpp-csit-verify-hw-perf-master-all
    • triggers: "verify-perf-all" patch comment
    • vpp code: vpp master branch PLUS gerrit change under test
    • csit code: csit-verified-tag
    • testbed: LF HW compute nodes - hw-tb1,tb2,tb3
    • status: WORKING
  4. task: verify csit code changes as they come - performance tests
    • job: csit-vpp-perf-hw-all
    • triggers: gerrit change commit, "recheck" patch comment
    • vpp code: vpp-master-verified vpp build
    • csit code: csit master branch PLUS gerrit change
    • testbed: LF HW compute nodes - hw-tb1,tb2,tb3
    • status: WORKING

Performance Test Cases - Working

  1. All 3 LF physical testbeds are all live - hw-tb1,tb2,tb3.
  2. Performance test cases - 56 working in FD.io physical testbeds today.
  3. List of working performance test cases
    • ~/csit$ grep "^| [a-zA-Z1-9]" -nr tests | grep -vi " | " | sort | grep performance
    1. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:123:| Find NDR by using RFC2544 binary search and 9000B frames through bridge domain in 3-node topology
    2. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:143:| Find PDR by using RFC2544 binary search and 9000B frames through bridge domain in 3-node topology
    3. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:166:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through bridge domain in 3-node topology
    4. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:187:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through bridge domain in 3-node topology
    5. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:211:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through bridge domain in 3-node topology
    6. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:232:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through bridge domain in 3-node topology
    7. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:256:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through bridge domain in 3-node topology
    8. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:276:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through bridge domain in 3-node topology
    9. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:299:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through bridge domain in 3-node topology
    10. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:320:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through bridge domain in 3-node topology
    11. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:33:| Find NDR by using RFC2544 binary search and 64B frames through bridge domain in 3-node topology
    12. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:345:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through bridge domain in 3-node topology
    13. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:366:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through bridge domain in 3-node topology
    14. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:390:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 9000B frames through bridge domain in 3-node topology
    15. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:410:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 9000B frames through bridge domain in 3-node topology
    16. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:54:| Find PDR by using RFC2544 binary search and 64B frames through bridge domain in 3-node topology
    17. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:78:| Find NDR by using RFC2544 binary search and 1518B frames through bridge domain in 3-node topology
    18. tests/suites/performance/Long_Bridge_Domain_Intel-X520-DA2.robot:99:| Find PDR by using RFC2544 binary search and 1518B frames through bridge domain in 3-node topology
    19. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:103:| Find PDR by using RFC2544 binary search and 1518B frames through IPv4 forwarding in 3-node topology
    20. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:127:| Find NDR by using RFC2544 binary search and 9000B frames through IPv4 forwarding in 3-node topology
    21. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:147:| Find PDR by using RFC2544 binary search and 9000B frames through IPv4 forwarding in 3-node topology
    22. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:170:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through IPv4 forwarding in 3-node topology
    23. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:191:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through IPv4 forwarding in 3-node topology
    24. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:215:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through IPv4 forwarding in 3-node topology
    25. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:236:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through IPv4 forwarding in 3-node topology
    26. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:260:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through IPv4 forwarding in 3-node topology
    27. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:280:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through IPv4 forwarding in 3-node topology
    28. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:303:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through IPv4 forwarding in 3-node topology
    29. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:324:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through IPv4 forwarding in 3-node topology
    30. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:348:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through IPv4 forwarding in 3-node topology
    31. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:369:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through IPv4 forwarding in 3-node topology
    32. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:37:| Find NDR by using RFC2544 binary search and 64B frames through IPv4 forwarding in 3-node topology
    33. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:393:| Find NDR with 4 cores and rss 2 by using RFC2544 linear search and 9000B frames through IPv4 forwarding in 3-node topology
    34. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:413:| Find PDR with 4 cores and rss 2 by using RFC2544 linear search and 9000B frames through IPv4 forwarding in 3-node topology
    35. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:58:| Find PDR by using RFC2544 binary search and 64B frames through IPv4 forwarding in 3-node topology
    36. tests/suites/performance/Long_IPv4_Intel-X520-DA2.robot:82:| Find NDR by using RFC2544 binary search and 1518B frames through IPv4 forwarding in 3-node topology
    37. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:122:| Find NDR by using RFC2544 binary search and 9000B frames through IPv6 forwarding in 3-node topology
    38. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:142:| Find PDR by using RFC2544 binary search and 9000B frames through IPv6 forwarding in 3-node topology
    39. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:165:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 78B frames through IPv6 forwarding in 3-node topology
    40. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:186:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 78B frames through IPv6 forwarding in 3-node topology
    41. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:210:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through IPv6 forwarding in 3-node topology
    42. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:231:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through IPv6 forwarding in 3-node topology
    43. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:255:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through IPv6 forwarding in 3-node topology
    44. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:275:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through IPv6 forwarding in 3-node topology
    45. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:298:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 78B frames through IPv6 forwarding in 3-node topology
    46. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:319:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 78B frames through IPv6 forwarding in 3-node topology
    47. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:32:| Find NDR by using RFC2544 binary search and 78B frames through IPv6 forwarding in 3-node topology
    48. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:343:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through IPv6 forwarding in 3-node topology
    49. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:364:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through IPv6 forwarding in 3-node topology
    50. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:388:| Find NDR with 4 cores and rss 2 by using RFC2544 linear search and 9000B frames through IPv6 forwarding in 3-node topology
    51. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:409:| Find PDR with 4 cores and rss 2 by using RFC2544 linear search and 9000B frames through IPv6 forwarding in 3-node topology
    52. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:53:| Find PDR by using RFC2544 binary search and 78B frames through IPv6 forwarding in 3-node topology
    53. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:77:| Find NDR by using RFC2544 binary search and 1518B frames through IPv6 forwarding in 3-node topology
    54. tests/suites/performance/Long_IPv6_Intel-X520-DA2.robot:98:| Find PDR by using RFC2544 binary search and 1518B frames through IPv6 forwarding in 3-node topology
    55. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:101:| Find NDR by using RFC2544 binary search and 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    56. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:131:| Find PDR by using RFC2544 binary search and 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    57. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:164:| Find NDR by using RFC2544 binary search and 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    58. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:193:| Find PDR by using RFC2544 binary search and 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    59. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:225:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    60. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:255:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    61. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:288:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    62. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:319:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    63. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:352:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    64. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:382:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    65. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:38:| Find NDR by using RFC2544 binary search and 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    66. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:414:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    67. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:445:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    68. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:479:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    69. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:510:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    70. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:544:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    71. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:574:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    72. tests/suites/performance/Long_Xconnect_Dot1q_Intel-X520-DA2.robot:68:| Find PDR by using RFC2544 binary search and 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect in 3-node topology
    73. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:100:| Find PDR by using RFC2544 binary search and 1518B frames through L2 cross connect in 3-node topology
    74. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:124:| Find NDR by using RFC2544 binary search and 9000B frames through L2 cross connect in 3-node topology
    75. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:144:| Find PDR by using RFC2544 binary search and 9000B frames through L2 cross connect in 3-node topology
    76. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:167:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through L2 cross connect in 3-node topology
    77. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:188:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 64B frames through L2 cross connect in 3-node topology
    78. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:212:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through L2 cross connect in 3-node topology
    79. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:233:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 1518B frames through L2 cross connect in 3-node topology
    80. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:257:| Find NDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through L2 cross connect in 3-node topology
    81. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:277:| Find PDR with 2 cores and rss 1 by using RFC2544 binary search and 9000B frames through L2 cross connect in 3-node topology
    82. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:300:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through L2 cross connect in 3-node topology
    83. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:321:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 64B frames through L2 cross connect in 3-node topology
    84. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:345:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through L2 cross connect in 3-node topology
    85. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:34:| Find NDR by using RFC2544 binary search and 64B frames through L2 cross connect in 3-node topology
    86. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:366:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 1518B frames through L2 cross connect in 3-node topology
    87. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:390:| Find NDR with 4 cores and rss 2 by using RFC2544 binary search and 9000B frames through L2 cross connect in 3-node topology
    88. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:410:| Find PDR with 4 cores and rss 2 by using RFC2544 binary search and 9000B frames through L2 cross connect in 3-node topology
    89. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:55:| Find PDR by using RFC2544 binary search and 64B frames through L2 cross connect in 3-node topology
    90. tests/suites/performance/Long_Xconnect_Intel-X520-DA2.robot:79:| Find NDR by using RFC2544 binary search and 1518B frames through L2 cross connect in 3-node topology
    91. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:106:| 2core VPP with rss 1 passes 9000B frames through bridge domain at 2x 138,580pps in 3-node topology
    92. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:121:| 4core VPP with rss 2 passes 64B frames through bridge domain at 2x 7.4Mpps in 3-node topology
    93. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:137:| 4core VPP with rss 2 passes 1518B frames through bridge domain at 2x 812,743pps in 3-node topology
    94. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:153:| 4core VPP with rss 2 passes 9000B frames through bridge domain at 2x 138,580pps in 3-node topology
    95. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:27:| 1core VPP passes 64B frames through bridge domain at 2x 3.2Mpps in 3-node topology
    96. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:43:| 1core VPP passes 1518B frames through bridge domain at 2x 812,743pps in 3-node topology
    97. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:59:| 1core VPP passes 9000B frames through bridge domain at 2x 138,580pps in 3-node topology
    98. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:74:| 2core VPP with rss 1 passes 64B frames through bridge domain at 2x 6.9Mpps in 3-node topology
    99. tests/suites/performance/Short_Bridge_Domain_Intel-X520-DA2.robot:90:| 2core VPP with rss 1 passes 1518B frames through bridge domain at 2x 812,743pps in 3-node topology
    100. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:110:| 2core VPP with rss 1 passes 9000B frames through IPv4 forwarding at 2x 138,580pps in 3-node topology
    101. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:125:| 4core VPP with rss 2 passes 64B frames through IPv4 forwarding at 2x 7.8Mpps in 3-node topology
    102. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:141:| 4core VPP with rss 2 passes 1518B frames through IPv4 forwarding at 2x 812,743pps in 3-node topology
    103. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:157:| 4core VPP with rss 2 passes 9000B frames through IPv4 forwarding at 2x 138,580pps in 3-node topology
    104. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:31:| 1core VPP passes 64B frames through IPv4 forwarding at 2x 3.5Mpps in 3-node topology
    105. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:47:| 1core VPP passes 1518B frames through IPv4 forwarding at 2x 812,743pps in 3-node topology
    106. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:63:| 1core VPP passes 9000B frames through IPv4 forwarding at 2x 138,580pps in 3-node topology
    107. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:78:| 2core VPP with rss 1 passes 64B frames through IPv4 forwarding at 2x 7.5Mpps in 3-node topology
    108. tests/suites/performance/Short_IPv4_Intel-X520-DA2.robot:94:| 2core VPP with rss 1 passes 1518B frames through IPv4 forwarding at 2x 812,743pps in 3-node topology
    109. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:106:| 2core VPP with rss 1 passes 9000B frames through IPv6 forwarding at 2x 138,580pps in 3-node topology
    110. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:121:| 4core VPP with rss 2 passes 78B frames through IPv6 forwarding at 2x 7.3Mpps in 3-node topology
    111. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:137:| 4core VPP with rss 2 passes 1518B frames through IPv6 forwarding at 2x 812,743pps in 3-node topology
    112. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:153:| 4core VPP with rss 2 passes 9000B frames through IPv6 forwarding at 2x 138,580pps in 3-node topology
    113. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:27:| 1core VPP passes 78B frames through IPv6 forwarding at 2x 2.9Mpps in 3-node topology
    114. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:43:| 1core VPP passes 1518B frames through IPv6 forwarding at 2x 812,743pps in 3-node topology
    115. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:59:| 1core VPP passes 9000B frames through IPv6 forwarding at 2x 138,580pps in 3-node topology
    116. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:74:| 2core VPP with rss 1 passes 78B frames through IPv6 forwarding at 2x 5.9Mpps in 3-node topology
    117. tests/suites/performance/Short_IPv6_Intel-X520-DA2.robot:90:| 2core VPP with rss 1 passes 1518B frames through IPv6 forwarding at 2x 812,743pps in 3-node topology
    118. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:108:| 2core VPP with rss 1 passes 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x5.8Mpps in 3-node topology
    119. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:134:| 2core VPP with rss 1 passes 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x 720,000pps in 3-node topology
    120. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:160:| 2core VPP with rss 1 passes 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x 120,000pps in 3-node topology
    121. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:185:| 4core VPP with rss 2 passes 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x9.0Mpps in 3-node topology
    122. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:211:| 4core VPP with rss 2 passes 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x 720,000pps in 3-node topology
    123. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:237:| 4core VPP with rss 2 passes 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x 120,000pps in 3-node topology
    124. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:31:| 1core VPP passes 64B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x2.9Mpps in 3-node topology
    125. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:57:| 1core VPP passes 1518B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x 720,000pps in 3-node topology
    126. tests/suites/performance/Short_Xconnect_Dot1q_Intel-X520-DA2.robot:83:| 1core VPP passes 9000B frames through VLAN dot1q sub-interfaces inter-connected using L2 cross connect at 2x 120,000pps in 3-node topology
    127. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:107:| 2core VPP with rss 1 passes 9000B frames through L2 cross connect at 2x 138,580pps in 3-node topology
    128. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:122:| 4core VPP with rss 2 passes 64B frames through L2 cross connect at 2x 9.3Mpps in 3-node topology
    129. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:138:| 4core VPP with rss 2 passes 1518B frames through L2 cross connect at 2x 812,743pps in 3-node topology
    130. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:154:| 4core VPP with rss 2 passes 9000B frames through L2 cross connect at 2x 138,580pps in 3-node topology
    131. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:28:| 1core VPP passes 64B frames through L2 cross connect at 2x 3.6Mpps in 3-node topology
    132. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:44:| 1core VPP passes 1518B frames through L2 cross connect at 2x 812,743pps in 3-node topology
    133. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:60:| 1core VPP passes 9000B frames through L2 cross connect at 2x 138,580pps in 3-node topology
    134. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:75:| 2core VPP with rss 1 passes 64B frames through L2 cross connect at 2x 8.3Mpps in 3-node topology
    135. tests/suites/performance/Short_Xconnect_Intel-X520-DA2.robot:91:| 2core VPP with rss 1 passes 1518B frames through L2 cross connect at 2x 812,743pps in 3-node topology

Performance Test Cases - Plan and Status

For CSIT project to work and be useful, it is proposed that CSIT code and test cases development should be done in coordination with VPP Release Candidate content and release schedule. CSIT development work needs to be prioritized based on FD.io community needs and feedback, and csit-dev work distributed. To facilitate this, we compiled an initial list of proposed performance test cases that must, should, could be tested. Each csit-dev work item is prefixed with status and/or proposed priority relative to FD.io VPP release:

  • [Pn-Rm] – Proposed coverage of VPP functionality by FD.io CSIT functional and performance test cases.
    • Pn – CSIT development and integration priority; n=0,1,2, 0-for-MUST, 1-for-SHOULD, 2-for-COULD.
    • Rm – VPP major release, R1 stands for the first FD.io VPP release, R2 for the 2nd FD.io VPP release.
  • [DONE-Rm] - Done and working test cases, executed for VPP Rm code.
  • [TBD] - Development work scheduling priority to-be-defined.

Plan execution tracked on CSIT jira

Current list of identified csit-dev work items:

  1. Long throughput benchmarks - libraries
    1. [DONE-R1] IPv4 NonDropRate search
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] gerrit
      • [DONE-R1] BESTofN/WORSTofN search gerrit
      • [DONE-R1] combined binary+linear search gerrit
    2. [DONE-R1] L2 NonDropRate search [gerrit-ref]
    3. [DONE-R1] IPv6 NonDropRate search [gerrit-ref]
  2. Long throughput benchmarks - test cases
    1. [DONE-R1] IPv4 baseline NonDropRate search
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] [gerrit-ref]
      • [DONE-R1] BESTofN/WORSTofN search [gerrit-ref]
      • [DONE-R1] combined binary+linear search [gerrit-ref]
    2. [DONE-R1] L2 NonDropRate search
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] [gerrit-ref]
      • [DONE-R1] BESTofN/WORSTofN search [gerrit-ref]
      • [DONE-R1] combined binary+linear search [gerrit-ref]
    3. [DONE-R1] IPv6 NonDropRate search [gerrit-ref]
      • [DONE-R1] linear search [rfc2544] [gerrit-ref]
      • [DONE-R1] binary search [rfc2544] [gerrit-ref]
      • [DONE-R1] BESTofN/WORSTofN search [gerrit-ref]
      • [DONE-R1] combined binary+linear search [gerrit-ref]
  3. Short throughput benchmarks - libraries
    1. [DONE-R1] short IPv4 throughput test against reference NDR [gerrit-ref]
    2. [DONE-R1] short L2 throughput test against reference NDR [gerrit-ref]
    3. [DONE-R1] short IPv6 throughput test against reference NDR [gerrit-ref]
  4. Short throughput benchmarks - test cases
    1. [DONE-R1] short IPv4 throughput - routed-forwarding [gerrit-ref]
    2. [DONE-R1] short L2 throughput - bridge-domain, l2-xconnect [gerrit-ref]
    3. [DONE-R1] short IPv6 throughput - routed-forwarding [gerrit-ref]
  5. 3-node physical testbed - libraries
    • [DONE-R1] Create host topology files for tb2
    • [DONE-R1] Extend host topology with NIC type filtering
    • [DONE-R1] Create host topology files for tb1, tb3
  6. Multi-core multi-thread tests - libraries
    • [DONE-R1] multicore tests gerrit
  7. Large-scale FIB tests - libraries
    • [P1-R1] IPv4 same prefix length
    • [P2-R1] IPv4 varying prefix lengths
    • [P1-R1] IPv6 same prefix length
    • [P2-R1] IPv6 varying prefix lengths
    • [P1-R1] MAC addresses
  8. Packet encapsulations - test cases
    1. IP4 encapsulations
    2. [P1-R1] ip4-gre-ip4
    3. [P1-R2] ip4-ipsec-ip4
  9. IP6 encapsulations
    1. [P1-R1] ip4-o-ip6 softwire
      • [P1-R1] lightweight46 [rfc7596]
      • [P2-R1] map-e [rfc7597]
      • [P2-R1] map-t [rfc7599]
    2. [TBD] ip6-segment-routing
    3. [P1-R2] ip6-ipsec-ip6
  10. Security feature
    1. [DONE-R1] Classify IPv4 iACL
    2. [DONE-R1] Classify IPv6 iACL
    3. [DONE-R1] IPv4 COP
    4. [TBD] IPv6 COP
  11. L2 encapsulations
    1. [DONE-R1] l2-vlan-dot1q
    2. [DONE-R1] l2-vlan-dot1ad
    3. [P1-R1] l2-vxlan-ipv4
    4. [TBD] l2-eth-l2tpv3-ip4
    5. [TBD] l2-eth-l2tpv3-ip6
    6. [TBD] l2-eth-gre-ip4
    7. [TBD] l2-eth-gre-ip6
  12. VPP vNet topology baselines - libraries
    1. [TBD] Host stack baseline - Phy1-App-Phy2
      • [TBD] Reference host user-mode app, L3FWD-xconnect, DPDK driver per NIC type
    2. [TBD] Host stack VPP baseline - Phy1-VPP-Phy2
      • [TBD] Host VPP IPv4 baseline config, DPDK driver per NIC type
    3. [TBD] Guest stack baseline - Phy1-pcipt-AppVM-pcipt-Phy2
      • [TBD] Reference guest user-mode app, L3FWD-xconnect, DPDK driver per NIC type
    4. [TBD] Guest stack VPP baseline - Phy1-pcipt-VPPVM-pcipt-Phy2
      • [TBD] VPP IPv4 baseline config, DPDK driver per NIC type
    5. [TBD] Host and guest stack baseline - Phy1-VPP-AppVM-VPP-Phy2
      • [TBD] VPP IPv4 baseline config, vhost-user to AppVM
  13. vNet VPP topologies - libraries
    • [DONE-R1] Phy1-VPP-Phy2
    • [P0-R1] Phy1-VPP-VM-VPP-Phy2
  14. Traffic generator drivers
    1. [DONE-R1] T-Rex
      • [DONE-R1] T-Rex driver
      • [TBD] TRex support for packet delay and packet delay variation (T-REX support to be confirmed)
    2. [TBD] IXIA IxNetwork driver
      • [TBD] driver design
      • [TBD] driver implementation
        • [TBD] test case setup, teardown
        • [TBD] traffic stream configuration
        • [TBD] statistics retrieval
      • [TBD] latency measurements
        • [TBD] feasibility
        • [TBD] implementation
      • [TBD] integration with CSIT tests
    3. [TBD] Moongen driver
      • [TBD] verify requirements and available HW against latest MoonGen codebase
      • [TBD] CSIT integration
  15. Other work items
Retrieved from "https://wiki.fd.io/index.php?title=CSIT/PerfTestPlan&oldid=1534"