Difference between revisions of "CSIT/VnetInfraPlan"

From fd.io
Jump to: navigation, search
 
(4 intermediate revisions by one other user not shown)
Line 1: Line 1:
−
==CSIT Hardware testbeds==
+
__TOC__
  
−
# [DONE, ckoester] Initial installation
+
=CSIT development plan - background=
−
#* 3x 3-node-ucsc240m4 at LF
+
−
#* Operating System installation
+
−
#* Topology and connectivity tests completed
+
−
#** FD.io CSIT physical testbeds wiring [link to wiring file]
+
−
# [DONE, ckoester] All NICs installed and verified
+
−
#* 2p10GE 82599 Niantic, Intel
+
−
#* 2p10GE X710 Fortville, Intel
+
−
#* 2p40GE XL710 Fortville, Intel
+
−
#* 2p40GE VIC1385, Cisco
+
−
#* 2p10GE VIC1227, Cisco
+
−
# [TBD] NIC onboarding into CSIT
+
−
#* [DONE] 2p10GE 82599 Niantic, Intel
+
−
#* [TBD] 2p10GE X710 Fortville, Intel
+
−
#* [TBD] 2p40GE XL710 Fortville, Intel
+
−
#* [TBD] 2p40GE VIC1385, Cisco
+
−
#* [TBD] 2p10GE VIC1227, Cisco
+
−
#* [TBD] Need to resolve NIC fancy features getting in the way
+
−
#* [TBD] pending functional testcases to utilize additional NIC
+
−
# [TBD] Establish HW performance boundaries
+
−
#* [TBD] Need to validate functional testing for all NICs first
+
−
#* [TBD] UCS servers with NICs, CPU, PCI and Memory sub-systems
+
−
# [TBD] Scripted UCS reinstallation
+
−
#* switch between OS/distributions
+
−
#* re-install after failure
+
  
−
==CSIT VIRL testbeds==
+
CSIT test development priorities are aligned with VPP releases timeline, with relative dev work prioritization for each VPP release driven by perceived importance of test verification for VPP functionality and performance. Specific CSIT dev work priorities are driven by rough consensus within the project and contributors and committers availability and willingness to do work.
  
−
# [DONE, ckoester] Initial setup
+
Following priority tagging scheme is used in this CSIT wiki and FD.io jira for CSIT:
−
#* 3-node topology (similar to physical testbeds)
+
−
#* Automatic spawning of VIRL topology, VPP installation, creation of topology file for CSIT testing
+
−
# [TBD, ckoester] Expand hardware - PRIO 1 - ETA week of 04/11
+
−
#* Install two more ucs c240m4
+
−
#* Waiting for UCS servers to be delivered and installed, ETA 2 weeks
+
−
# [TBD] Implement additional topologies
+
−
#* [TBD] 2-node testbed
+
−
#* [TBD] Star topologies
+
−
#* [TBD] Larger rings
+
−
# [DONE, ckoester] Nested VM [gerrit.fd.io change]
+
−
#* [DONE] used for vhost-user interface testing
+
−
# [TBD] Host management and monitoring
+
−
#* Reservations, load-balancing, redundancy
+
−
#* Usage monitoring
+
  
−
==CSIT LF VM cloud testbeds==
+
* [$VppRlsNo-$RelPrio]
 +
** VppRlsNo - aligned VPP release number - 16.09, 16.12, 17.03.
 +
** RelPrio - CSIT release priority - P0 for Must Have, P1 for Should Have, P2 for Could Have.
 +
* CSIT dev work tagging examples:
 +
** [1609-P0] - CSIT Must Have for vpp-v1609.
 +
** [1609-P1] - CSIT Should Have for vpp-v1609.
 +
** [1609-P2] - CSIT Could Have for vpp-v1609.
  
−
# [TBD] Develop nested virtualisation
+
CSIT dev work plan execution tracked on [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=1&projectKey=CSIT CSIT jira], with priority tags encoded as Jira labels for ease of tracking and reporting.
−
#* [DONE, ckoester] KVM-in-KVM working on VIRL machines [https://gerrit.fd.io/r/#/c/665/ gerrit.fd.io change]
+
−
#* [TBD] Run 3-node topology inside LF hosted VM cloud
+
−
#** Once running, it will enable elastic scaling of funcational VPP tests in LF VM cloud
+
  
−
==CSIT Testsuite Portability==
+
Two different testbeds are used to execute CSIT tests:
 +
# CSIT Hardware testbeds - run performance tests, [https://wiki.fd.io/view/CSIT/CSIT_LF_testbed description of physical setup].
 +
# CSIT VIRL testbeds - run functional tests.
  
−
# [TBD] Distributable VIRL
+
=CSIT infra documentation=
−
#* [TBD] Small VIRL topology for portable/laptop use
+
−
#* [TBD] Public repository of topologies
+
−
#* [TBD] Allow VPP code developers to test their code in their environment before committing
+
  
−
==Multiple Operating System testing==
+
JIRA [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=82&quickFilter=86 board view], [https://jira.fd.io/issues/?filter=10662 list view] - '''infra''', '''docs''' labels for CSIT work tracking.
  
−
(tbd: Hardware and/or virtual)
+
# [1609-p0] Document VIRL image build and upgrade process, CSIT-333.
 +
# [1609-p0] Document configuration of VIRL server and HW testbeds from operational pools., CSIT-334.
  
−
# [TBD] Be able to switch between various OS [distributions]x[versions]
+
=CSIT infra operations=
 +
 
 +
JIRA [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=82&quickFilter=85 board view], [https://jira.fd.io/issues/?filter=10665 list view] - '''infra''', '''oper''' labels for CSIT work tracking.
 +
 
 +
# [1609-p0] Migration of perf tests from semiweekly to new daily job. CSIT-311.
 +
# [1609-p1] Long-term storage for error/diagnostic logs. CSIT-146.
 +
# [1609-p1] Test cases to log/store diagnostic data to external storage. CSIT-144.
 +
# [1609-p1] Move bootstrap_* files from CSIT to ci-management. CSIT-3.
 +
 
 +
=CSIT Hardware testbeds=
 +
 
 +
JIRA [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=82&quickFilter=83 board view], [https://jira.fd.io/issues/?filter=10663 list view] - '''infra''', '''env-phy''' labels for CSIT work tracking.
 +
 
 +
# [1609-p0] Improve Qemu affinity setting. CSIT-337.
 +
# [1609-p0] Hugepage configuration on physical LF testbeds. CSIT-217.
 +
# [1609-p0] HW testbed installation scripts to include PIP requirements. CSIT-157.
 +
# [1609-p1] Out-of-band access to SUTs. CSIT-145.
 +
# [1609-p1] Improve test case logging. CSIT-149.
 +
# [1609-p1] Nested VM to include l3fwd startup script. CSIT-210.
 +
# [1609-p1] Create separate test case for VPP start. CSIT-147.
 +
# [1609-p2] Analyze usage of Python VPP API in CSIT instead of VAT. CSIT-7.
 +
# [1609-p2] dut_setup.sh enables trace on dpdk-input. CSIT-122.
 +
# [1609-p2] Fix keyword for VPP threads setup. CSIT-339.
 +
# [1609-p2] Detect CPU on all nodes. CSIT-338.
 +
 
 +
=CSIT VIRL testbeds=
 +
 
 +
JIRA [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=82&quickFilter=84 board view], [https://jira.fd.io/issues/?filter=10664 list view] - '''infra''', '''env-virl''' labels for CSIT work tracking.
 +
 
 +
# [1609-p0] Include bridge-utils, docker and java in VM images. CSIT-192.
 +
# [1609-p1] Out-of-band access to SUTs. CSIT-145.
 +
# [1609-p1] Update nested VM qemu library to use 3rd serial console. CSIT-161.
 +
# [1609-p1] VIRL infrastructure periodic creation and distribution of images. CSIT-112.
 +
# [1609-p1] Health-check to capture TG/SUT environment after failed test case. CSIT-150.
 +
# [1609-p1] Execute start/stop-testcase scripts from git repository. CSIT-202.
 +
# [1609-p1] Do not destroy VM in case of test failure due to infrastructure issue. CSIT-151.
 +
# [1609-p1] Usage and status monitoring of VIRL hosts. CSIT-115.
 +
# [1609-p1] Improve test case logging. CSIT-149.
 +
# [1609-p1] Fix path creation KWs in to take *the whole path* in parameter list. CSIT-4.
 +
# [1609-p1] Create separate test case for VPP package installation. CSIT-148.
 +
# [1609-p1] Create separate test case for VPP start. CSIT-147.
 +
# [1609-p1] Add one general post-suite test to show vpp console output. CSIT-95.
 +
# [1609-p2] Nested VM: Replace cisco/cisco credentials with csit/csit. CSIT-159.
 +
# [1609-p2] Ubuntu VM: Replace cisco login with csit. CSIT-160.
 +
# [1609-p2] Modify VIRL and nested-VM username/password. CSIT-116.
 +
# [1609-p2] Nested-VM boot-up failed. CSIT-90.
 +
# [1609-p2] Use gcov for verifying VPP code test coverage by functional tests. CSIT-108.
 +
 
 +
=Other CSIT infra work=
 +
 
 +
# [1609-p1] Move bootstrap_* files from CSIT to ci-management., CSIT-3.
 +
# [1609-p1] Nested VM to include l3fwd startup script, CSIT-210.
 +
# [1609-p1] Test cases to log/store diagnostic data to external storage, CSIT-144.
 +
# [1609-p1] Long-term storage for error/diagnostic logs, CSIT-146.

Latest revision as of 18:37, 2 August 2016

CSIT development plan - background

CSIT test development priorities are aligned with VPP releases timeline, with relative dev work prioritization for each VPP release driven by perceived importance of test verification for VPP functionality and performance. Specific CSIT dev work priorities are driven by rough consensus within the project and contributors and committers availability and willingness to do work.

Following priority tagging scheme is used in this CSIT wiki and FD.io jira for CSIT:

  • [$VppRlsNo-$RelPrio]
    • VppRlsNo - aligned VPP release number - 16.09, 16.12, 17.03.
    • RelPrio - CSIT release priority - P0 for Must Have, P1 for Should Have, P2 for Could Have.
  • CSIT dev work tagging examples:
    • [1609-P0] - CSIT Must Have for vpp-v1609.
    • [1609-P1] - CSIT Should Have for vpp-v1609.
    • [1609-P2] - CSIT Could Have for vpp-v1609.

CSIT dev work plan execution tracked on CSIT jira, with priority tags encoded as Jira labels for ease of tracking and reporting.

Two different testbeds are used to execute CSIT tests:

  1. CSIT Hardware testbeds - run performance tests, description of physical setup.
  2. CSIT VIRL testbeds - run functional tests.

CSIT infra documentation

JIRA board view, list view - infra, docs labels for CSIT work tracking.

  1. [1609-p0] Document VIRL image build and upgrade process, CSIT-333.
  2. [1609-p0] Document configuration of VIRL server and HW testbeds from operational pools., CSIT-334.

CSIT infra operations

JIRA board view, list view - infra, oper labels for CSIT work tracking.

  1. [1609-p0] Migration of perf tests from semiweekly to new daily job. CSIT-311.
  2. [1609-p1] Long-term storage for error/diagnostic logs. CSIT-146.
  3. [1609-p1] Test cases to log/store diagnostic data to external storage. CSIT-144.
  4. [1609-p1] Move bootstrap_* files from CSIT to ci-management. CSIT-3.

CSIT Hardware testbeds

JIRA board view, list view - infra, env-phy labels for CSIT work tracking.

  1. [1609-p0] Improve Qemu affinity setting. CSIT-337.
  2. [1609-p0] Hugepage configuration on physical LF testbeds. CSIT-217.
  3. [1609-p0] HW testbed installation scripts to include PIP requirements. CSIT-157.
  4. [1609-p1] Out-of-band access to SUTs. CSIT-145.
  5. [1609-p1] Improve test case logging. CSIT-149.
  6. [1609-p1] Nested VM to include l3fwd startup script. CSIT-210.
  7. [1609-p1] Create separate test case for VPP start. CSIT-147.
  8. [1609-p2] Analyze usage of Python VPP API in CSIT instead of VAT. CSIT-7.
  9. [1609-p2] dut_setup.sh enables trace on dpdk-input. CSIT-122.
  10. [1609-p2] Fix keyword for VPP threads setup. CSIT-339.
  11. [1609-p2] Detect CPU on all nodes. CSIT-338.

CSIT VIRL testbeds

JIRA board view, list view - infra, env-virl labels for CSIT work tracking.

  1. [1609-p0] Include bridge-utils, docker and java in VM images. CSIT-192.
  2. [1609-p1] Out-of-band access to SUTs. CSIT-145.
  3. [1609-p1] Update nested VM qemu library to use 3rd serial console. CSIT-161.
  4. [1609-p1] VIRL infrastructure periodic creation and distribution of images. CSIT-112.
  5. [1609-p1] Health-check to capture TG/SUT environment after failed test case. CSIT-150.
  6. [1609-p1] Execute start/stop-testcase scripts from git repository. CSIT-202.
  7. [1609-p1] Do not destroy VM in case of test failure due to infrastructure issue. CSIT-151.
  8. [1609-p1] Usage and status monitoring of VIRL hosts. CSIT-115.
  9. [1609-p1] Improve test case logging. CSIT-149.
  10. [1609-p1] Fix path creation KWs in to take *the whole path* in parameter list. CSIT-4.
  11. [1609-p1] Create separate test case for VPP package installation. CSIT-148.
  12. [1609-p1] Create separate test case for VPP start. CSIT-147.
  13. [1609-p1] Add one general post-suite test to show vpp console output. CSIT-95.
  14. [1609-p2] Nested VM: Replace cisco/cisco credentials with csit/csit. CSIT-159.
  15. [1609-p2] Ubuntu VM: Replace cisco login with csit. CSIT-160.
  16. [1609-p2] Modify VIRL and nested-VM username/password. CSIT-116.
  17. [1609-p2] Nested-VM boot-up failed. CSIT-90.
  18. [1609-p2] Use gcov for verifying VPP code test coverage by functional tests. CSIT-108.

Other CSIT infra work

  1. [1609-p1] Move bootstrap_* files from CSIT to ci-management., CSIT-3.
  2. [1609-p1] Nested VM to include l3fwd startup script, CSIT-210.
  3. [1609-p1] Test cases to log/store diagnostic data to external storage, CSIT-144.
  4. [1609-p1] Long-term storage for error/diagnostic logs, CSIT-146.
Retrieved from "https://wiki.fd.io/index.php?title=CSIT/VnetInfraPlan&oldid=2380"