Difference between revisions of "CSIT/Jobs"

From fd.io
Jump to: navigation, search
 
(38 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
__TOC__
 
__TOC__
  
−
=Refactoring in Progress=
+
==VPP Jenkins Verify Jobs: Verify a VPP patch using an operational CSIT branch==
−
This page is being refactored into tabular form from the existing list form and the content updated for clarity.
+
The following jenkins jobs verify VPP patches by running CSIT tests based on gerrit events for the VPP git repository or manual scheduling based on gerrit comment trigger keywords posted to a VPP patch in gerrit. The version of the CSIT tests which are executed is determined by a script that is checked into each VPP branch.  See [[CSIT/Branching_Strategy]] for details.
−
-dwallacelf-
+
−
 
+
−
==VPP Jenkins Jobs which run CSIT tests to verify a VPP patch==
+
−
The following jenkins jobs run CSIT tests based on gerrit events for a branch in the VPP git repository or manual scheduling based on the specified gerrit comment trigger keyword posted to a specific VPP patch in gerrit. The version of the CSIT tests which are executed is determined by a script that is checked into each VPP branch.  See [[CSIT/Branching_Strategy]] for details.
+
  
 
{| class="wikitable"
 
{| class="wikitable"
 
!Jenkins Job!!Description!!Gerrit Comment Trigger!!Event Triggers!!VPP Code!!Testbed!!Status
 
!Jenkins Job!!Description!!Gerrit Comment Trigger!!Event Triggers!!VPP Code!!Testbed!!Status
 +
|-
 +
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-1707/ vpp-csit-verify-hw-perf-1707]<br />[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-1710/ vpp-csit-verify-hw-perf-1710]<br />[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-1801/ vpp-csit-verify-hw-perf-1801]<br />[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-1804/ vpp-csit-verify-hw-perf-1804]<br />[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-master/ vpp-csit-verify-hw-perf-master]
 +
|VPP performance verify - Verify VPP patches by running selected MRR tests running on Intel x710 NIC
 +
|perftest [tag1ANDtag2 OR tag3 !tag4]
 +
|None
 +
|vpp branch 'stable/1707' + gerrit patch<br />vpp branch 'stable/1710' + gerrit patch<br />vpp branch 'stable/1801' + gerrit patch<br />vpp branch 'stable/master' + gerrit patch<br />vpp branch 'stable/1804' + gerrit patch
 +
|LF HW compute nodes
 +
|WORKING
 
|-
 
|-
 
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-virl-master/ vpp-csit-verify-virl-master]
 
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-virl-master/ vpp-csit-verify-virl-master]
−
|verify vpp patches by running csit functional tests
+
|Verify VPP patches by running csit functional tests
 
|recheck
 
|recheck
 
|gerrit patch commit, gerrit rebase, gerrit edit
 
|gerrit patch commit, gerrit rebase, gerrit edit
−
|vpp branch PLUS gerrit patch
+
|vpp branch 'master' + gerrit patch
 
|LF VIRL compute nodes
 
|LF VIRL compute nodes
 
|WORKING
 
|WORKING
 
|-
 
|-
−
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-test-short/ vpp-csit-verify-hw-perf-test-short]
+
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-virl-1801/ vpp-csit-verify-virl-1801]
−
|verify vpp patches by running csit short performance tests
+
|Verify VPP patches by running csit functional tests
−
|verify-perf-short
+
|recheck
−
|none
+
|gerrit patch commit, gerrit rebase, gerrit edit
−
|vpp branch PLUS gerrit patch
+
|vpp branch 'stable/1801' + gerrit patch
 +
|LF VIRL compute nodes
 +
|WORKING
 +
|-
 +
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-virl-1804/ vpp-csit-verify-virl-1804]
 +
|Verify VPP patches by running csit functional tests
 +
|recheck
 +
|gerrit patch commit, gerrit rebase, gerrit edit
 +
|vpp branch 'stable/1804' + gerrit patch
 +
|LF VIRL compute nodes
 +
|WORKING
 +
|}
 +
 
 +
==CSIT Jenkins Verify Jobs: Verify a CSIT patch using a validated VPP image==
 +
The following jenkins jobs verify CSIT patches by running CSIT tests based on gerrit events for the CSIT git repository or manual scheduling based on gerrit comment trigger keywords posted to a CSIT patch in gerrit. The VPP images used to verify the CSIT patches is based on the latest VPP image that has been validated by the CSIT semi-weekly verify job.
 +
 
 +
{| class="wikitable"
 +
!Jenkins Job!!Description!!Gerrit Comment Trigger!!Event Triggers!!CSIT Code!!Testbed!!Status
 +
|-
 +
|[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-verify-1707/ csit-vpp-perf-verify-1707]<br />[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-verify-1710/ csit-vpp-perf-verify-1710]<br />[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-verify-1801/ csit-vpp-perf-verify-1801]<br />[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-verify-master/ csit-vpp-perf-verify-master]<br />[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-verify-1804/ csit-vpp-perf-verify-1804]
 +
|perf verify - per csit gerrit patch
 +
default tags:
 +
- perftest
 +
|csit-perftest [tag1ANDtag2 OR tag3 !tag4]
 +
|None
 +
|csit branch 'rls1707' + csit patch<br />csit branch 'rls1710' + csit patch<br />csit branch 'rls1801' + csit patch<br />csit branch 'master' + csit patch<br />csit branch 'rls1804' + csit patch
 
|LF HW compute nodes
 
|LF HW compute nodes
 
|WORKING
 
|WORKING
 
|-
 
|-
−
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-test-long/ vpp-csit-verify-hw-perf-test-long]
+
|[https://jenkins.fd.io/view/csit/job/csit-vpp-functional-master-virl/ csit-vpp-functional-master-virl]
−
|verify vpp patches by running csit long performance tests
+
|Verify CSIT functional tests against a validated VPP 'master' image
−
|verify-perf-long
+
|recheck
 +
|gerrit patch commit, gerrit rebase, gerrit edit
 +
|csit branch 'master' + csit patch
 +
|LF VIRL compute nodes
 +
|WORKING
 +
|-
 +
|[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-master-all/ csit-vpp-perf-master-all]
 +
|Verify CSIT code changes against VPP 'master' - all performance tests
 +
|verify-perf-all
 
|none
 
|none
−
|vpp branch PLUS gerrit patch
+
|csit branch 'master' + csit patch
 
|LF HW compute nodes
 
|LF HW compute nodes
 
|WORKING
 
|WORKING
 
|-
 
|-
−
|[https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-test-all/ vpp-csit-verify-hw-perf-test-all]
+
|[https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1804-virl/ csit-vpp-functional-1804-virl]
−
|verify vpp patches by running all csit tests (func., short perf., long perf.)
+
|Verify CSIT functional tests against a validated VPP 'stable/1804' image
 +
|recheck
 +
|gerrit patch commit, gerrit rebase, gerrit edit
 +
|csit branch 'rls1804' + csit patch
 +
|LF VIRL compute nodes
 +
|WORKING
 +
|-
 +
|[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1804-all/ csit-vpp-perf-1804-all]
 +
|Verify CSIT code against VPP 18.04 release 'stable/1804' - all performance tests
 
|verify-perf-all
 
|verify-perf-all
 
|none
 
|none
−
|vpp branch PLUS gerrit patch
+
|csit branch 'rls1804' + csit patch
 
|LF HW compute nodes
 
|LF HW compute nodes
 
|WORKING
 
|WORKING
 
|}
 
|}
  
−
==CSIT Jobs Listing==
+
==CSIT Jenkins Periodic Jobs: Periodically verify VPP branch and CSIT branch==
−
# task: verify vpp code changes as they come - functional tests
+
Because these jobs take many hours to run, they are only triggered periodically to prevent blocking per patch verification by tying up hardware resources. There are two types of Jenkins periodic jobs:  
−
#* job: [https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-virl-master/ vpp-csit-verify-virl-master]
+
 
−
#* triggers: gerrit change commit, "recheck" patch comment
+
1. Jobs that verify VPP images using all CSIT tests.
−
#* vpp code: vpp master branch PLUS gerrit change under test
+
 
−
#* csit code: csit-verified-tag
+
2. Jobs that verify CSIT branch with verified stable VPP build.
−
#* testbed: LF VIRL compute nodes
+
 
−
#* status: WORKING
+
{| class="wikitable"
−
# task: verify vpp code changes as they come - performance short tests
+
!Jenkins Job!!Description!!Execution Period!!CSIT Code!!VPP Code!!Status
−
#* job: [https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-test-short/ vpp-csit-verify-hw-perf-test-short]
+
|-
−
#* triggers: "verify-perf-short" patch comment
+
|[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-trend-daily-master/ csit-vpp-perf-trend-daily-master]
−
#* vpp code: vpp master branch PLUS gerrit change under test
+
|Daily perf trending of vpp master build
−
#* csit code: csit-verified-tag
+
default tags:
−
#* testbed: LF HW compute nodes
+
- 1T1C: one worker thread, one core
−
#* status: WORKING
+
- 2T2C: two worker threads, two cores
−
# task: verify vpp code changes as they come - performance long tests
+
- X520: x520 Niantic NIC
−
#* job: [https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-test-long/ vpp-csit-verify-hw-perf-test-long]
+
- NDRDISC: discovery of NDR throughput
−
#* triggers: "verify-perf-long" patch comment
+
|Once a day between 20:00 and 23:00
−
#* vpp code: vpp master branch PLUS gerrit change under test
+
|csit branch 'oper-<$DATE>'
−
#* csit code: csit-verified-tag
+
|vpp branch 'master' from nexus.fd.io
−
#* testbed: LF HW compute nodes
+
|PAUSED
−
#* status: WORKING
+
|-
−
# task: verify vpp code changes as they come - performance all tests
+
|[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master/ csit-vpp-perf-mrr-daily-master]
−
#* job: [https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-hw-perf-test-all/ vpp-csit-verify-hw-perf-test-all]
+
|Check vpp perf mrr daily
−
#* triggers: "verify-perf-all" patch comment
+
default tags:
−
#* vpp code: vpp master branch PLUS gerrit change under test
+
- 1T1C: one worker thread, one core
−
#* csit code: csit-verified-tag
+
- 2T2C: two worker thread, two core
−
#* testbed: LF HW compute nodes
+
- 4T4C: four worker thread, four core
−
#* status: WORKING
+
- MRR: Check of MRR throughput
−
# task: identify vpp-master-verified vpp build for csit-* jobs and provide trending reports – semiweekly
+
|Twice a day at 02:00 and 16:00
−
#* job: [https://jenkins.fd.io/view/csit/job/csit-vpp-master-verify-semiweekly/ csit-vpp-master-verify-semiweekly]  
+
|csit branch 'oper-<$DATE>'
−
#* triggers: periodic semiweekly
+
|vpp branch 'master' from nexus.fd.io
−
#* vpp code: latest build from nexus repository (deposited by vpp-merge-master job)
+
|WORKING
−
#** https://nexus.fd.io/content/repositories/fd.io.dev/io/fd/vpp/vpp/
+
|-
−
#** on success: update vpp-master-verified vpp build reference in csit bootstrap scripts
+
|[https://jenkins.fd.io/view/csit/job/csit-vpp-perf-trend-semi-weekly-master/ csit-vpp-perf-trend-semi-weekly-master]
−
#* csit code: csit-verified-tag
+
|semi-weekly perf trending of vpp master build
−
#* testbed: LF VIRL for functional and HW compute nodes for performance
+
default tags for vpp startup config and perf test type
−
#* status: OPEN / PARTIAL
+
- 1T1C: one worker thread, one core
−
#** [https://jenkins.fd.io/view/csit/job/csit-vpp-master-verify-weekly/ csit-vpp-master-verify-weekly]
+
- 2T2C: two worker threads, two cores
−
# task: verify csit code changes as they come - functional tests
+
- X710: x710 Fortville NIC
−
#* job: [https://jenkins.fd.io/view/csit/job/csit-vpp-functional-virl/ csit-vpp-functional-virl]
+
- XL710: xl710 Fortville NIC
−
#* triggers: gerrit change commit, "recheck" patch comment
+
- NDRDISC: discovery of NDR throughput
−
#* vpp code: vpp-master-verified vpp build
+
|Tuesday and Saturday, between 20:00 and 23:00
−
#* csit code: csit master branch PLUS gerrit change
+
|csit branch 'oper-<$DATE>'
−
#* testbed: LF VIRL compute nodes
+
|vpp branch 'master' from nexus.fd.io
−
#* status: WORKING
+
|PAUSED
−
# task: verify csit code changes as they come - performance tests
+
|-
−
#* job: [https://jenkins.fd.io/view/csit/job/csit-vpp-perf-hw-all/ csit-vpp-perf-hw-all]
+
|[https://jenkins.fd.io/view/csit/job/csit-vpp-verify-master-semiweekly/ csit-vpp-verify-master-semiweekly]
−
#* triggers: gerrit change commit, "recheck" patch comment
+
|Verify latest VPP image of 'master' from nexus.fd.io. If all tests pass, the "VPP_STABLE_VER" of 'master' image used to verify CSIT patches can be updated (by CSIT patch).
−
#* vpp code: vpp-master-verified vpp build
+
|Twice a week (start at 4AM UTC every Tuesday and Saturday)
−
#* csit code: csit master branch PLUS gerrit change
+
|csit branch 'oper-<$DATE>'
−
#* testbed: LF HW compute nodes
+
|vpp branch 'master' from nexus.fd.io
−
#* status: WORKING
+
|WORKING
−
# task: verify csit code against vpp-master-verified vpp build – weekly
+
 
−
#* job: [https://jenkins.fd.io/view/csit/job/csit-vpp-verify-weekly/ csit-vpp-verify-weekly]
+
|-
−
#* triggers: periodic weekly
+
|[https://jenkins.fd.io/view/csit/job/csit-vpp-verify-1804-semiweekly/ csit-vpp-verify-1804-semiweekly]
−
#* vpp code: vpp-master-verified vpp build
+
|Verify VPP image of 'stable/1801' from nexus.fd.io. If all tests pass, the "VPP_STABLE_VER" of 'stable/1609' image used to verify CSIT patches can be updated (by CSIT patch).
−
#* csit code: csit master branch
+
 
−
#** on success: update csit-verified-tag
+
|Twice a week (start at 4AM UTC every Tuesday and Saturday)
−
#* testbed:  LF VIRL and LF HW compute nodes
+
|csit branch 'oper-rls1804-<$DATE>'
−
#* status: OPEN - NOT WORKING
+
|vpp branch 'stable/1804' from nexus.fd.io
 +
|WORKING
 +
|-
 +
|[https://jenkins.fd.io/view/csit/job/csit-vpp-verify-master-weekly/ csit-vpp-verify-master-weekly]
 +
|Verify CSIT 'master' HEAD image using "VPP_STABLE_VER" of 'master' branch. If all tests pass, create csit branch 'oper-<$DATE>' that can be used for VPP verify Jenkins jobs.
 +
|Once a week (start at 4AM UTC every Sunday)
 +
|csit branch 'master' HEAD
 +
|stable vpp branch 'master' image from nexus.fd.io
 +
|WORKING
 +
|-
 +
|[https://jenkins.fd.io/view/csit/job/csit-vpp-verify-1801-weekly/ csit-vpp-verify-1804-weekly]
 +
|Verify CSIT 'master' HEAD image using "VPP_STABLE_VER" of 'stable/1804' branch. If all tests pass, create csit branch 'oper-rls1801-<$DATE>' that can be used for VPP verify Jenkins jobs.
 +
|As required
 +
|csit branch 'rls1804' HEAD
 +
|stable vpp branch 'stable/1804' image from nexus.fd.io
 +
|WORKING
 +
|}

Latest revision as of 06:54, 4 July 2018

VPP Jenkins Verify Jobs: Verify a VPP patch using an operational CSIT branch

The following jenkins jobs verify VPP patches by running CSIT tests based on gerrit events for the VPP git repository or manual scheduling based on gerrit comment trigger keywords posted to a VPP patch in gerrit. The version of the CSIT tests which are executed is determined by a script that is checked into each VPP branch. See CSIT/Branching_Strategy for details.

Jenkins Job Description Gerrit Comment Trigger Event Triggers VPP Code Testbed Status
vpp-csit-verify-hw-perf-1707
vpp-csit-verify-hw-perf-1710
vpp-csit-verify-hw-perf-1801
vpp-csit-verify-hw-perf-1804
vpp-csit-verify-hw-perf-master 		VPP performance verify - Verify VPP patches by running selected MRR tests running on Intel x710 NIC perftest [tag1ANDtag2 OR tag3 !tag4] None vpp branch 'stable/1707' + gerrit patch
vpp branch 'stable/1710' + gerrit patch
vpp branch 'stable/1801' + gerrit patch
vpp branch 'stable/master' + gerrit patch
vpp branch 'stable/1804' + gerrit patch 		LF HW compute nodes WORKING
vpp-csit-verify-virl-master Verify VPP patches by running csit functional tests recheck gerrit patch commit, gerrit rebase, gerrit edit vpp branch 'master' + gerrit patch LF VIRL compute nodes WORKING
vpp-csit-verify-virl-1801 Verify VPP patches by running csit functional tests recheck gerrit patch commit, gerrit rebase, gerrit edit vpp branch 'stable/1801' + gerrit patch LF VIRL compute nodes WORKING
vpp-csit-verify-virl-1804 Verify VPP patches by running csit functional tests recheck gerrit patch commit, gerrit rebase, gerrit edit vpp branch 'stable/1804' + gerrit patch LF VIRL compute nodes WORKING

CSIT Jenkins Verify Jobs: Verify a CSIT patch using a validated VPP image

The following jenkins jobs verify CSIT patches by running CSIT tests based on gerrit events for the CSIT git repository or manual scheduling based on gerrit comment trigger keywords posted to a CSIT patch in gerrit. The VPP images used to verify the CSIT patches is based on the latest VPP image that has been validated by the CSIT semi-weekly verify job.

Jenkins Job Description Gerrit Comment Trigger Event Triggers CSIT Code Testbed Status
csit-vpp-perf-verify-1707
csit-vpp-perf-verify-1710
csit-vpp-perf-verify-1801
csit-vpp-perf-verify-master
csit-vpp-perf-verify-1804 		perf verify - per csit gerrit patch

default tags: 

- perftest
csit-perftest [tag1ANDtag2 OR tag3 !tag4] None csit branch 'rls1707' + csit patch
csit branch 'rls1710' + csit patch
csit branch 'rls1801' + csit patch
csit branch 'master' + csit patch
csit branch 'rls1804' + csit patch 		LF HW compute nodes WORKING
csit-vpp-functional-master-virl Verify CSIT functional tests against a validated VPP 'master' image recheck gerrit patch commit, gerrit rebase, gerrit edit csit branch 'master' + csit patch LF VIRL compute nodes WORKING
csit-vpp-perf-master-all Verify CSIT code changes against VPP 'master' - all performance tests verify-perf-all none csit branch 'master' + csit patch LF HW compute nodes WORKING
csit-vpp-functional-1804-virl Verify CSIT functional tests against a validated VPP 'stable/1804' image recheck gerrit patch commit, gerrit rebase, gerrit edit csit branch 'rls1804' + csit patch LF VIRL compute nodes WORKING
csit-vpp-perf-1804-all Verify CSIT code against VPP 18.04 release 'stable/1804' - all performance tests verify-perf-all none csit branch 'rls1804' + csit patch LF HW compute nodes WORKING

CSIT Jenkins Periodic Jobs: Periodically verify VPP branch and CSIT branch

Because these jobs take many hours to run, they are only triggered periodically to prevent blocking per patch verification by tying up hardware resources. There are two types of Jenkins periodic jobs:

1. Jobs that verify VPP images using all CSIT tests.

2. Jobs that verify CSIT branch with verified stable VPP build.

Jenkins Job Description Execution Period CSIT Code VPP Code Status
csit-vpp-perf-trend-daily-master Daily perf trending of vpp master build

default tags: 

- 1T1C: one worker thread, one core
- 2T2C: two worker threads, two cores
- X520: x520 Niantic NIC
- NDRDISC: discovery of NDR throughput
Once a day between 20:00 and 23:00 csit branch 'oper-<$DATE>' vpp branch 'master' from nexus.fd.io PAUSED
csit-vpp-perf-mrr-daily-master Check vpp perf mrr daily

default tags: 

- 1T1C: one worker thread, one core
- 2T2C: two worker thread, two core
- 4T4C: four worker thread, four core
- MRR: Check of MRR throughput
Twice a day at 02:00 and 16:00 csit branch 'oper-<$DATE>' vpp branch 'master' from nexus.fd.io WORKING
csit-vpp-perf-trend-semi-weekly-master semi-weekly perf trending of vpp master build

default tags for vpp startup config and perf test type 

- 1T1C: one worker thread, one core
- 2T2C: two worker threads, two cores
- X710: x710 Fortville NIC
- XL710: xl710 Fortville NIC
- NDRDISC: discovery of NDR throughput
Tuesday and Saturday, between 20:00 and 23:00 csit branch 'oper-<$DATE>' vpp branch 'master' from nexus.fd.io PAUSED
csit-vpp-verify-master-semiweekly Verify latest VPP image of 'master' from nexus.fd.io. If all tests pass, the "VPP_STABLE_VER" of 'master' image used to verify CSIT patches can be updated (by CSIT patch). Twice a week (start at 4AM UTC every Tuesday and Saturday) csit branch 'oper-<$DATE>' vpp branch 'master' from nexus.fd.io WORKING
csit-vpp-verify-1804-semiweekly Verify VPP image of 'stable/1801' from nexus.fd.io. If all tests pass, the "VPP_STABLE_VER" of 'stable/1609' image used to verify CSIT patches can be updated (by CSIT patch). Twice a week (start at 4AM UTC every Tuesday and Saturday) csit branch 'oper-rls1804-<$DATE>' vpp branch 'stable/1804' from nexus.fd.io WORKING
csit-vpp-verify-master-weekly Verify CSIT 'master' HEAD image using "VPP_STABLE_VER" of 'master' branch. If all tests pass, create csit branch 'oper-<$DATE>' that can be used for VPP verify Jenkins jobs. Once a week (start at 4AM UTC every Sunday) csit branch 'master' HEAD stable vpp branch 'master' image from nexus.fd.io WORKING
csit-vpp-verify-1804-weekly Verify CSIT 'master' HEAD image using "VPP_STABLE_VER" of 'stable/1804' branch. If all tests pass, create csit branch 'oper-rls1801-<$DATE>' that can be used for VPP verify Jenkins jobs. As required csit branch 'rls1804' HEAD stable vpp branch 'stable/1804' image from nexus.fd.io WORKING
Retrieved from "https://wiki.fd.io/index.php?title=CSIT/Jobs&oldid=8027"