Difference between revisions of "CSIT/csit1901 plan"
From fd.io
< CSIT
Mackonstan (Talk | contribs) (Created page with "Category:Release Plan == Introduction == This page tracks release information for FD.io CSIT-19.01. It is updated regularly by hand. Real-time information is available i...") |
Mackonstan (Talk | contribs) |
||
| Line 3: | Line 3: | ||
== Introduction == | == Introduction == | ||
| − | This page tracks release information for FD.io CSIT | + | This page tracks release information for FD.io CSIT 1901. It is updated regularly by hand. Real-time information is available in FD.io CSIT code repository and auto-generated docs. |
== Release Milestones == | == Release Milestones == | ||
| Line 20: | Line 20: | ||
| RC1 | | RC1 | ||
| 2019-01-16 (F0+7) | | 2019-01-16 (F0+7) | ||
| − | | Code complete. Pull first release branch. Only bug fixes accepted in release branch. Date aligned with [https://wiki.fd.io/view/Projects/vpp/Release_Plans/ | + | | Code complete. Pull first release branch. Only bug fixes accepted in release branch. Date aligned with [https://wiki.fd.io/view/Projects/vpp/Release_Plans/Release_Plan_19.01#Release_Milestones VPP RC1]. |
|- | |- | ||
| RC2 | | RC2 | ||
| 2019-01-23 (RC1+7) | | 2019-01-23 (RC1+7) | ||
| − | | Dry-run testing begins of VPP RC2, performance and functional. Date aligned with [https://wiki.fd.io/view/Projects/vpp/Release_Plans/ | + | | Dry-run testing begins of VPP RC2, performance and functional. Date aligned with [https://wiki.fd.io/view/Projects/vpp/Release_Plans/Release_Plan_19.01#Release_Milestones VPP RC2]. |
|- | |- | ||
| CSIT Release | | CSIT Release | ||
| 2019-01-30 (RC2+7) | | 2019-01-30 (RC2+7) | ||
| − | | CSIT release complete. VPP release testing starts. Date aligned with [https://wiki.fd.io/view/Projects/vpp/Release_Plans/ | + | | CSIT release complete. VPP release testing starts. Date aligned with [https://wiki.fd.io/view/Projects/vpp/Release_Plans/Release_Plan_19.01#Release_Milestones VPP Formal Release]. |
|- | |- | ||
| Report Publish | | Report Publish | ||
Revision as of 14:16, 21 November 2018
Contents
[hide]Introduction
This page tracks release information for FD.io CSIT 1901. It is updated regularly by hand. Real-time information is available in FD.io CSIT code repository and auto-generated docs.
Release Milestones
| Milestone | Date | Deliverables |
|---|---|---|
| F0 | 2019-01-09 | Test case keywords code complete. Only low-risk changes accepted. |
| RC1 | 2019-01-16 (F0+7) | Code complete. Pull first release branch. Only bug fixes accepted in release branch. Date aligned with VPP RC1. |
| RC2 | 2019-01-23 (RC1+7) | Dry-run testing begins of VPP RC2, performance and functional. Date aligned with VPP RC2. |
| CSIT Release | 2019-01-30 (RC2+7) | CSIT release complete. VPP release testing starts. Date aligned with VPP Formal Release. |
| Report Publish | 2019-02-13 (Rls+14) | CSIT report published for VPP release. |
Release Deliverables
| Name | Status | Jira Category | Description |
|---|---|---|---|
| 3n-skx Tests | Open | 1901-Test | If new NICs present (e.g. 100GE), add tests for these. |
| 2n-skx Tests | Open | 1901-Test | Drive parity of test coverage with 3n-skx, focusing on non-tunnel encapsulations. |
| AVF Driver Tests | Open | 1901-Test | Drive parity of test coverage with DPDK for all baseline and scale tests. |
| Soak Tests | Open | 1901-Framework | Productize soak performamce tests for extended test duration and throughput discovery at given PLR. |
| VPP_Device Xeon | Open | 1901-Test | Execute on migrating VIRL tests to VPP_Device Container based functional VPP device tests running on 1n-skx. |
| VPP_Device Arm | Open | 1901-Framework | Container based functional VPP device tests framework for 1n-arm testbeds. |
| VPP_Device Arm | Open | 1901-test | Execute on migrating VIRL tests to VPP_Device Container based functional VPP device tests running on 1n-arm. |
| VPP_Device Atom | Open | 1901-Framework | Container based functional VPP device tests framework for 1n-atom testbeds. |
| VPP_Device Atom | Open | 1901-test | Execute on migrating VIRL tests to VPP_Device Container based functional VPP device tests running on 1n-atom. |
| VPP_Path | Open | 1901-Framework | Continuing migration of CSIT VIRL tests to VPP-make_test VPP integration tests for functional acceptance of VPP feature path(s) driven by use case(s). See P1 and P2 markup in CSIT_VIRL migration progress. |
| Per VPP Patch Performance Checks | Open | 1901-Framework | Per VPP patch vs. parent performance tests, anomaly detection, voting (-1/0/+1). Automated trigger subject to available physical testbed processing capacity. |
| CSIT PAPI Support | Open | 1901-Framework | Implementation of PAPI L1 KWs in CSIT. Required for migraing away from VAT. |
| Clock cycles per VPP node into CSIT-PAL | Open | 1901-PAL | Use the new VPP stats infra available via PAPI to retrieve runtime counters instead of using "show run". Blocked by no PAPI support in FD.io CSIT. |
| 2n-dnv Environment | Open | 1901-Framework | Onboard 2n-dnv (3n-dnv?) testbeds in LFN FD.io hosted lab. |
| 2n-dnv Tests | Open | 1901-Test | Port existing 2n-skx tests to 2n-dnv environment. |
| 2n-arm Environment | Open | 1901-Framework | Onboard 2n-arm (3n-arm?) testbeds in LFN FD.io hosted lab. |
| 2n-arm Tests | Open | 1901-Test | Port existing 2n-skx tests to 2n-arm environment. |
Jira Task Tracking
All CSIT release deliverables should be tracked in FDio CSIT Jira using one of the following Jira Epic categories:
- CSIT Framework
- Operations
- Test
- PAL
- VIRL
- HoneyComb
- [DMM]
Multi-Release Work Areas
| Work Area | Description |
|---|---|
| Xeon Skx testbeds | Make Skylake performance test coverage complete: i) Boost tests in 2-Node setups, complete 3-Node setups; ii) Complete Memif/Container and Vhost-user/VM with latest greatest QEMU etc; iii) Push vpp-dev to Ubuntu 18.04. |
| Arm testbeds | Introduce Arm performance tests. |
| Atom testbeds | Introduce Atom performance tests. |
| Better vhost, memif coverage | Make CSIT produce more complete test data for scaled-out Vhost-user/VM and Memif/Container: i) Complete same packet paths and topologies for a low number of VMs and Containers, then scale-up VM and Container numbers; ii) See if we can isolate the actual cost of Vhostuser-virtio and Memif-Memif virtual interfaces based on the test and system telemetry. |
| VPP per patch performance tests | Productise per VPP patch performance tests with change detection, prepare for voting: i) Improve detection accuracy and precision; ii) Nail down current results variance; iii) Apply improvements to continuous trending and (future) git auto-bisection. |
| Trending Improved Detection | Make trending job use new Burst MRR trending tests for better anomaly detection. Currently postponed, as the algorithm detects performance changes not related to VPP code. We need heavy workarounds or way more predictable SUT behavior. |
| More VPP telemetry reported and analysed | API based consumption of VPP telemetry including existing general counters, and future extended per node counters. |
| Evolve throughput search | Build upon MLRsearch experience vs. ordinary binary search: i) New POC for extended soak test for validating NDR (zero packet-loss-ratio PLR) and(?) PDR (non-zero PLR). |
| General enhancements | General CSIT and VPP performance test and infrastructure enhancements: i) Productize VPP_Device container-based functional tests in 1-Node Skylake testbeds, assist with the same for Arm; ii) Add proper packet latency measurements with T-Rex HDRhistogram, push T-Rex to productize HDRh'gram; iii) Start using the new VPP stats infra for per test counters and "gauges" collection incl. "show runtime", instead of VPP show CLI; iv) Start migration from VAT to VPP Python API; v) Nail down "broken"/not-performing VPP data plane feature arcs (incl. multi-threading) indicated by CSIT-18.07 results data. |
External Dependencies
- No known external dependencies.
