This page tracks release information for FD.io CSIT-2001. It is updated regularly by hand. Real-time information is available in FD.io CSIT code repository and auto-generated docs.
|F0||2020-01-xx||Test case keywords code complete. Only low-risk changes accepted.|
|RC1||2020-01-xx (F0+7)||Code complete. Pull first release branch. Only bug fixes accepted in release branch. Date aligned with VPP RC1. Start dry-runs to identify CSIT gaps on less frequently run tests.|
|RC2||2020-01-xx (RC1+7)||Dry-run testing begins of VPP RC2, performance and functional. Date aligned with VPP RC2.|
|CSIT Release||2020-01-xx (RC2+7)||CSIT release complete. VPP release testing starts. Date aligned with VPP Formal Release.|
|Report Publish||2020-01-xx (Rls+14)||CSIT report published for VPP release.|
Jira Task Tracking
All CSIT release deliverables should be tracked in FDio CSIT Jira using one of the following Jira Epic categories:
Multi-Release Work Areas
|Xeon Skx testbeds|| Make Skylake performance test coverage complete:
i) Boost tests in 2-Node setups, complete 3-Node setups;
ii) Complete Memif/Container and Vhost-user/VM with latest QEMU;
iii) Push vpp-dev to Ubuntu 18.04.
|Arm testbeds||Introduce Arm performance tests.|
|Atom testbeds||Introduce Denverton and Rangeley performance tests.|
|Better vhost, memif coverage|| Produce more complete test data for NFV service density:
i) Scaled-out Vhost-user/VM and Memif/Container tests; ii) Test the same packet paths and NF topologies: service chains, service pipelines;
iii) See if we can isolate the actual cost of Vhostuser-virtio and Memif-Memif virtual interfaces based on the test and system telemetry.
iv) Test with VM and Containers running on a single Processor (single socket) with no core oversubscription and with.
v) Extend the test over two Processors to quantify impact of UPI latency (and bandwidth).
|VPP per patch performance tests|| Productise per VPP patch performance tests with change detection, prepare for voting:
i) Improve detection accuracy and precision;
ii) Nail down current results variance;
iii) Apply improvements to continuous trending and (future) git auto-bisection.
|Trending Improved Detection|| Make trending job use new Burst MRR trending tests for better anomaly detection:
i) Currently postponed, as the algorithm detects performance changes not related to VPP code.
ii) We need heavy workarounds or way more predictable SUT behavior.
|More VPP telemetry reported and analysed||API based consumption of VPP telemetry including existing general counters, and future extended per node counters.|
|Evolve throughput search|| Build upon MLRsearch and PLRsearch experience vs. ordinary binary search:
i) Compare MLRsearch with PLRsearch soak test results.
|General enhancements|| General CSIT and VPP performance test and infrastructure enhancements:
i) Productize VPP_Device container-based functional tests in 1-Node Skylake testbeds, assist with the same for Arm;
ii) Add proper packet latency measurements with T-Rex HDRhistogram, push T-Rex to productize HDRh'gram;
iii) Start using the new VPP stats infra for per test counters and "gauges" collection incl. "show runtime", instead of VPP show CLI;
iv) Start migration from VAT to VPP Python API;
v) Nail down "broken"/not-performing VPP data plane feature arcs (incl. multi-threading) indicated by CSIT-18.10 results data.
- No known external dependencies.