CSIT/csit2110 plan

From fd.io
Jump to: navigation, search


Introduction

This page tracks release information for FD.io CSIT-2110. It is updated regularly by hand. Real-time information is available in FD.io CSIT code repository and auto-generated docs.

Release Milestones

Milestone Date Deliverables
RC1 2021-09-22 Code complete. Pull first release branch. Only bug fixes, infra changes and framework optimizations are accepted in release branch. Date aligned with VPP RC1. Start dry-runs to identify CSIT gaps on less frequently run tests.
RC2 2021-10-13 (RC1+21) Dry-run testing begins of VPP RC2, performance and functional. Date aligned with VPP RC2.
CSIT Release 2021-10-27 (RC2+14) CSIT release complete. VPP release testing starts. Date aligned with VPP Formal Release. Only critical changes (as they usually require re-starting release testing) or changes not affecting results (such as documentation improvements) are accepted in release branch. Release testing also includes testing previous VPP version with new CSIT code, needed for comparison tables.
Report Publish 2021-11-10 (Rls+14) CSIT report published for VPP release. On master branch, start replacing (or dropping support for) APIs deprecated in the newly released VPP version (as we do not need to run the previous VPP version with newest CSIT code from now on).

Release Deliverables

Name Status Jira Category Description
Policy IPsec with outbound SPD rules Next rls New Test Suites Extend the current policy tests with policy tests with outbound SPD rules - use only the nth rule with just IPv4 forwarding. Blocked by incomplete VPP code.
Export test outputs in JSON In progress New functionality for existing tests XML processing is heavy, new processing using S3 queries need outputs in JSON files of a more stable layout.
Regular perf tests in AWS Done Framework Complete onboarding of AWS test environment, integrate into Jenkins CI, execute regularly (weekly) a subset of tests.
IPv4 /30 tests In progress New Test IPv4 scale tests with prefix length /30, in addition to current /32.
Add packet size support in ASTF tests Next rls New Test More packets sizes in addition to 64B used currently.
Automate TRex TG b2b tests Done New Test TRex back-to-back tests for continuous TRex only trending and calibration.
Failure summary report emails Done Reporting Normalized test failure summary in csit-report trending emails to simplify analysis.
Add Linux perfmon telemetry In progress Telemetry Adding per DUT (VPP, DPDK) thread Linux performance telemetry.
Migrate data into AWS/fd.io S3 storage Done Ci-management Migrate storage solution from Nexus into AWS S3 and internal storage. Consolidate scripting.
Scale tests optimisations Done Framework Migrate away from PAPI for large scale tests. Using exec cli in the interim.
CSIT & FD.io CI operations & maintenance Done Framework A number of CSIT & CI platform and operational improvements.

Jira Task Tracking

All CSIT release deliverables should be tracked in FDio CSIT Jira using one of the following Jira Epic categories:

Framework
CI process
Performance
Device
Methodology
Telemetry
Tools
Presentation
Aarch64

Multi-Release Work Areas

Work Area Description

External Dependencies

  1. No known external dependencies.