Difference between revisions of "Project Proposals/CSIT"

From fd.io
Jump to: navigation, search
 
Line 3: Line 3:
  
 
== Name ==
 
== Name ==
<!-- The name of your project, for example Honeycomb BridgeDomain or NSH Advanced Features etc. -->
+
Performance Measurement and Analysis Tools
 
+
CSIT (Continuous System Integration and Testing) for FD.io VPP code projects.
+
  
 
== Project Contact Name and Email ==
 
== Project Contact Name and Email ==
<!-- Name and email of the project contact -->
+
Patrick Lu <patrick.lu@intel.com>
 
+
[mailto:mkonstan@cisco.com Maciek Konstantynowicz], LF-ID: mackonstan.
+
  
 
== Repository Name ==
 
== Repository Name ==
<!-- Project Repository Name, should be:
+
pma_tools
    i. Lower case
+
    ii. Short
+
    iii. Suitable for use as a C identifier
+
-->
+
 
+
csit
+
  
 
== Description ==
 
== Description ==
<!-- Description of the work that will take place in this project -->
+
pma_tools project aims to develop software tools for performance and efficiency measurements of compute optimized network data plane applications like FD.io VPP (and similar) while running on compute platform(s) . pma_tools will provide tools that enable systematic measurement methodologies of such applications and are to be directly integrated within the FD.io operating environment(s) and FD.io projects like VPP, CSIT and TRex using developed APIs. This could facilitate further standardization of performance measurement and analysis methodologies for compute platforms running network workloads.
 
+
CSIT (Continuous System Integration and Testing) project is to focus on:
+
# Development of software code for fully automated VPP code testing, functionality, performance, regression and new functions.
+
# Execution of CSIT test suites on VPP code running on LF FD.io virtual and physical compute environments.
+
# Integration with FD.io continuous integration systems (Gerrit, Jenkins and such).
+
# Identified existing FD.io project dependencies and interactions:
+
#* vpp - Vector Packet Processing.
+
#* honeycomb - Honeycomb Agent for management plane testing.
+
#* ci-management - Management repo for Jenkins Job Builder, script and management related to the Jenkins CI configuration.
+
  
 
== Scope ==
 
== Scope ==
<!-- Project scope.  The project scope should be well defined.  It should be possible from the scope to crisply answer whether something belongs or not within the scope of this particular project.  Scopes should not be overly broad.  A Project scope must also lie within the overall scope set by the board for projects in fd.io:
+
Project initial target development areas include:
    - IO
+
        – Hardware/vHardware <-> threads/cores
+
    - Processing
+
        - Classify
+
        - Transform
+
        - Prioritize
+
        - Forward
+
        - Terminate
+
    - Management Agents
+
        - Control/Manage IO/Processing
+
    - Supporting Projects
+
        - Testing/Tools/Infrastructure
+
        - Integration with other systems
+
-->
+
  
Fd.io CSIT project scope:
+
* Automation tooling with python interface (API) for packet traffic generation (e.g. Ixia) and throughput benchmark test definitions, life-cycle management and granular measurement data analytic.
 
+
* A set of scripts to create Ubuntu live image with benchmark-centric customizations (pre-installed benchmarks and library dependencies).
# Automated regression testing of VPP code changes
+
** The identical image will be able to deploy in various test environments, such as bare-metal, VM setup. This enable a consistent and calibrated environment for performance testing.
#* Functionality of VPP data plane, network control plane, management plane against functional specifications.
+
* Tools to calibrate OS level CPU “jitter” (CPU core context switching) introduced by kernel- and other user-space tasks.
#* Performance of VPP data plane including non-drop-rate packet throughput and delay, against established reference benchmarks.
+
#* Performance of network control plane against established reference benchmarks.
+
#* Performance of management plane against established reference benchmarks.
+
# Test case definitions driven by supported and planned VPP functionality, interfaces and performance:
+
#* Uni-dimensional tests: Data plane, (Network) Control plane, Management plane.
+
#* Multi-dimensional tests: Use case driven.
+
# Integration with FD.io Continuous Integration system including FD.io Gerrit and Jenkins
+
#* Automated test execution triggered by VPP-VERIFY jobs other VPP and CSIT project jobs.
+
# Integration with LF VPP test execution environment
+
#* Functional tests execution on LF hosted VM environment.
+
#* Performance and functional tests execution on LF hosted physical compute environment.
+
#* Subset of tests executed on LF hosted physical compute running VIRL (Virtual Internet Routing Lab).
+
  
 
== Initial Committers ==
 
== Initial Committers ==
<!-- A list of the name/email/IRC nick of the initial project committers
+
* [mailto:patrick.lu@intel.com Patrick Lu], LF-ID: yplu.
 
+
* [mailto:georgii.tkachuk@intel.com George Tkachuk], LF-ID: tbc.
IMPORTANT:  Committers should be people who will actually write code, being a committer is an actual commitment of time.  Please also note that committerness is an individual trait.  If a committer changes employers, they remain a committer.  New committers arise via meritocracy after the project is created, this typically involves some time of establishing history of meritocractic code contribution to the project..  Therefore, it is crucial that a committer is committed to ongoing work on the project in the longer term, not just short term.  For more information on how committers are added after project creation see:
+
* [mailto:subhiksha.ravisundar@intel.com Subhiksha Ravisundar], LF-ID: tbc.
https://fd.io/sites/cpstandard/files/pages/files/exhibit_c_-_fd.io_technical_community_charter.pdf section 3.2.2.1.
+
 
+
-->
+
 
+
* [mailto:mklotton@cisco.com Matej Klotton], LF-ID: mklotton.
+
* [mailto:skobza@cisco.com Stefan Kobza], LF-ID: skobza.
+
* [mailto:ckoester@cisco.com Carsten Koester], LF-ID: ckoester.
+
 
* [mailto:mkonstan@cisco.com Maciek Konstantynowicz], LF-ID: mackonstan.
 
* [mailto:mkonstan@cisco.com Maciek Konstantynowicz], LF-ID: mackonstan.
* [mailto:mmiklus@cisco.com Miroslav Miklus], LF-ID: mmiklus.
+
* [mailto:pmikus@cisco.com Peter Mikus] LF-ID: pmikus.
* [mailto:dwallace@cisco.com Dave Wallace], LF-ID: dwallace.
+
  
 
== Vendor Neutral ==
 
== Vendor Neutral ==
<!--
 
The goal here is to capture the degree of vendor neutrality of the code.
 
The concerns are two fold: avoiding trademark issues, and maintaining openness.
 
 
For this reason, use of vendor names should be purely functional, and only if necessary to
 
reasonably communicate functional information to the user.
 
 
Acceptable Examples:
 
Indicating the presence of particular hardware
 
Indicating drivers for particular hardware
 
Indicating integration with particular technologies.
 
 
Unacceptable Examples:
 
Use of vendor trademarks or product names purely for marketing purposes.
 
 
Please describe any such issues here.
 
-->
 
 
 
No current or expected issues with vendor neutrality.
 
No current or expected issues with vendor neutrality.
  
 
== Meets Board Policy (including IPR, being within Board defined Scope etc) ==
 
== Meets Board Policy (including IPR, being within Board defined Scope etc) ==
  
Meets board policy as expressed in [https://fd.io/sites/cpstandard/files/pages/files/exhibit_c_-_fd.io_technical_community_charter.pdf Technical Community Charter] and [https://fd.io/sites/cpstandard/files/pages/files/exhibit_b_-_fd.io_ip_policy.pdf IP Policy].
+
Meets board policy as expressed in [https://fd.io/sites/cpstandard/files/pages/files/exhibit_c_-_fd.io_technical_community_charter.pdf Technical Community Charter] and [https://fd.io/sites/cpstandard/files/pages/files/exhibit_b_-_fd.io_ip_policy.pdf IP Policy]
 
+
== Collateral ==
+
  
[https://docs.google.com/presentation/d/1WnbX_S-ZmHef-SEBtdouuFAtLG5FIM0a6mJkIOreHR4/edit?usp=sharing CSIT Project Creation Review Slides] discussed on FD.io TSC call on 17-Mar-2016.
+
== Administrata ==
 +
* Request for Project proposal consideration
 +
** Email: (place link to email to TSC proposing project, this can be obtained from [https://lists.fd.io/pipermail/tsc/ TSC Archives]
 +
** Date: (date proposed, makes it simpler to calculate the pre-requisite 2 week time period of gestation before being permitted to be voted on)

Latest revision as of 20:15, 23 October 2017


Name

Performance Measurement and Analysis Tools

Project Contact Name and Email

Patrick Lu <patrick.lu@intel.com>

Repository Name

pma_tools

Description

pma_tools project aims to develop software tools for performance and efficiency measurements of compute optimized network data plane applications like FD.io VPP (and similar) while running on compute platform(s) . pma_tools will provide tools that enable systematic measurement methodologies of such applications and are to be directly integrated within the FD.io operating environment(s) and FD.io projects like VPP, CSIT and TRex using developed APIs. This could facilitate further standardization of performance measurement and analysis methodologies for compute platforms running network workloads.

Scope

Project initial target development areas include:

  • Automation tooling with python interface (API) for packet traffic generation (e.g. Ixia) and throughput benchmark test definitions, life-cycle management and granular measurement data analytic.
  • A set of scripts to create Ubuntu live image with benchmark-centric customizations (pre-installed benchmarks and library dependencies).
    • The identical image will be able to deploy in various test environments, such as bare-metal, VM setup. This enable a consistent and calibrated environment for performance testing.
  • Tools to calibrate OS level CPU “jitter” (CPU core context switching) introduced by kernel- and other user-space tasks.

Initial Committers

Vendor Neutral

No current or expected issues with vendor neutrality.

Meets Board Policy (including IPR, being within Board defined Scope etc)

Meets board policy as expressed in Technical Community Charter and IP Policy

Administrata

  • Request for Project proposal consideration
    • Email: (place link to email to TSC proposing project, this can be obtained from TSC Archives
    • Date: (date proposed, makes it simpler to calculate the pre-requisite 2 week time period of gestation before being permitted to be voted on)