Difference between revisions of "CSIT"

Last updated 29 days ago

Revision as of 15:01, 19 July 2016

Description

CSIT (Continuous System Integration and Testing) project is focusing on:

1. Development of software code for fully automated VPP code testing, functionality, performance, regression and new functions.
2. Execution of CSIT test suites on VPP code running on LF FD.io virtual and physical compute environments.
3. Integration with FD.io continuous integration systems (Gerrit, Jenkins and such).
4. Identified existing FD.io project dependencies and interactions:
   • vpp - Vector Packet Processing.
   • honeycomb - Honeycomb Agent for management plane testing.
   • ci-management - Management repo for Jenkins Job Builder, script and management related to the Jenkins CI configuration.

Get Involved

• Weekly CSIT Meeting.
• Join the CSIT Mailing List.
• Join fdio-csit IRC channel.
• CSIT dev work tracking

Scope

Fd.io CSIT project scope:

1. Automated regression testing of VPP code changes
   • Functionality of VPP data plane, network control plane, management plane against functional specifications.
   • Performance of VPP data plane including non-drop-rate packet throughput and delay, against established reference benchmarks.
   • Performance of network control plane against established reference benchmarks.
   • Performance of management plane against established reference benchmarks.
2. Test case definitions driven by supported and planned VPP functionality, interfaces and performance:
   • Uni-dimensional tests: Data plane, (Network) Control plane, Management plane.
   • Multi-dimensional tests: Use case driven.
3. Integration with FD.io Continuous Integration system including FD.io Gerrit and Jenkins
   • Automated test execution triggered by VPP-VERIFY jobs other VPP and CSIT project jobs.
4. Integration with LF VPP test execution environment
   • Functional tests execution on LF hosted VM environment.
   • Performance and functional tests execution on LF hosted physical compute environment.
   • Subset of tests executed on LF hosted physical compute running VIRL (Virtual Internet Routing Lab).

CSIT Documentation

Documentation

1. CSIT Documentation
   • CSIT Code Structure
   • CSIT Test Code Guidelines

Branching Strategy

CSIT Branching Strategy

Submitting Patches

• Submitting Patches to CSIT

Test Information Model

• CSIT Test Information Model

CSIT VPP-16.06 Test Report

• CSIT/VPP-16.06_Test_Report

CSIT Test Plans

Functional

• CSIT Functional Test Plan

Performance

• CSIT Performance Test Plan

Test Infrastructure

• CSIT vNet Infrastructure Plan

CSIT Jobs

• CSIT Jenkins Jobs listing

CSIT Performance Testbed in LF

• CSIT Performance Testbed

CSIT Tutorials

• Local execution of CSIT using Vagrant+Virtualbox on Windows 7
• Local execution of CSIT using Vagrant+Virtualbox on Mac OS X
• Local execution of CSIT using Vagrant+Virtualbox on Ubuntu

CSIT Materials

• FD.io DevBoot - San Jose 4th to 7th of April 2016 - CSIT + Performance testing

CSIT Project Page Content to_be_added

• Documentation
  • Descriptions
    • CSIT code that's running: functional and performance tests.
    • CSIT testbeds
      • Operational: physical, virtual-VIRL.
      • Non-operational: virtual-VM.
    • Running CSIT integration with FD.io gerrit and jenkins.
  • CSIT usability manual
    • Instructions how to use CSIT to verify VPP code commits and avoid breaking it.
    • Onboarding of new VPP test cases onto CSIT system.
• CSIT work in progress
  • Code in development, priorities.
  • Development plan, priorities.
  • FD.io jira for tracking execution against the plan, scheme proposal.