Difference between revisions of "CSIT/Honeycomb/Status"

From fd.io
Jump to: navigation, search
(Added CSIT-Honeycomb status page)
 
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
(Up to date as of 12. October 2016). This page will (hopefully) be updated every Wednesday, before the [[CSIT/Meeting | CSIT weekly public call]].
 
 
== Status of CSIT-Honeycomb tests ==
 
== Status of CSIT-Honeycomb tests ==
  
−
* '''Tests are currently not automated!''' The Jenkins job triggers need to be updated, and the [https://gerrit.fd.io/r/3018 relevant commit] is stuck in review limbo.
+
Currently running two types of test jobs on Ubuntu16.04:
−
* Manual test runs are possible on Ubuntu 14.04 DUTs with an installed Honeycomb .deb package
+
* hc2vpp-csit-'''integration''':
−
* Testing on Ubuntu 16.04 DUTs currently not possible due to a Honeycomb issue with Linux service integration - [https://jira.fd.io/browse/HONEYCOMB-217 Jira ticket]
+
** for testing integration of VPP, HC2VPP and Honeycomb projects.
 +
** runs after a patch is merged to VPP or HC2VPP projects.
 +
* hc2vpp-csit-'''verify''':
 +
** for verifying commits to HC2VPP
 +
** can be run manually on specific commits to HC2VPP project
 +
Centos7 jobs will be added after CSIT-functional has stable Centos jobs.
  
 +
Both jobs are duplicated for HC2VPP branches, eg. master, stable/1701, stable/1704, ...
  
−
== Observed Test Failures ==
+
Negative test results do not prevent commits from being merged, as issues are often caused by VPP changes and not only HC2VPP. They do provide history as well as rapid issue detection.
−
 
+
−
'''Note:''' Unlike CSIT-core, Honeycomb tests do not add Gerrit "verified" labels to changes (for now). They are only meant to provide information and track issue history with logs.
+
−
 
+
−
Lists cause of failure, explanation and Jira ticket where applicable:
+
−
+
−
(Results from local testing, since Jenkins job is not running yet.)
+
−
* Interface management:
+
−
** IPv4 address mismatch in VAT dump:
+
−
*** VAT dump returns IP addresses in reversed byte order
+
−
*** [https://jira.fd.io/browse/VPP-132 VPP-132]
+
−
** Cannot read operational data for IPv4/IPv6 neighbor
+
−
*** missing VPP API for reading neighbor list
+
−
*** [https://jira.fd.io/browse/HONEYCOMB-102 HONEYCOMB-102 ] [https://jira.fd.io/browse/VPP-333 VPP-333 ]
+
−
** Cannot set IPv6 address
+
−
*** not implemented
+
−
*** [https://jira.fd.io/browse/HONEYCOMB-126 HONEYCOMB-126 ] [https://jira.fd.io/browse/VPP-82 VPP-82 ]
+
−
* VxLAN GPE:
+
−
** cannot verify next-protocol value:
+
−
*** VAT dump returns incorrect next-protocol value
+
−
*** [https://jira.fd.io/browse/VPP-217 VPP-217], to be resolved in [https://jira.fd.io/browse/VPP-156 VPP-156]
+
−
* Vlan sub-interface:
+
−
** IPv4 address mismatch in VAT dump, same as with generic interfaces
+
−
* ACL:
+
−
** Cannot remove ACL table if multiple tables exist:
+
−
*** VPP removes entire table chain, causing desync between Honeycomb operational and config data
+
−
*** [https://jira.fd.io/browse/VPP-206 VPP-206]
+

Latest revision as of 10:06, 7 April 2017

Status of CSIT-Honeycomb tests

Currently running two types of test jobs on Ubuntu16.04:

  • hc2vpp-csit-integration:
    • for testing integration of VPP, HC2VPP and Honeycomb projects.
    • runs after a patch is merged to VPP or HC2VPP projects.
  • hc2vpp-csit-verify:
    • for verifying commits to HC2VPP
    • can be run manually on specific commits to HC2VPP project

Centos7 jobs will be added after CSIT-functional has stable Centos jobs.

Both jobs are duplicated for HC2VPP branches, eg. master, stable/1701, stable/1704, ...

Negative test results do not prevent commits from being merged, as issues are often caused by VPP changes and not only HC2VPP. They do provide history as well as rapid issue detection.

Retrieved from "https://wiki.fd.io/index.php?title=CSIT/Honeycomb/Status&oldid=4808"