|
|
| (9 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| − | == Honeycomb testing Summary ==
| + | For up-to-date information on Honeycomb testing see [https://docs.fd.io/csit/master/report the report.] |
| − | For details on the Honeycomb project itself, [[Honeycomb | check out its own page]]. This page assumes you are also familiar with the rest of CSIT. | + | Test case and code documentation can be found [https://docs.fd.io/csit/master/doc here], in the "honeycomb" sub-section of each topic. |
| − | | + | |
| − | | + | |
| − | Execution info:
| + | |
| − | * test suite is run separately from the rest of CSIT
| + | |
| − | * a test run is triggered by every successful run of "honeycomb-integration" Jenkins job (which includes unit testing)
| + | |
| − | * in addition to CSIT requirements, test runs require Honeycomb to be set up on the primary DUT as a Linux service
| + | |
| − | | + | |
| − | | + | |
| − | Topology and misc:
| + | |
| − | * tests are primarily CRUD, as such test cases only utilize a single DUT node with no traffic generator
| + | |
| − | * planned traffic tests will use standard two-node double-link topology with one DUT and TG
| + | |
| − | * unlike CSIT core tests, VPP is not restarted before/after every test but only after a test failure - due to time constraints
| + | |
| − | * all tests are tagged with "honeycomb_sanity" (subject to change)
| + | |
| − | | + | |
| − | | + | |
| − | Test structure: | + | |
| − | * separate "honeycomb" folders exist for Honeycomb-specific python libraries, robot files, test data and command templates
| + | |
| − | * Restconf communication with Honeycomb uses python's Requests library
| + | |
| − | * Netconf communication with Honeycomb uses python's Paramiko library
| + | |
| − | * Honeycomb maintains two data stores:
| + | |
| − | *# Operational data - read-only, built from VPP-API data dumps and should reflect state of VPP configuration
| + | |
| − | *# Configuration data - desired configuration; pushed to VPP whenever a change is made, and on initial connection or reconnection to VPP
| + | |
| − | * CRUD test flow is generally:
| + | |
| − | *# Check current status of Honeycomb operational data
| + | |
| − | *# Check current status from VAT dump
| + | |
| − | *# Make a change to configuration data
| + | |
| − | *# Verify change in operational data
| + | |
| − | *# Verify change in VAT dump
| + | |
| − | *# [Teardown] Revert changes
| + | |
| − | | + | |
| − | == Summary of Existing Tests ==
| + | |
| − | The Honeycomb suite contains tests for the following areas:
| + | |
| − | * (physical) Interface management:
| + | |
| − | ** state changes
| + | |
| − | ** IP addresses
| + | |
| − | ** IP neighbors
| + | |
| − | * Bridge domains:
| + | |
| − | ** CRUD, interface assignment
| + | |
| − | ** L2 FIB entries
| + | |
| − | * Vxlan, VxLan GPE tunnels
| + | |
| − | * TAP interfaces
| + | |
| − | * Vhost-user interfaces
| + | |
| − | * Vlan sub-interfaces
| + | |
| − | * ACL (access-control-lists)
| + | |
| − | * PBB (provider-backbone-bridge) sub-interface
| + | |
| − | * non-CRUD tests:
| + | |
| − | ** Persistence suite:
| + | |
| − | ***persist and/or restore configuration through a restart of Honeycomb, VPP or both
| + | |
| − | ** Netconf suite:
| + | |
| − | *** basic stress test - replicates a specific issue with executing many commands in quick succession
| + | |
| − | *** Notifications - listen for notifications that should be sent out on interface creation, deletion and state change
| + | |
| − | | + | |
| − | == Planned Test Cases ==
| + | |
| − | * High-level ACLs (IETF-ACL):
| + | |
| − | ** utilizing an alternative ACL node/model, should simplify the process of setting up complex classify chains
| + | |
| − | ** may require traffic tests instead of CRUD, supposedly a comprehensive VAT dump is "not possible"
| + | |
| − | * LISP
| + | |
| − | * NSH_SFC
| + | |
| − | * Netconf transaction revert
| + | |
