Honeycomb/Plan

UNDER CONSTRUCTION

Honeycomb plan #1

Story: Going formal

Various tasks required by honeycomb project officially joining fd.io

1. Evacuate vbd sub-project into dedicated ODL project

Story: DataTree

Refactor/Redesign Data-store utilization in Honeycomb.

Using global data-store (current design) is very restrictive and does not allow for features like: commit refusal, change processing ordering, additional validation etc. Dedicated Data-tree needs to be used internally in order to introduce better control over the data processing in Honeycomb agent

Tasks

1. Analyze DataTree APIs and design
2. Document and put on wiki
3. Implement custom DataBroker on top of a DataTree
   1. Provide APIs for translation layer
4. Add a dedicated mount-point for the new pipeline while still keeping former implementation in place
   1. Wrap HC DataTree in a mountpoint
   2. Configure a dedicated NETCONF northbound just for HC mountpoint

Story: Translation layer

Refactor/Redesign Honeycomb Translation layer (YANG <-> VPP API) and introduce a framework.

The translation layer is very monolithic, hard to extend and buggy. Before any new VPP functionality is added, refactoring and redesign needs to take place to allow for “easy to develop and deploy” extensions

Tasks

1. Design translation layer (extensible, easy to use, Binding aware, supporting CRUD, R separated from the rest of CRUD ops etc.)
2. Document and put on wiki
3. Implement R from CRUD
   1. Introduce Reader APIs
   2. Implement Readers in a composite, extensible manner
      1. Provide SPIs to customize read behavior
   3. Migrate existing reading code from Honeycomb under new translation layer
4. Implement CUD from CRUD
   1. Introduce Writer APIs
   2. Implement Writers in a composite, extensible manner
      1. Provide SPIs to customize write behavior
   3. Migrate existing writing code from Honeycomb under new translation layer
5. Integrate with DataTree story
6. Remove former pipeline and mapping code from Honeycomb and keep only new pipeline (DataTree and Translation layer stories)

Story: vpp-japi refactoring

Generated vpp-japi (part of VPP project) today is asynchronous and works, but there are some drawbacks:

• Not really asynchronous from Java perspective, requires Java to perform active wait loops (for all generated functions) – big overhead due to JNI boundary being crossed lots of times
• Requires hand crafted implementations for dump calls

The japi should be:

• Truly asynchronous with callbacks into Java
• Lightweight (no caching in the C code, generate all the methods except connect, disconnect and ping etc.)

Tasks

1. Design new version of japi with above requirements
2. Document and put on wiki
3. Implement a POC to verify and measure base performance
4. Develop the POC into a full japi v2 (sub task are subject to changes)
   1. Update the code generation to generate async vpp api (generate interface also for easier integration)
   2. Update the build of VPP to produce and deploy vpp-japi v2 jar
5. Migrate Honeycomb to use vpp-japi v2
6. Deprecate vpp-japi
7. Remove vpp-japi

Story: VPP protobuf support

VPP is getting new fast path management channel using Google Protocol Buffers that will allow management of VPP routes without the overhead of NETCONF / RESTCONF protocols.

Honeycomb is official way of managing VPP so we need to add support for configuring the GPB channel on VPP as well as some features that will allow remote VPP routes management client to get all the important information for managing the VPP routes (e.g. interface state changes).

Tasks

1. Honeycomb
   1. Interface state change notifications support
   2. VPP GPB plugin configuration
   3. Tap interface configuration & state
   4. VXLAN & NSH configuration & state
   5. BGP & ARP capture and encap through VXLAN+NSH tunnel configuration
   6. Once VPP routes will be supported by Honeycomb we will need a way to lock the routes subtree so routes cannot be modified when GPB channel is used
2. ODL
   1. GPB server receiving routes from route managing machine
   2. Topology handling / separating routes for different VPPs
   3. Configure VPP interfaces on both route managing machine as well as VPP along with VXLAN+NSH SI/SPI mapping
   4. Configure VXLAN + NSH tunnel between VPP and route managing machine
   5. Configure BGP & ARP capture and encap through VXLAN+NSH on VPP
   6. GPB client that will connect with VPP GPB plugin and send route updates to VPP
3. VPP
   1. Capture BGP & ARP traffic and send it through VXLAN+NSH over UDP
4. VPP protobuf plugin
   1. Handle TCP connections to/from clients on port configured by Honeycomb
   2. GPB proto file with messages definition
   3. Process GPB requests based on the proto file definition
   4. Update routes on VPP based on data processed from GPB requests

Story: Minimal distro

Today, Honeycomb includes many (for Honycomb unnecessary) ODL features e.g. clustered DS.

The distribution needs to be shrunk either by minimizing ODL features in karaf or completely removing karaf from the agent using lightweight or static wiring and configuration.

Tasks

1. Introduce new wiring based on a simple DI framework
   1. Analyze and pick suitable DI framework
   2. Add new wiring into existing honeycomb components
2. Implement new startup mechanism (maybe just a simple Main)
3. Provide minimal distribution including new wiring and startup mechanism
4. Remove karaf related and ODL related stuff(distrobution, wiring etc) keeping only minimal distribution
5. Document and put on wiki

Misc

Various tasks

Tasks

1. Put this plan into JIRA
2. Present new Honeycomb pipeline in TWS meeting. Blocked by: DataTree story, Translation layer story
3. Provide tutorial/samples on: How-to-add-new-features-to-HC. Blocked by: DataTree story, Translation layer story
4. Use Java8 in Honeycomb sources
5. Cleanup maven structure in Honeycomb
   1. Enable proper checkstyle checks + license checks
   2. Enable jacoco coverage reports and display in Jenkins at least
   3. Remove unnecessary empty parent poms
6. Sonar