Difference between revisions of "Archived-Sweetcomb"

From fd.io
Jump to: navigation, search
m (Valderrv moved page Sweetcomb to Archived-Sweetcomb)
 
(61 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
{{Project Facts
 
{{Project Facts
|name=Honeycomb
+
|name=Sweetcomb
|shortname=honeycomb
+
|shortname=sweetcomb
|jiraName=HONEYCOMB
+
|jiraName=sweetcomb
|projectLead=Marek Gradzki
+
|projectLead=[mailto:hongjun.ni@intel.com Hongjun Ni], @ Intel
 
|committers=
 
|committers=
* Dave Wallace
+
* [mailto:drenfong.wang@intel.com Drenfong Wang], @ Intel,
* Maros Marsalek
+
* [mailto:wangchuanguo@huachentel.com Chuanguo Wang], @ HuachenTel,
* Marek Gradzki
+
* [mailto:chenhui@huachentel.com Hui Chen], @ HuachenTel,
* Ed Warnicke
+
* [mailto:18867105320@139.com Zhuoqun Li], @ China Mobile,
* Robert Varga
+
* [mailto:yangjj731@chinaunicom.cn Jianjun Yang], @ China Unicom,
* Juraj Sloboda
+
* [mailto:zhijl@chinatelecom.cn Jinglong Zhi], @ China Telecom,
 +
* [mailto:changlin.li@nxp.com Changlin Li], @ NXP,
 +
* [mailto:tianyi.wang@tieto.com Tianyi Wang], @ Tieto,
 +
* [mailto:davidfgao@tencent.com Feng Gao], @ Tencent,
 +
* [mailto:jingqing.cjy@alibaba-inc.com Jianyong Chen], @ Alibaba,
 +
* [mailto:gu.jian1@zte.com.cn Jian Gu], @ ZTE,
 +
* [mailto:jtollet@cisco.com Jerome Tollet], @ Cisco,
 +
* [mailto:mkonstan@cisco.com Maciek Konstantynowicz], @ Cisco,
 +
* [mailto:rastislav.szabo@pantheon.tech Rastislav Szabo], @ Pantheon Technologies,
 +
* [mailto:hongjun.ni@intel.com Hongjun Ni], @ Intel,
 
}}
 
}}
 
== Intro ==
 
== Intro ==
  
Honeycomb is a generic NETCONF/RESTCONF java-based management agent and provides a framework for building specialized agents.
+
Sweetcomb is a management agent written in C that runs on the same host as a VPP instance, and exposes yang models via NETCONF, RESTCONF and gNMI to allow the management of VPP instances.
  
It uses many features and utilities from different [http://www.opendaylight.org Opendaylight (ODL)] projects (e.g. yangtools, controller, mdsal and netconf).
+
Sweetcomb work as a plugin (ELF shared library) for sysrepo datastore.
The biggest use case of honeycomb is VPP (fd.io’s Vector packet processor). Honeycomb delivers a management agent to enable integration of VPP with e.g. SDN controllers such as Opendaylight.
+
 
+
For VPP specific distribution of Honeycomb, please refer to [https://wiki.fd.io/view/Hc2vpp hc2vpp] project.
+
  
 
=== Overview ===
 
=== Overview ===
 +
[[File:Sweetcomb-overview.png|frameless|Sweetcomb Architecture and environment]]
  
[[File:Honeycomb + VPP stack.png|frameless|Honeycomb in VPP stack]]
+
=== Project Termination ===
 +
[[Sweetcomb/Termination Review | Sweetcomb/Termination Review]]
  
==== Slides ====
 
  
Honeycomb overview presentation from Paris FD.io 2016 event: [[File:Honeycomb overview.pptx|thumb|Honeycomb overview prepared for Paris FD.io 2016 event]]
+
Draw.io source [https://github.com/YohanPipereau/drawio_sweetcomb/blob/master/sweetcomb.xml here]
  
Honeycomb 1609 overview update from fd.io Seattle summit 2016: [[File:HC 1609 overview.pptx|thumb|Slides detailing Honeycomb's goal, architecture and features as of 1609 release]]
+
== Project Contact ==
 +
* [mailto:hongjun.ni@intel.com Hongjun Ni], @ Intel,
 +
* [mailto:lixingfu@huachentel.com Xingfu Li], @ HuachenTel,
 +
* [mailto:18867105320@139.com Zhuoqun Li], @ China Mobile,
 +
* [mailto:yangjj731@chinaunicom.cn Jianjun Yang], @ China Unicom,
 +
* [mailto:zhijl@chinatelecom.cn Jinglong Zhi], @ China Telecom,
 +
* [mailto:changlin.li@nxp.com Changlin Li], @ NXP,
 +
* [mailto:tianyi.wang@tieto.com Tianyi Wang], @ Tieto,
 +
* [mailto:davidfgao@tencent.com Feng Gao], @ Tencent,
 +
* [mailto:jingqing.cjy@alibaba-inc.com Jianyong Chen], @ Alibaba,
 +
* [mailto:gu.jian1@zte.com.cn Jian Gu], @ ZTE,
 +
* [mailto:jtollet@cisco.com Jerome Tollet], @ Cisco,
 +
* [mailto:George.Y.Zhao@huawei.com George Zhao], @ Huawei,
  
==== Generic nature ====
+
== Scope ==
 +
<!-- Project scope.  The project scope should be well defined.  It should be possible from the scope to crisply answer whether something belongs or not within the scope of this particular project.  Scopes should not be overly broad.  A Project scope must also lie within the overall scope set by the board for projects in fd.io:
 +
    - IO
 +
        – Hardware/vHardware <-> threads/cores
 +
    - Processing
 +
        - Classify
 +
        - Transform
 +
        - Prioritize
 +
        - Forward
 +
        - Terminate
 +
    - Management Agents
 +
        - Control/Manage IO/Processing
 +
    - Supporting Projects
 +
        - Testing/Tools/Infrastructure
 +
        - Integration with other systems
 +
-->
 +
Sweetcomb's main responsibility is to enable communication between northbound interfaces and VPP's management APIs, performing all necessary translations in the background. It is important to note that many features and utilities will be reused from open source projects and tools (e.g. netopeer2, Sysrepo) and will not be a direct part of Sweetcomb. This section is splitted into 2 sections: in-scope and out-of-scope to clearly define what is developed as part of Sweetcomb project and what will be just reused from other projects (or where Sweetcomb relies on other projects).
  
Honeycomb provides a distribution to control VPP. However HC is developed as a generic agent that can be positioned on top of any other software/system etc. that can be controlled from Java(or some other JVM language) code. This is achieved thanks to the modularity of HC enabling any "translation plugin" such as V3PO (VPP core management translation code) to work in HC framework.
+
'''Sweetcomb project scope:'''
  
==== Minimal distribution ====
+
* Northbound interfaces exposed:
 +
** [http://tools.ietf.org/html/rfc4741, Netconf] from [https://github.com/CESNET/Netopeer2 CESNET/Netopeer2]
 +
** [https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md, gNMI] northbound interface from [https://github.com/YohanPipereau/sysrepo-gnxi sysrepo-gnxi]
 +
** [https://tools.ietf.org/html/draft-ietf-netconf-restconf-05, Restconf] (does not exist yet)
  
Honeycomb provides a minimal distribution that contains only:
+
* Translation layer between VPP management and Yang based data structures
* HC framework jars + plugin jars + dependencies from Opendaylight
+
** Must support all features of VPP exposed in its APIs in an extensible manner
* HC framework config files + plugin config files
+
* Runnable shell scripts
+
  
== Project Contact Name and Email ==
+
* Write YANG models for VPP specific features
* [mailto:honeycomb-dev@lists.fd.io honeycomb-dev list],
+
* [mailto:margradz@gmail.com Marek Gradzki], LF-ID: mgradzki,
+
* [mailto:mmarsale@cisco.com Maros Marsalek], LF-ID: mmarsale,
+
* [mailto:hagbard@gmail.com Ed Warnicke], LF-ID: hagbard,
+
* [mailto:dwallacelf@gmail.com Dave Wallace], LF-ID: dwallacelf
+
  
== Meeting ==
+
* Expose APIs to integrate with other open source projects
 
+
** Base implementation of all generic southbound interfaces leverage VPP-VAPI
[https://wiki.fd.io/view/Honeycomb/Meeting Honeycomb meeting]
+
** expose APIs to integrate with SD-WAN control plane, such as SDN Controller.
 
+
** expose APIs to integrate with Routing Daemon, such as FRR.
== Scope ==
+
** expose APIs to integrate with IKE protocol, such as strongswan.
 
+
** expose APIs to integrate with DPI control plane, such as nDPI.
This section is devided into 2 sections: in-scope and out-of-scope to clearly define what is developed as part of Honeycomb project and what will be just reused from other projects (or where Honeycomb relies on other projects).
+
** expose APIs to integrate with BRAS control plane, such as OpenBRAS.
 
+
'''Honeycomb project scope:'''
+
* Northbound interfaces:
+
** [http://tools.ietf.org/html/rfc4741, Netconf] northbound interface
+
** [https://tools.ietf.org/html/draft-ietf-netconf-restconf-05, Restconf] northbound interface
+
** [https://www.ietf.org/rfc/rfc4271, BGP]
+
 
+
* Translation layer between device management and Yang based data structures
+
 
+
* Honeycomb distribution
+
** Minimal distribution for Honeycomb management agent with all dependencies from ODL and their wiring
+
  
 
'''Out of scope:'''
 
'''Out of scope:'''
* Yang models and translation code for VPP management
+
* VPP API (VAPI or VOM)
** Moved to [https://wiki.fd.io/view/Hc2vpp hc2vpp] in 17.01
+
** '''VAPI:''' C APIs for VPP, allowing C-based applications to interact with VPP is out of scope of Sweetcomb project and is part of the base VPP project.
 +
** '''VOM:''' C++ based API offering storage of networking objects
  
* Vpp-japi
+
* Writing yang models for VPP management
** Java APIs for VPP, allowing JVM to interact with VPP is out of scope of Honeycomb project and is part of the base VPP project
+
** IETF and Openconfig YANG models are used as much as possible
 +
** For missing features, we rely on Honeycomb YANG models
  
* Yang parser
+
* Compliancy with YANG models, storage of YANG tree
** Available from ODL's [https://git.opendaylight.org/gerrit/#/admin/projects/yangtools, yangtools] project
+
** Provided by sysrepo project and libraries it uses
  
* Base implementation of northbound interfaces
+
* Server implementation of northbound interfaces
** Base implementation of generic northbound interfaces is out of scope and will be reused from ODL's [https://git.opendaylight.org/gerrit/#/admin/projects/netconf, netconf] project
+
** [http://tools.ietf.org/html/rfc4741, Netconf] from [https://github.com/CESNET/Netopeer2 CESNET/Netopeer2]
 
+
** [https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md, gNMI] northbound interface from [https://github.com/YohanPipereau/sysrepo-gnxi sysrepo-gnxi]
* Yang based data structures and a dedicated DataTree(storage)
+
** Available from ODL's [https://git.opendaylight.org/gerrit/#/admin/projects/yangtools, yangtools] and [https://git.opendaylight.org/gerrit/#/admin/projects/mdsal, mdsal] projects, not necessarily the entire projects, only required parts in binary form.
+
  
 
* Integration/performance testing
 
* Integration/performance testing
** Complex integration or performance tests are out of scope of Honeycomb project and are part of CSIT project
+
** Complex integration or performance tests are out of scope of Sweetcomb project and are part of CSIT project
  
* Any other application based on top of Honeycomb is out of scope of this project and needs to be hosted in a dedicated project inside or outside of fd.io
+
* Any other application based on top of Sweetcomb is out of scope of this project and needs to be hosted in a dedicated project inside or outside of fd.io
  
 
== Releases ==
 
== Releases ==
  
=== 18.07 - Last stable  ===
+
Sweetcomb releases are based on VPP version numbers as sweetcomb uses VPP API which is not backward compatible.
  
'''[[Honeycomb/Release_Plans/Release_Plan_18.07| Release plan]]''':
+
* [[Sweetcomb/20.01 Release Plan | 20.01 Release Plan]]
* Work planned for release
+
* Milestones leading to the release
+
  
[https://docs.fd.io/honeycomb/1.18.07/honeycomb-aggregator/index.html '''Documentation''']:
+
* [[Sweetcomb/19.08 Release Notes | 19.08 Release Notes]]
* Devel documentation
+
* [[Sweetcomb/19.04 Release Notes | 19.04 Release Notes]]
* Overview
+
* [[Sweetcomb/19.01 Release Notes | 19.01 Release Notes]]
* Details
+
* Javadocs
+
* ... for each module in Honeycomb project
+
  
[https://docs.fd.io/honeycomb/1.18.07/release-notes-aggregator/release_notes.html '''Release notes''']:
+
== Contributing ==
* New/Updated/Removed/Deprecated Features
+
* Install guide
+
* Devel guide
+
* User guide
+
  
=== 18.10 - Current master ===
+
Contributions must go through code-review before being merged:
 +
    git clone https://gerrit.fd.io/r/sweetcomb
  
'''[[Honeycomb/Release_Plans/Release_Plan_18.10| Release plan]]''':
 
* Work planned for release
 
* Milestones leading to the release
 
  
[https://docs.fd.io/honeycomb/1.18.10-SNAPSHOT/honeycomb-aggregator/index.html '''Documentation''']:
+
Feel free to subscribe to the following mailing lists:
* Devel documentation
+
* '''[https://lists.fd.io/g/sweetcomb-dev sweetcomb-dev@lists.fd.io]''' : to be notified about sweetcomb talks
* Overview
+
* '''[https://lists.fd.io/g/vpp-api-dev/ vpp-api-dev@lists.fd.io]''': to be notified about updated of the non-backward compatible VPP API
* Details
+
* '''[http://www.sysrepo.org/ sysrepo-announce@sysrepo.org ]''' : to be notified about sysrepo releases
* Javadocs
+
* '''[http://www.sysrepo.org/ sysrepo-devel@sysrepo.org ]''' : to be notified about sysrepo bugs, tips, discussions
* ... for each module in Honeycomb project
+
  
[https://docs.fd.io/honeycomb/1.18.10-SNAPSHOT/release-notes-aggregator/release_notes.html '''Release notes''']:
+
== Devel ==
* New/Updated/Removed/Deprecated Features
+
* Install guide
+
* Devel guide
+
* User guide
+
  
=== Previous releases ===
+
Source code is divided in two directories:
==== 18.04 ====
+
* '''scvpp''': library source code to communicate with VAPI (VPP API).
* '''[[Honeycomb/Release_Plans/Release_Plan_18.04| Release plan]]''':
+
* '''plugins''': sysrepo plugin library source code implementing xpath callbacks
* [https://docs.fd.io/honeycomb/1.18.04/release-notes-aggregator/release_notes.html '''Release notes''']
+
* [https://docs.fd.io/honeycomb/1.18.04/honeycomb-aggregator/index.html '''Documentation''']:
+
==== 18.01 ====
+
* '''[[Honeycomb/Release_Plans/Release_Plan_18.01| Release plan]]''':
+
* [https://docs.fd.io/honeycomb/1.18.01/release-notes-aggregator/release_notes.html '''Release notes''']
+
* [https://docs.fd.io/honeycomb/1.18.01/honeycomb-aggregator/index.html '''Documentation''']:
+
==== 17.10 ====
+
* '''[[Honeycomb/Release_Plans/Release_Plan_17.10| Release plan]]''':
+
* [https://docs.fd.io/honeycomb/1.17.10/release-notes-aggregator/release_notes.html '''Release notes''']
+
* [https://docs.fd.io/honeycomb/1.17.10/honeycomb-aggregator/index.html '''Documentation''']:
+
==== 17.07 ====
+
* '''[[Honeycomb/Release_Plans/Release_Plan_17.07| Release plan]]''':
+
* [https://docs.fd.io/honeycomb/1.17.07/release-notes-aggregator/release_notes.html '''Release notes''']
+
* [https://docs.fd.io/honeycomb/1.17.07/honeycomb-aggregator/index.html '''Documentation''']:
+
==== 17.04 ====
+
* '''[[Honeycomb/Release_Plans/Release_Plan_17.04| Release plan]]''':
+
* [https://docs.fd.io/honeycomb/1.17.04/release-notes-aggregator/release_notes.html '''Release notes''']
+
* [https://docs.fd.io/honeycomb/1.17.04/honeycomb-aggregator/index.html '''Documentation''']:
+
==== 17.01 ====
+
* '''[[Honeycomb/Release_Plans/Release_Plan_17.01| Release plan]]''':
+
* [https://docs.fd.io/honeycomb/1.17.01/release-notes-aggregator/release_notes.html '''Release notes''']
+
* [https://docs.fd.io/honeycomb/1.17.01/honeycomb-aggregator/index.html '''Documentation''']:
+
==== 16.09 ====
+
<b>[[Honeycomb/Releases/1609| Honeycomb 1609 release page]]</b>
+
  
== Honeycomb backlog ==
+
=== Requirements ===
Backlog can be found in: [https://jira.fd.io/secure/Dashboard.jspa?selectPageId=10200 Honeycomb's JIRA].
+
  
== Code quality ==
+
* [[Sweetcomb/RPCop | RPC operations]], what data I am reading, writing
Current sonar analysis can be found at: [https://sonar.fd.io/overview?id=1107 Honeycomb's sonar]
+
* [[Sweetcomb/FaultTolerance | Fault Tolerance]], what happens if sweetcomb dies, or VPP dies
  
== Devel ==
+
=== Implement your own module ===
  
[[Honeycomb/BranchCutting|HowTo: Cutting stable branches]]
+
* [[Sweetcomb/NewModel | New model]], how to support a new YANG model
 +
* [[Sweetcomb/Sysrepo | Notes about sysrepo ]], sysrepo part of a new YANG model
  
[[Honeycomb/BumpingVersions|HowTo: Bumping versions]]
+
=== gNMI server ===
  
=== Design and architecture ===
+
* [[Sweetcomb/gNMIServer | gNMI Server]], how to use gNMI server
  
<span style="color:red">// TODO These all should be part of the code in e.g. adoc format, built during CI, deployed and just linked here</span>
+
=== VOM ===
  
[[VPP/Java_API|JVpp Java facade for VPP]]
+
* [[Sweetcomb/VOM | VOM as a middleware]], Everything related to our attempt to use [[VPP/VOM | VOM ]] rather than scvpp.
  
[[Honeycomb/Translation_Layer|Honeycomb Translation layer/SPI]]
+
== FAQ ==
  
[[Honeycomb/Persistence|Honeycomb persistence/Reconciliation with VPP]]
+
[https://wiki.fd.io/view/Sweetcomb/FAQ FAQ]
  
[[Honeycomb/VPPJAPI workflow|Modifying the VPP JVPP API and testing in Honeycomb]]
+
== Meeting ==
  
[[Honeycomb/IETF-ACL model|IETF_ACL model implementation]]
+
[https://wiki.fd.io/view/Sweetcomb/Meeting Sweetcomb meeting]
 +
 
 +
== Sweetcomb backlog ==
 +
Backlog can be found in: [https://jira.fd.io/projects/SWEET/ sweetcomb's JIRA].
 +
 
 +
== Code quality ==
 +
There is no current sonar analysis on: [https://sonar.fd.io https://sonar.fd.io]

Latest revision as of 21:30, 3 December 2020

Sweetcomb Facts

Project Lead: Hongjun Ni, @ Intel
Committers:

Repository: git clone https://gerrit.fd.io/r/sweetcomb
Mailing List: sweetcomb-dev@lists.fd.io
Jenkins: jenkins silo
Gerrit Patches: code patches/reviews
Bugs: sweetcomb bugs

Intro

Sweetcomb is a management agent written in C that runs on the same host as a VPP instance, and exposes yang models via NETCONF, RESTCONF and gNMI to allow the management of VPP instances.

Sweetcomb work as a plugin (ELF shared library) for sysrepo datastore.

Overview

Sweetcomb Architecture and environment

Project Termination

Sweetcomb/Termination Review


Draw.io source here

Project Contact

Scope

Sweetcomb's main responsibility is to enable communication between northbound interfaces and VPP's management APIs, performing all necessary translations in the background. It is important to note that many features and utilities will be reused from open source projects and tools (e.g. netopeer2, Sysrepo) and will not be a direct part of Sweetcomb. This section is splitted into 2 sections: in-scope and out-of-scope to clearly define what is developed as part of Sweetcomb project and what will be just reused from other projects (or where Sweetcomb relies on other projects).

Sweetcomb project scope:

  • Translation layer between VPP management and Yang based data structures
    • Must support all features of VPP exposed in its APIs in an extensible manner
  • Write YANG models for VPP specific features
  • Expose APIs to integrate with other open source projects
    • Base implementation of all generic southbound interfaces leverage VPP-VAPI
    • expose APIs to integrate with SD-WAN control plane, such as SDN Controller.
    • expose APIs to integrate with Routing Daemon, such as FRR.
    • expose APIs to integrate with IKE protocol, such as strongswan.
    • expose APIs to integrate with DPI control plane, such as nDPI.
    • expose APIs to integrate with BRAS control plane, such as OpenBRAS.

Out of scope:

  • VPP API (VAPI or VOM)
    • VAPI: C APIs for VPP, allowing C-based applications to interact with VPP is out of scope of Sweetcomb project and is part of the base VPP project.
    • VOM: C++ based API offering storage of networking objects
  • Writing yang models for VPP management
    • IETF and Openconfig YANG models are used as much as possible
    • For missing features, we rely on Honeycomb YANG models
  • Compliancy with YANG models, storage of YANG tree
    • Provided by sysrepo project and libraries it uses
  • Integration/performance testing
    • Complex integration or performance tests are out of scope of Sweetcomb project and are part of CSIT project
  • Any other application based on top of Sweetcomb is out of scope of this project and needs to be hosted in a dedicated project inside or outside of fd.io

Releases

Sweetcomb releases are based on VPP version numbers as sweetcomb uses VPP API which is not backward compatible.

Contributing

Contributions must go through code-review before being merged:

   git clone https://gerrit.fd.io/r/sweetcomb


Feel free to subscribe to the following mailing lists:

Devel

Source code is divided in two directories:

  • scvpp: library source code to communicate with VAPI (VPP API).
  • plugins: sysrepo plugin library source code implementing xpath callbacks

Requirements

Implement your own module

gNMI server

VOM

FAQ

FAQ

Meeting

Sweetcomb meeting

Sweetcomb backlog

Backlog can be found in: sweetcomb's JIRA.

Code quality

There is no current sonar analysis on: https://sonar.fd.io