Difference between revisions of "NSH SFC/Meeting"

Open: 1. NSH_SFC 17.07 released last week after feature test for both VxLAN-GPE and Eth transports. 2. NSH patch is merged to the OVS-DPDK mainline to hit OVS 2.8 release. Thanks Jan Scheurich (Ericsson) and Yi Yang(Intel) and several others for excellent collaboration to make it an acceptable solution in OVS community. We believe this could accelerate the industry to embrace NSH based SFC solution via open source community projects (e.g. OVS, VPP, OpenDaylight, OPNFV, ONAP). Yi Yang is working on the NSH patch for net-next now.

17.10 new features 1. NSH_SFC performance automation test enablement 2. NSH TTL field and manipulation introduced in recent NSH draft 3. Offload NSH traffic steering to NSH-aware NIC (stretch goal)

1) Reworked VxLAN-GPE module to support FIB 2.0 and performance optimization, patch merged.

2) Implemented and NSH over Ethernet feature, patch merged.

3) Measured the performance of SFF, Classifier, Inbound Proxy and Outbound Proxy using Ethernet transport, still WIP and SFF shows 60% performance improvement than VxLAN-GPE

4) Hold off adding Geneve feature to VPP due to no solid requirement

5) NSH_SFC CSIT

 5.1) Patch for the ci-mangement are merged
 5.2) NSH_SFC functional test patch https://gerrit.fd.io/r/#/c/6126/ in good shape now to be merged. 
 5.3) Trigger performance automation test manually by adding the “nsh_sfc perf weekly”" to the patch comment.

1) NSH_SFC presentation on FD.io mini-summit and Nirvana day at Openstack summit. 1.1) Slides uploaded at https://wiki.fd.io/view/File:NSH_SFC_Openstack_Fdio_Minisummit_widescreen_v3.pdf which contains the NSH_SFC internals as well performance data which clearly suggests that VxLAN-GPE impacts 1C1T performance, and vhost_user impacts end-to-end SFC throughput as it does not scale for supporting massive EW traffics even more cores are assigned.

2) NSH_SFC 17.07 feature 2.1)Initial patch for Geneve at https://gerrit.fd.io/r/#/c/6206/ submitted for review 2.2)Eth transport works on ingress in a PoC but needs to add support into the NSH_SFC to enable egress direction

1) NSH_SFC integration with VPP 17.04

  1.1) Fix two bugs for NSH_SFC 17.04-RC2

1.1.1) https://gerrit.fd.io/r/#/c/6358/ Remove unsed parameter

              from fib_table_entry_special_add().

1.1.2) https://gerrit.fd.io/r/#/c/6359/ Fix dual-loop issue of NSH-SNAT.

  1.2) A recent VPP bug fix of “MAC address check” cause the failure of NSH-aware SNAT feature in NSH_SFC plugin. Hongjun raised this to VPP community for a solution.

2) NSH_SFC 17.04 release  2.1) Will ask test team to perform functional testing using VPP 17.04 using NSH_SFC RC2, once the above incomparability issue is fixed  2.2) Target release date is April 28, 2017.

3) NSH_SFC 17.07 feature development plan  3.1) Support Geneve and Eth as NSH transports, for compliance with IETF approved next generation tunneling protocol as well as high performance transport respectively  3.2) Performance automation test integration to VNF, by leveraging Trex  3.3) Performance optimization of NSH_SFC for end-to-end throughput as well as reduce maximum latency.  3.4) Real NSH-aware VNF by enabling NSH_SFC plugin to pass per-packet metadata to real SF (stretch goal)

1) Open

HC API freeze tomorrow, Hongjun will submit a patch that contains a Yang model for new APIs supporting MD Type 2 before API freeze.

2) How to enable iOAM based on initial MD Type 2 support patch? MD Type 2 patches are merged, after tests cases of using iOAM initial patches from Prasad passed. Rely on Prasad to submit patches to enable iOAM over NSH, the handler for push/pop/swap/traces are already supported.

3) Status and plan for NSH-aware VNF by collocating Proxy and SF. Ingress packets after NSH and VxLAN de-capsulation could be processed by SNAT, after VPP fixed a issue. Now blocked by the packets cannot be routed back to the NSH_SFC plugin for re-encapsulation. Hongjun figured out a solution to use install a forwarding rule into the IPv4 lookup graph node to forward packets to a virtual port connecting to the NSH_SFC. Since there is no changes to the SNAT module so it could easily scale for other SFs in the VPP. Initial patches will be developed and submitted this week, and it should have no blocking issue to hit 17.04 release.

4) NSH_SFC performance data review and CSIT enabling plan CSIT team has measured performance of the NSH_SFC, the single core and core scaling performance are pretty good, but the maximum latency is unusually much bigger than VPP, WIP root-causing the issue.

1) Open

- OpenDaylight patches for enabling NSH_SFC based classifier has been merged, after reporting several HC issues which are fixed recently. Testing the existing ODL code for NSH_SFC based Proxy is ongoing. Basically, goal to enable ODL Carbon to support NSH_SFC based SFF, Classifier and Proxy is in good shape.

2) Status of 17.04 development

- Initial MD Type 2 support: v10 patch submitted, Prasad has comments to it and Hongjun is working on consolidating comments.

- NSH-aware VNF: no progress yet, will be working on it this week.

3) NHS_SFC 17.01 performance measurement

- Single core performance for NSH_SFC based SFF, Classifier and Proxy ongoing, Classifier performance is about 2x then SFF and Proxy due to only 1 encapsulation operation, and SFF and Proxy needs at least 2 de-capsulation/re-encapsulation.

- Core scaling testing does not show linear scaling due to test configuration problem, issue root caused and re-testing ongoing.

1) NSH_SFC 17.01 release and package for other Linux foundation projects (e.g. ODL and Open-O) to use

1.1) Two issues (missing next node and hton endian) founded during integration tests, so it is very important to enable CSIT automation

1.2) Package VPP and NSH_SFC for ODL and Open-O 1.2.1)Docker image for ODL SFC 104 1.2.2)KVM/Qemu image for Open-O Murcury release enabling EPA

2) NSH_SFC 17.04 feature discussion and status

2.1) Co-locate NSH_SFC and VPP in a same box, blocked on a issue on handling packets returned from SNAT node to NSH_SFC plugin, prefer solution is to connect them to a L2 bridge instead of SNAT directly transmit the packet to wire. Hongjun is working on this

2.2) Plan for the MD type 2 support and iOAM support based on discussion between Hongjun and Prasad

2.2.1) Generic TLV routines in NSH_SFC, initial patches to send out for review this week

2.2.2) iOAM plugin by Prasad to reuse those routines for NSH based iOAM implementation

2.2.3) Move TLV generic routines to VPP if needed (e.g. other graph nodes need to use those routines)

1) A small patch https://gerrit.fd.io/r/#/c/4745/ applied to NSH_SFC RC1, hence NSH_SFC RC2 tag is created. NSH_SFC RC2 integration test with VPP 17.01 and Honeycomb RC2 (https://gerrit.fd.io/r/#/c/4781/) is ongoing, blocked on environment configuration. Expect to release NSH_SFC 17.01 by Jan 27th , once integration test passed, and Honeycomb 17.01 depends on NSH_SFC 17.01.

2) Shared preliminary NSH_SFC performance using RC2 to cover SFF, Proxy and Classifier use cases by feeding small packets. See details at https://wiki.fd.io/view/NSH_SFC/Docs/Performance

a) The comprehensive performance report will be shared once CSIT team fully integrate it to CSIT automation test environment.

b) NSH Proxy (1core, 1thread) performance drops from 16.09’ 5.8Mpps to 4.25Mpps for 128B small packet, might due to decoupling of VxLAN-GPE and NSH, will be analyzing it further post 17.04 release after MD Type 2 is supported.

c) 2 to 3 dedicated cores could achieve 10G line rate for small packet for all NSH_SFC use cases, assuming performance scales linearly.

1) NSH_SFC build failure tracked by https://jira.fd.io/browse/NSHSFC-18, which caused by recent build infrastructure change. Hongjun is working with Damjan to resolve this, seems it only need to add the missed file to the file src/vpp-api/java/Makefile.am. Nobody knows how to publish NSH_SFC artifacts for NSH_SFC rc1.

AR: Danny to drop email to help desk to ask how to publish NSH_SFC artifacts.

2) 0.5 dedicated resource is working on including NSH_SFC to the FD.io CSIT tests to cover both feature and performance automation tests. Goals are:

  * Can configure the NSH_SFC to act as SFF/Classifier/Proxy via Postman
  * Enable a Jeinkins job that drives TREX to send traffics in different size to NSH_SFC and
  measure/report the NSH_SFC performance when it acts different roles.

Test plan is underdeveloped, and the initial version is supposedly to be ready by the end of this week. Target is that the CSIT infrastructure for NSH_SFC to hit 17.04 release. All the new patches to be submitted in the future will benefit from this automation performance tests, to understand the performance impact the patch will be introducing.

3) A separated meeting is schedule with Prasad to discuss the TLV support for NSH MD Type 2. Prasad and his team is doing the in-band IOAM for VxLAN-GPE at this point which enables a framework to handle TLV. Target is to ensure common TLV parser in VPP could be used by various VPP plugin for different purpose. Keith is going to raise this in the VPP weekly meeting.

1) 17.01 Release

1.a Patches for NSH Proxy and NSH Classifier have been merged to both NSH_SFC and HC, after performing unit test. Integration with ODL SFC ongoing.

1.b Get some request from FD.io STV team about what tests need to be performed for NSH_SFC 17.01 release, will ask them to do integration test below:

    * Develops some CSIT test cases to test the dataplane and VPP CLI of NSH SFF, 
      NSH-Proxy and NSH Classifier.
    * Integrates  Honeycomb, VPP and NSH_SFC to test the HC NB API of NSH SFF, NSH-Proxy 
      and NSH Classifier.

1.c Hongjun is working on creating a Docker image that includes VPP 17.01 and NSH_SFC 17.01 as well as virtio_user PMD (based on DPDK 16.11) to enable easy-to-deploy NSH SFC ingredients (Classifier, SFF, Proxy, etc). In the future, the ODL and NSH-aware VNFs (co-locating Proxy and SFs) will be packaged the same way to enable an all-in-one setup, which includes a Vagrant image within several Docker images to create a SFC setup that has all ingredients specified in RFC 7665.

2) 17.04 release discussion

Targeted new features for 17.04 release:

    * Co-locate Proxy with SF 
    * Basic MD Type 2 support, such as encap/decap NSH within MD Type 2

3) Planned new features for 17.07 release

    * Release NSH-aware SFs being able to consume context header embedded in the NSH via 
      per-packet metadata
    * Enable match-action for MD Type 2 NSH 

4) Open: P4 for VPP and NSH_SFC

Idea is to develop a "compiler" that dynamically generate graph nodes from P4 program' HLIR, each node on the P4' parse graph will be mapped to a graph node of VPP and the graph node is responsible for extracting header defined in P4 and determine next graph nodes. This way, supporting new protocols such as VxLAN-GPE and NSH will be much easier. In addition, a "compiler" for ODL will be introduced to convert P4 program to Yang model for NETCONF SB plugin and HC to consume, so ODL could control VPP based P4 switch via RPC. Note: there is no need to support Openflow and OVSDB for this case.

Will add this to JIRA and no resources are assigned for this story.

1) NSH Proxy and NSH Classifier those two features are most done. Relevant NSH_SFC and HC patches are mostly merged, only two patches related to Yang model under review without impact to APIs. Those two patches will be merged soon if no critical comments.

2) Will send out API freeze mail this Thursday as planned.

3) Reviewed the investigation from Hongjun about enabling per-packet metadata for NSH-aware VNF implementation. There are two options: 1) Add vlib_buf_opaque2_t (64 bytes fit in a cache line) to vlib_buf_t to hold the metadata. 2) Add Service Path, Service Index and a pointer (pointing to NSH context header stored in the NSH_SFC graph node) to vlib_buf_opaque_t data struct.

The option 2 is preferred due to option 1 might introduce negative performance impact due to more data to store and two memory copies are needed. Hongjun will continue investigate which NSH specific fields and how many bits required in vlib_buf_opaque_t data struct, rough review shows it still has unused space for NSH.

4) No meetings in coming two weeks for Christmas holiday.

1) NSH_SFC 1701 release plan updated on this wiki. The milestones will be one week post VPP' milestones.

2) NSH Proxy and NSH Classifier those two features are in good shape. Features implementation done after initial test, there is no dependency on VPP' API change, but it still needs to depend on Yang model change in HC to ensure they can be integrated with ODL SFC. HC API freeze is Dec 14th, and Hongjun is working to ensure the Yang model and API changes will be accepted before HC API freeze.

3) ODL SFC in Carbon release will support above two features, and the SFC 104 test will enable a working demo.

4) NSH-aware VNFs in current proposal could be treated as NSH Proxy collocates with SF, there are two issues problems raised by Joel need to address in the near future:

   a. We need SFs to be natively NSH-aware, meaning they can manipulate metadata inside the NSH. 
      Current implementation looks like NSH Proxy co-locate with SFs but SFs can only 
      see the original frames.
   b. How to handle NAT like or TCP Proxy like SFs after they change the headers 
      of the inner headers which are typically used as key to determine which 
      next graph node is?  The initial thought is to add some information to 
      per-packet metadata before packets go into the SF, and the metadata can 
      be used to determine if it needs to be re-encapsulated by NSH_SFC graph node. 

AR: Hongjun to investigate how to support NAT and TCP Proxy sort of SFs by leveraging per-packet metadata. Also, think about the framework and libraries to enable SF being able to manipulate NSH metadata.

1) Hongjun to consolidate review comments to his patch https://gerrit.fd.io/r/3944 for enabling VPP Proxy

2) Keith mentioned VPP has included initial TLV support which could be used for enabling NSH MD Type 2, but seems there are only a few use cases (e.g. IOAM support by Frank and NSH timestamping support) demanding this feature. Implementing MD Type 2 is already in JIRA but mark it a a low priority item to be enabled post 1701 release unless there is solid requirement.

3) Rotating meeting every other week proposal is rejected due to people prefer consistent meeting time and it is hard to pick up a suitable time slot for three GEOs: US, Europe and PRC. Currently, 4:00pm PST every Monday is selected for two major code contributors Keith Burn and Hongjun Ni to attend, and Danny will reschedule the new weekly Lync meeting serious and figure out the way to sync up with Hakan and Markus on a regular basis.

4) Danny needs to update NSH_SFC Wiki page to reflect his role as new PTL and ensure this wiki is active.

5) Danny to work out release schedule for 1701 release and add stories to JIRA, below three ones are already added: a) https://jira.fd.io/browse/NSHSFC-15 NSHSFC-15 Enable NSH Proxy feature to support integrating NSH-unware SFs into SFC b) https://jira.fd.io/browse/NSHSFC-16 NSHSFC-16 NSH Classifier enablement c) https://jira.fd.io/browse/NSHSFC-17 NSHSFC-17 Enable VPP to be NSH-aware

6) Not able to review NSH_Aware SFs high level design diagram, so Danny/Hongjun need to upload the diagram to Google doc for NSH_SFC community to review offline, and probably discuss it in next weekly meeting.

The following list of gerrit patches do the integration work:

(1).VPP:

https://gerrit.fd.io/r/#/c/2118/ jvpp support for nsh.

(2).Honeycomb:

https://gerrit.fd.io/r/#/c/2527/ HONEYCOMB-46: Add Nsh Map in Honeycomb

which depends on this patch:

https://gerrit.fd.io/r/#/c/2481/9 HONEYCOMB-46: Add Nsh Entry in Honeycomb

(3).ODL SFC:

https://git.opendaylight.org/gerrit/#/c/42699/ Add VPP Renderer

Actually, there are only two issues remained, but I think they won’t block the integration test.

(4).The building failure due to “mvn” commands are not supported in Jenkins.

       We have discussed this issue on weekly meeting, and Ed Warnicke will fix it.

(5). HC Plugin could not reuse the VxlanGpeCustomizer within the HC code base.

      I have sent an email to discuss this issue with Maros and Marek.

      The details are as follows:

In nsh-map Yang model, there is an augmentation:

 augment /vpp-nsh/nsh-maps/nsh-map {

   ext:augment-identifier vxlan-gpe-encap-augment;

   when "/encap-type = 'vpp-nsh:vxlan-gpe-encap-type' ";

   uses v3po:vxlan-gpe-base-attributes;

 }

How can I reuse the VxlanGpeCustomizer within the HC code base?

I tried to register VxlanGpeCustomizer into VppNshWriterFactory, but it failed.

• Minutes
• Minutes (text)
• Log
• ODL SFC + fd.io (HC + VPP) integration plan - diagram
• Webex Recording

Agenda:

• AIs from 6/6 meeting
• Gerrit patch review by committers
• Hongjun as committer

• Meeting ended Mon Jun 20 14:17:17 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
• Minutes: http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-06-20-13.58.html
• Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-06-20-13.58.txt
• Log: http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-06-20-13.58.log.html

Agenda:

• Patch 1124
  • Builds with both VPP-1606 and MASTER
  • Still rebuilding test rig - Instructions - VMs only
• Release discussion - most of these topics will have to be covered on mailer
  • Best time for longer meeting.
  • Alignment to VPP releases
  • Timing
  • Features
  • Repo structure
  • Packaging
• Jenkins
  • Need Jenkins verify job
• CSIT
  • See ONE, their repo is all CSIT testing

• Minutes:        http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-06-06-14.04.html
• Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-06-06-14.04.txt
• Log:            http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-06-06-14.04.log.html

Actions:

AI: Keith to document VM setup for testing traffic (SFF client and SF "proxy") Status: AI: Keith to schedule next TWS call (Jun13) for edwarnicke to do Jenkins TOI at TWS Status: Done AI: (sic CSIT) Keith to take this to the mailer looking for volunteers. Status:

• Meeting ended Mon May 23 14:37:05 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
• Minutes: http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-05-23-14.01.html
• Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-05-23-14.01.txt
• Log: http://ircbot.wl.linuxfoundation.org/meetings/fdio-nsh/2016/fdio-nsh.2016-05-23-14.01.log.html

• Meeting ended Mon May 16 14:30:51 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
• Minutes: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-05-16-14.01.html
• Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-05-16-14.01.txt
• Log: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-05-16-14.01.log.html

• Meeting ended Mon May 9 14:29:06 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
• Minutes: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-05-09-14.00.html
• Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-05-09-14.00.txt
• Log: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-05-09-14.00.log.html

• Meeting ended Mon Apr 25 14:43:21 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
• Minutes: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-04-25-13.58.html
• Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-04-25-13.58.txt
• Log: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-04-25-13.58.log.html

• Meeting ended Mon Apr 11 17:31:19 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
• Minutes: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-04-11-14.04.html
• Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-04-11-14.04.txt
• Log: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-04-11-14.04.log.html