CSIT/Honeycomb/HC performance

From fd.io
Jump to: navigation, search

Honeycomb Performance Testing

Performance tests for Honeycomb as a VPP agent.

NETCONF northbound performance test - read

The goal of this test is to measure the performance of an external NETCONF client performing read operations on VPP operational data using Honeycomb's NETCONF northbound server.

Scenario

1. VPP and Honeycomb are started on device

2. test script establishes session(s) with Honeycomb's netconf northbound server over localhost TCP

2. script sends netconf read requests over the established session(s)

3. script measures time since sending out the first request until the last response is received

4. performance is calculated in terms of requests processed per second


Request used is the same as what would be generated by ODL netconf southbound:

   <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="m-1">
   <get>
   <filter xmlns:ns0="urn:ietf:params:xml:ns:netconf:base:1.0" ns0:type="subtree">
   <vpp-state xmlns="urn:opendaylight:params:xml:ns:yang:vpp:management">
   <version/>
   </vpp-state>
   </filter>
   </get>
   </rpc>
   ]]>]]>

Results

Environment

Host configuration:

  • Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz, 18c, 512GB RAM)
  • BIOS settings:
    • Hyperthreading disabled.
    • SpeedStep disabled.
    • TurboBoost disabled.
    • Power Technology: Performance.
  • Ubuntu 16.04.1, kernel 4.4.0-42-generic

More details about the testbed on https://wiki.fd.io/view/CSIT/CSIT_LF_testbed

  • VPP 17.07-RC0:
    • default configuration
  • Honeycomb 1.17.07:
    • custom handler for vpp-management/version that invokes only show_version
    • optimizations - JVM parameters:
      • JIT compiler mode: server
      • Memory size min/max: 128m/512m
      • Metaspace size min/max: 128m/512m
      • NUMA optimization
      • Parallel garbage collector for NUMA
   java -server -Xms128m -Xmx512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=512m -XX:+UseNUMA -XX:+UseParallelGC

Measured numbers with single netconf thread

"netconf-netty-threads":1

# of clients Client type TCP performance Total reads
1 Synchronous 4475 reads/s 100K
2 Synchronous 4923 reads/s 100K
4 Synchronous 5024 reads/s 100K
8 Synchronous 5099 reads/s 100K
16 Synchronous 5217 reads/s 100K

Measured numbers with multiple netconf threads

"netconf-netty-threads":16

# of clients Client type TCP performance Total reads
1 Synchronous 4505 reads/s 100K
2 Synchronous 9345 reads/s 100K
4 Synchronous 18484 reads/s 100K
8 Synchronous 31210 reads/s 100K
16 Synchronous 42833 reads/s 100K

Data source: https://jenkins.fd.io/job/csit-hc2vpp-verify-perf/23/

How to reproduce

Work in progress