Difference between revisions of "CSIT/csit2001 plan"

i) Boost tests in 2-Node setups, complete 3-Node setups;

ii) Complete Memif/Container and Vhost-user/VM with latest QEMU;

iii) Push vpp-dev to Ubuntu 18.04.

i) Scaled-out Vhost-user/VM and Memif/Container tests; ii) Test the same packet paths and NF topologies: service chains, service pipelines;

iii) See if we can isolate the actual cost of Vhostuser-virtio and Memif-Memif virtual interfaces based on the test and system telemetry.

iv) Test with VM and Containers running on a single Processor (single socket) with no core oversubscription and with.

v) Extend the test over two Processors to quantify impact of UPI latency (and bandwidth).

i) Improve detection accuracy and precision;

ii) Nail down current results variance;

iii) Apply improvements to continuous trending and (future) git auto-bisection.

i) Currently postponed, as the algorithm detects performance changes not related to VPP code.

ii) We need heavy workarounds or way more predictable SUT behavior.

iii) Denverton is more precise, but we need to avoid TRex duration stretching first.

i) Compare MLRsearch with PLRsearch soak test results.

i) Make sure no job gives -1 on correct commit.

ii) Enable voting on vpp-csit-devicetest job.

iii) Specify how CSIT should properly react to VPP code moving between plugins.

i) Productize VPP_Device container-based functional tests in 1-Node Skylake testbeds, assist with the same for Arm;

ii) Add proper packet latency measurements with T-Rex HDRhistogram, push T-Rex to productize HDRh'gram;

iii) Start using the new VPP stats infra for per test counters and "gauges" collection incl. "show runtime", instead of VPP show CLI;

iv) Start migration from VAT to VPP Python API;

v) Nail down "broken"/not-performing VPP data plane feature arcs (incl. multi-threading) indicated by CSIT-18.10 results data.