Difference between revisions of "CSIT/AArch64"

From fd.io
Jump to: navigation, search
(→‎CSIT TOI)
(→‎Functional test)
Line 43: Line 43:
 
* 3-Nodes topology
 
* 3-Nodes topology
 
* On VM only
 
* On VM only
−
* honeycomb disabled
+
* honeycomb '''disabled'''
 +
* nested VM '''disabled'''
  
−
''No guarantee that the tests are failing because of arm or a bad setup of mine ... 145/199 of the test fail are caused by "Traffic script execution failed"''
+
'' There are a total of 348 Functional tests''
  
−
'' There are 348 Functional tests''
+
'' list format is path_to_testfile:PASS/FAIL ''
  
 +
=== PASS ===
 
<pre>
 
<pre>
−
# path_to_testfile:PASS/FAIL
 
 
crypto_default_eth2p-ethip4ipsectnl-ip4base-func.out:19/0
 
crypto_default_eth2p-ethip4ipsectnl-ip4base-func.out:19/0
−
crypto_default_eth2p-ethip4ipsectpt-ip4base-func.out:18/1
 
−
crypto_default_eth2p-ethip4ipsectptlispgpe-ip4base-func.out:0/4
 
−
crypto_default_eth2p-ethip4ipsectptlispgpe-ip6base-func.out:0/2
 
−
crypto_default_eth2p-ethip4ipsectptlispgpe-ip6basevrf-func.out:0/1
 
−
crypto_default_eth2p-ethip6ipsectnl-ip6base-func.out:12/7
 
−
crypto_default_eth2p-ethip6ipsectpt-ip6base-func.out:12/7
 
−
crypto_default_eth2p-ethip6ipsectptlispgpe-ip4base-func.out:1/1
 
−
crypto_default_eth2p-ethip6ipsectptlispgpe-ip6base-func.out:1/1
 
−
crypto_sw_device_eth2p-ethip4ipsectnlsw-ip4base-func.out:0/0
 
−
crypto_sw_device_eth2p-ethip4ipsectptsw-ip4base-func.out:0/0
 
 
interfaces_api-crud-tap-func.out:1/0
 
interfaces_api-crud-tap-func.out:1/0
 
interfaces_eth2p-ethip4-ip4base-eth-1tap-func.out:2/0
 
interfaces_eth2p-ethip4-ip4base-eth-1tap-func.out:2/0
Line 68: Line 59:
 
ip4_eth2p-dot1q-ip4base-func.out:3/0
 
ip4_eth2p-dot1q-ip4base-func.out:3/0
 
ip4_eth2p-ethip4-ip4base-copblklistbase-func.out:1/0
 
ip4_eth2p-ethip4-ip4base-copblklistbase-func.out:1/0
−
ip4_eth2p-ethip4-ip4base-copwhlistbase-func.out:0/1
+
ip4_eth2p-ethip4-ip4base-copwhlistbase-func.out:1/0
−
ip4_eth2p-ethip4-ip4base-func.out:4/4
+
ip4_eth2p-ethip4-ip4base-func.out:0/0
−
ip4_eth2p-ethip4-ip4base-iaclbase-func.out:0/11
+
ip4_eth2p-ethip4-ip4base-ip4arp-func.out:2/0
−
ip4_eth2p-ethip4-ip4base-ip4arp-func.out:0/2
+
−
ip4_eth2p-ethip4-ip4base-ip4dhcpclient-func.out:0/5
+
−
ip4_eth2p-ethip4-ip4base-ip4dhcpproxy-func.out:0/2
+
−
ip4_eth2p-ethip4-ip4base-ip4ecmp-func.out:0/1
+
 
ip4_eth2p-ethip4-ip4base-ip4proxyarp-func.out:5/0
 
ip4_eth2p-ethip4-ip4base-ip4proxyarp-func.out:5/0
 
ip4_eth2p-ethip4-ip4base-ipolicemarkbase-func.out:4/0
 
ip4_eth2p-ethip4-ip4base-ipolicemarkbase-func.out:4/0
−
ip4_eth2p-ethip4-ip4base-rpf-func.out:2/2
+
ip4_eth2p-ethip4-ip4base-rpf-func.out:2/0
−
ip4_eth2p-ethip4-ip4basevrf-func.out:5/5
+
−
ip4_tunnels_gre_eth2p-ethip4gre-ip4base-func.out:1/5
+
 
ip4_tunnels_lisp_api-crud-lisp-func.out:5/0
 
ip4_tunnels_lisp_api-crud-lisp-func.out:5/0
 
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip4base-func.out:1/0
 
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip4base-func.out:1/0
 
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip4basevrf-func.out:1/0
 
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip4basevrf-func.out:1/0
−
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip6base-func.out:0/1
 
−
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip6basevrf-func.out:0/1
 
−
ip4_tunnels_lisp_eth2p-ethip4lisp-ip4base-func.out:0/1
 
−
ip4_tunnels_lisp_eth2p-ethip4lisp-l2bdbasemaclrn-func.out:0/1
 
 
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func.out:4/0
 
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func.out:4/0
−
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremape-func.out:8/1
 
 
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func.out:1/0
 
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func.out:1/0
 
ip4_tunnels_vxlan_eth2p-dot1qip4vxlan-l2bdbasemaclrn-func.out:1/0
 
ip4_tunnels_vxlan_eth2p-dot1qip4vxlan-l2bdbasemaclrn-func.out:1/0
 
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2bdbasemaclrn-func.out:1/0
 
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2bdbasemaclrn-func.out:1/0
 
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2xcbase-func.out:1/0
 
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2xcbase-func.out:1/0
−
ip4_tunnels_vxlan_eth4p-ethip4vxlan-l2bdbasemaclrn-l2shg-func.out:0/2
+
ip6_eth2p-ethip6-ip6base-copblklistbase-func.out:1/0
−
ip6_eth2p-ethip6-ip6base-copblklistbase-func.out:0/1
+
 
ip6_eth2p-ethip6-ip6base-copwhlistbase-func.out:1/0
 
ip6_eth2p-ethip6-ip6base-copwhlistbase-func.out:1/0
−
ip6_eth2p-ethip6-ip6base-func.out:4/3
+
ip6_eth2p-ethip6-ip6base-func.out:7/0
−
ip6_eth2p-ethip6-ip6base-iaclbase-func.out:5/7
+
ip6_eth2p-ethip6-ip6base-iaclbase-func.out:12/0
−
ip6_eth2p-ethip6-ip6base-ip6dhcpproxy-func.out:0/1
+
−
ip6_eth2p-ethip6-ip6base-ip6ecmp-func.out:0/1
+
−
ip6_eth2p-ethip6-ip6base-ip6ra-func.out:1/3
+
 
ip6_eth2p-ethip6-ip6base-ipolicemarkbase-func.out:4/0
 
ip6_eth2p-ethip6-ip6base-ipolicemarkbase-func.out:4/0
−
ip6_eth2p-ethip6-ip6basevrf-func.out:5/5
 
 
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip4base-func.out:1/0
 
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip4base-func.out:1/0
 
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6base-func.out:1/0
 
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6base-func.out:1/0
 
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6basevrf-func.out:1/0
 
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6basevrf-func.out:1/0
−
ip6_tunnels_lisp_eth2p-ethip6lisp-l2bdbasemaclrn-func.out:0/1
 
 
ip6_tunnels_vxlan_eth2p-ethip6vxlan-l2bdbasemaclrn-func.out:1/0
 
ip6_tunnels_vxlan_eth2p-ethip6vxlan-l2bdbasemaclrn-func.out:1/0
−
ip6_tunnels_vxlan_eth4p-ethip6vxlan-l2bdbasemaclrn-l2shg-func.out:1/1
+
l2bd_eth2p-dot1ad--dot1q-l2bdbasemaclrn-vlantrans21-func.out:4/0
−
l2bd_eth2p-dot1ad--dot1q-l2bdbasemaclrn-vlantrans21-func.out:0/4
+
l2bd_eth2p-dot1q--dot1ad-l2bdbasemaclrn-vlantrans12-func.out:8/0
−
l2bd_eth2p-dot1ad-l2bdbasemaclrn-vlantrans22-func.out:1/7
+
−
l2bd_eth2p-dot1q--dot1ad-l2bdbasemaclrn-vlantrans12-func.out:5/3
+
−
l2bd_eth2p-dot1q-l2bdbasemaclrn-vlantrans11-func.out:2/2
+
−
l2bd_eth2p-eth-l2bdbasemaclrn-func.out:3/0
+
 
l2bd_eth2p-eth-l2bdbasemacstc-func.out:1/0
 
l2bd_eth2p-eth-l2bdbasemacstc-func.out:1/0
−
l2bd_eth4p-eth-l2bdbasemaclrn-l2shg-func.out:0/1
 
−
l2xc_eth2p-dot1ad--dot1q-l2xcbase-vlantrans21-func.out:0/4
 
−
l2xc_eth2p-dot1ad-l2xcbase-func.out:1/0
 
−
l2xc_eth2p-dot1ad-l2xcbase-vlantrans22-func.out:0/8
 
−
l2xc_eth2p-dot1q--dot1ad-l2xcbase-vlantrans12-func.out:2/6
 
−
l2xc_eth2p-dot1q-l2xcbase-vlantrans11-func.out:0/4
 
 
l2xc_eth2p-eth-l2xcbase-func.out:2/0
 
l2xc_eth2p-eth-l2xcbase-func.out:2/0
−
l2xc_eth2p-eth-l2xcbase-iaclbase-func.out:0/4
+
telemetry_eth2p-ethip4-ip4base-spanrx-func.out:2/0
 +
telemetry_eth2p-ethip6-ip6base-spanrx-func.out:1/0
 +
</pre>
 +
 
 +
=== FAIL ===
 +
<pre>
 +
ip4_eth2p-ethip4-ip4base-func.out:0/0
 +
ip4_eth2p-ethip4-ip4base-ip4dhcpclient-func.out:0/5
 +
ip4_eth2p-ethip4-ip4base-ip4dhcpproxy-func.out:0/2
 +
ip4_eth2p-ethip4-ip4base-ip4ecmp-func.out:0/1
 +
ip6_eth2p-ethip6-ip6base-ip6dhcpproxy-func.out:0/1
 +
ip6_eth2p-ethip6-ip6base-ip6ecmp-func.out:0/1
 +
ip6_eth2p-ethip6-ip6base-ip6ra-func.out:0/4
 
telemetry_eth2p-ethip4-ip4base-ip4ipfixbase-func.out:0/5
 
telemetry_eth2p-ethip4-ip4base-ip4ipfixbase-func.out:0/5
 
telemetry_eth2p-ethip4-ip4base-ip4ipfixscale-func.out:0/2
 
telemetry_eth2p-ethip4-ip4base-ip4ipfixscale-func.out:0/2
−
telemetry_eth2p-ethip4-ip4base-spanrx-func.out:2/0
 
−
telemetry_eth2p-ethip6-ip6base-ip6ipfixbase-func.out:0/4
 
 
telemetry_eth2p-ethip6-ip6base-ip6ipfixscale-func.out:0/2
 
telemetry_eth2p-ethip6-ip6base-ip6ipfixscale-func.out:0/2
−
telemetry_eth2p-ethip6-ip6base-spanrx-func.out:1/0
+
</pre>
−
vm_vhost_ip4_eth2p-ethip4-ip4base-eth-2vhost-1vm.out:0/1
+
 
−
vm_vhost_ip4_eth2p-ethip4ipsectptlispgpe-ip4base-eth-2vhost-1vm-func.out:0/2
+
=== OTHER ===
−
vm_vhost_ip4_eth2p-ethip4ipsectptlispgpe-ip6base-eth-2vhost-1vm-func.out:0/2
+
<pre>
−
vm_vhost_ip4_eth2p-ethip4lispgpe-ip4base-eth-2vhost-1vm-func.out:0/1
+
ip4_eth2p-ethip4-ip4base-iaclbase-func.out:10/1
−
vm_vhost_ip4_eth2p-ethip4lispgpe-ip4basevrf-eth-2vhost-1vm-func.out:0/1
+
ip4_eth2p-ethip4-ip4basevrf-func.out:4/6
−
vm_vhost_ip4_eth2p-ethip4lispgpe-ip6base-eth-2vhost-1vm-func.out:0/1
+
ip6_eth2p-ethip6-ip6basevrf-func.out:5/5
−
vm_vhost_ip6_eth2p-ethip6ipsectptlispgpe-ip4base-eth-2vhost-1vm-func.out:0/2
+
l2bd_eth2p-dot1ad-l2bdbasemaclrn-vlantrans22-func.out:7/1
−
vm_vhost_ip6_eth2p-ethip6ipsectptlispgpe-ip6base-eth-2vhost-1vm-func.out:0/2
+
telemetry_eth2p-ethip6-ip6base-ip6ipfixbase-func.out:2/2
−
vm_vhost_ip6_eth2p-ethip6lispgpe-ip6base-eth-2vhost-1vm-func.out:0/1
+
−
vm_vhost_ip6_eth2p-ethip6lispgpe-ip6basevrf-eth-2vhost-1vm-func.out:0/1
+
 
</pre>
 
</pre>
  

Revision as of 15:14, 19 February 2018

CSIT TOI

https://wiki.fd.io/view/CSIT/TOIs

Recent Patches

add new topology parameter: arch Merged 2018-01-10 https://gerrit.fd.io/r/#/c/9474/
update nodes dependency Merged 2018-01-02 https://gerrit.fd.io/r/#/c/9584/
update vagrant to use ubuntu 16.04 images https://gerrit.fd.io/r/#/c/8295/
warn against virtualenv --system-site-packages Merged 2017-12-20 https://gerrit.fd.io/r/#/c/9428/

TEST for 2-nodes topology

(gg) based on my tests, in order to reproduce issues more easily across the team.

  • set up 2 aarch64 VMs (TG + DUT)
    • give them 3 interfaces
    • one for management, the other two down for the test
    • compile igb_uio (see known issue #1) and load it
  • write topology file with `arch: aarch64`
    • fill pci_address, and mac_address according to your setup
  • create virtualenv following csit README
    • virtualenv env
    • source env/bin/activate
    • pip install -r requirements.txt
    • export PATH with vpp_api_test, and PYTHONPATH

cmdline to run functional tests only with debug logs. You need the "__init__.robot" present in the folder you give to pybot. 

pybot --debugfile /tmp/debugfile --exitonerror -L TRACE -v TOPOLOGY_PATH:topologies/enabled/topology.yaml tests/vpp/func/

Functional test

Tests status:

  • CSIT: 54ad6efd342695d0a7dad5380cc989a8d846f518
  • 3-Nodes topology
  • On VM only
  • honeycomb disabled
  • nested VM disabled

There are a total of 348 Functional tests

list format is path_to_testfile:PASS/FAIL

PASS

crypto_default_eth2p-ethip4ipsectnl-ip4base-func.out:19/0
interfaces_api-crud-tap-func.out:1/0
interfaces_eth2p-ethip4-ip4base-eth-1tap-func.out:2/0
interfaces_eth2p-eth-l2bdbasemaclrn-eth-2tap-func.out:1/0
interfaces_eth2p-eth-l2bdbasemaclrn-l2shg-eth-2tap-func.out:2/0
ip4_eth2p-dot1q-ip4base-func.out:3/0
ip4_eth2p-ethip4-ip4base-copblklistbase-func.out:1/0
ip4_eth2p-ethip4-ip4base-copwhlistbase-func.out:1/0
ip4_eth2p-ethip4-ip4base-func.out:0/0
ip4_eth2p-ethip4-ip4base-ip4arp-func.out:2/0
ip4_eth2p-ethip4-ip4base-ip4proxyarp-func.out:5/0
ip4_eth2p-ethip4-ip4base-ipolicemarkbase-func.out:4/0
ip4_eth2p-ethip4-ip4base-rpf-func.out:2/0
ip4_tunnels_lisp_api-crud-lisp-func.out:5/0
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip4base-func.out:1/0
ip4_tunnels_lisp_eth2p-ethip4lispgpe-ip4basevrf-func.out:1/0
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func.out:4/0
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func.out:1/0
ip4_tunnels_vxlan_eth2p-dot1qip4vxlan-l2bdbasemaclrn-func.out:1/0
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2bdbasemaclrn-func.out:1/0
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2xcbase-func.out:1/0
ip6_eth2p-ethip6-ip6base-copblklistbase-func.out:1/0
ip6_eth2p-ethip6-ip6base-copwhlistbase-func.out:1/0
ip6_eth2p-ethip6-ip6base-func.out:7/0
ip6_eth2p-ethip6-ip6base-iaclbase-func.out:12/0
ip6_eth2p-ethip6-ip6base-ipolicemarkbase-func.out:4/0
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip4base-func.out:1/0
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6base-func.out:1/0
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6basevrf-func.out:1/0
ip6_tunnels_vxlan_eth2p-ethip6vxlan-l2bdbasemaclrn-func.out:1/0
l2bd_eth2p-dot1ad--dot1q-l2bdbasemaclrn-vlantrans21-func.out:4/0
l2bd_eth2p-dot1q--dot1ad-l2bdbasemaclrn-vlantrans12-func.out:8/0
l2bd_eth2p-eth-l2bdbasemacstc-func.out:1/0
l2xc_eth2p-eth-l2xcbase-func.out:2/0
telemetry_eth2p-ethip4-ip4base-spanrx-func.out:2/0
telemetry_eth2p-ethip6-ip6base-spanrx-func.out:1/0

FAIL

ip4_eth2p-ethip4-ip4base-func.out:0/0
ip4_eth2p-ethip4-ip4base-ip4dhcpclient-func.out:0/5
ip4_eth2p-ethip4-ip4base-ip4dhcpproxy-func.out:0/2
ip4_eth2p-ethip4-ip4base-ip4ecmp-func.out:0/1
ip6_eth2p-ethip6-ip6base-ip6dhcpproxy-func.out:0/1
ip6_eth2p-ethip6-ip6base-ip6ecmp-func.out:0/1
ip6_eth2p-ethip6-ip6base-ip6ra-func.out:0/4
telemetry_eth2p-ethip4-ip4base-ip4ipfixbase-func.out:0/5
telemetry_eth2p-ethip4-ip4base-ip4ipfixscale-func.out:0/2
telemetry_eth2p-ethip6-ip6base-ip6ipfixscale-func.out:0/2

OTHER

ip4_eth2p-ethip4-ip4base-iaclbase-func.out:10/1
ip4_eth2p-ethip4-ip4basevrf-func.out:4/6
ip6_eth2p-ethip6-ip6basevrf-func.out:5/5
l2bd_eth2p-dot1ad-l2bdbasemaclrn-vlantrans22-func.out:7/1
telemetry_eth2p-ethip6-ip6base-ip6ipfixbase-func.out:2/2

TODO

  • identify ARM64 hardware to replicate CSIT repo
  • make Jira EPIC for CSIT func
  • make Jira EPIC for CSIT performance


  • disk-image-builder scripts
  • patch the bootstrap-* scripts to use on for aarch64
  • add VPP_REPO_URL* VPP_STABLE_VER* files ... (could be any public server for now)

known issues

  1. dpdk does not compiles igb_uio on aarch64 (it seems to require a kernel patch introduced in kernel 4.12: f719582435afe9c7985206e42d804ea6aa315d33). it has been re-enabled in dpdk v17.11 (f1810113590373b157ebba555d6b51f38c8ca10f)
Retrieved from "https://wiki.fd.io/index.php?title=CSIT/AArch64&oldid=6962"