Difference between revisions of "CSIT/AArch64"

From fd.io
Jump to: navigation, search
(Functional test)
(Functional test)
Line 47: Line 47:
 
* 3-Nodes topology
 
* 3-Nodes topology
 
* On VM only
 
* On VM only
* only show failing tests
+
* honeycomb disabled
 +
* vm_host disabled
  
''No guarantee that the tests are failing because of arm.''
+
''No guarantee that the tests are failing because of arm or a bad setup of mine.''
  
 
<pre>
 
<pre>
==============================================================================
+
# path_to_testfile:PASS/FAIL
Func.Crypto.Default                                                         
+
interfaces_api-crud-tap-func.out:1/0
==============================================================================
+
interfaces_eth2p-eth-l2bdbasemaclrn-eth-2tap-func.out:1/0
Func.Crypto.Default.Eth2P-Ethip4Ipsectnl-Ip4Base-Func :: *IPv4 IPsec tunnel...
+
ip4_eth2p-dot1q-ip4base-func.out:3/0/3
==============================================================================
+
ip4_eth2p-ethip4-ip4base-ip4proxyarp-func.out:2/1
...
+
ip4_eth2p-ethip4-ip4base-ipolicemarkbase-func.out:3/1
------------------------------------------------------------------------------
+
ip4_eth2p-ethip4-ip4base-rpf-func.out:1/7
TC13: VPP process ESP packet in Tunnel Mode with AES-CBC-128 encry... | FAIL |
+
ip4_eth2p-ethip4-ip4basevrf-func.out:3/5
Expected error 'ESP packet Rx timeout' but got 'Traffic script execution failed'.
+
ip4_tunnels_gre_eth2p-ethip4gre-ip4base-func.out:1/0
------------------------------------------------------------------------------
+
ip4_tunnels_lisp_api-crud-lisp-func.out:5/0
TC14: VPP process ESP packet in Tunnel Mode with AES-CBC-128 encry... | FAIL |
+
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func.out:4/0
Expected error 'ESP packet Rx timeout' but got 'Traffic script execution failed'.
+
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremape-func.out:8/1
------------------------------------------------------------------------------
+
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func.out:1/0
TC15: VPP process ESP packet in Tunnel Mode with AES-CBC-128 encry... | FAIL |
+
ip4_tunnels_vxlan_eth2p-dot1qip4vxlan-l2bdbasemaclrn-func.out:1/0
Expected error 'ESP packet Rx timeout' but got 'Traffic script execution failed'.
+
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2bdbasemaclrn-func.out:1/0
------------------------------------------------------------------------------
+
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2xcbase-func.out:1/0
TC16: VPP process ESP packet in Tunnel Mode with AES-CBC-128 encry... | FAIL |
+
ip6_eth2p-ethip6-ip6base-func.out:4/3
Traffic script execution failed
+
ip6_eth2p-ethip6-ip6base-ip6ra-func.out:1/3
------------------------------------------------------------------------------
+
ip6_eth2p-ethip6-ip6base-ipolicemarkbase-func.out:4/0
TC17: VPP process ESP packet in Tunnel Mode with AES-CBC-128 encry... | FAIL |
+
ip6_eth2p-ethip6-ip6basevrf-func.out:5/5
Expected error 'ESP packet Rx timeout' but got 'Traffic script execution failed'.
+
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6base-func.out:1/0
------------------------------------------------------------------------------
+
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6basevrf-func.out:1/0
TC18: VPP process ESP packet in Tunnel Mode with AES-CBC-128 encry... | FAIL |
+
l2bd_eth2p-dot1q--dot1ad-l2bdbasemaclrn-vlantrans12-func.out:1/7
Expected error 'ESP packet Rx timeout' but got 'Traffic script execution failed'.
+
l2bd_eth2p-dot1q-l2bdbasemaclrn-vlantrans11-func.out:2/2
------------------------------------------------------------------------------
+
l2bd_eth2p-eth-l2bdbasemaclrn-func.out:3/0
TC19: VPP process ESP packet in Tunnel Mode with AES-CBC-128 encry... | FAIL |
+
l2bd_eth2p-eth-l2bdbasemacstc-func.out:1/0
Expected error 'ESP packet Rx timeout' but got 'Traffic script execution failed'.
+
l2xc_eth2p-dot1ad-l2xcbase-func.out:1/0
------------------------------------------------------------------------------
+
l2xc_eth2p-dot1q--dot1ad-l2xcbase-vlantrans12-func.out:2/6
Func.Crypto.Default.Eth2P-Ethip4Ipsectnl-Ip4Base-Func :: *IPv4 IPs... | FAIL |
+
l2xc_eth2p-eth-l2xcbase-func.out:2/0
19 critical tests, 12 passed, 7 failed
+
telemetry_eth2p-ethip4-ip4base-spanrx-func.out:2/0
19 tests total, 12 passed, 7 failed
+
telemetry_eth2p-ethip6-ip6base-spanrx-func.out:1/0
==============================================================================
+
Func.crypto.sw_default
+
==============================================================================
+
2 critical tests, 0 passed, 2 failed
+
2 tests total, 0 passed, 2 failed
+
==============================================================================
+
Func.Honeycomb
+
... Honeycomb not set up ...
+
All tests fail.
+
==============================================================================
+
Func.Interfaces                                                      | PASS |
+
6 critical tests, 6 passed, 0 failed
+
6 tests total, 6 passed, 0 failed
+
==============================================================================
+
Func.Ip4                                                                     
+
==============================================================================
+
Func.Ip4.Eth2P-Dot1Q-Ip4Base-Func :: *IPv4 with VLAN subinterfaces*         
+
3 critical tests, 3 passed, 0 failed
+
3 tests total, 3 passed, 0 failed
+
==============================================================================
+
Func.Ip4.Eth2P-Ethip4-Ip4Base-Copblklistbase-Func :: *COP Security IPv4 Bla...
+
==============================================================================
+
TC01: DUT drops IPv4 pkts with COP blacklist set with IPv4 src-add... | FAIL |
+
Expected error 'ICMP echo Rx timeout' but got 'Traffic script execution failed'.
+
------------------------------------------------------------------------------
+
Func.Ip4.Eth2P-Ethip4-Ip4Base-Copblklistbase-Func :: *COP Security... | FAIL |
+
1 critical test, 0 passed, 1 failed
+
1 test total, 0 passed, 1 failed
+
==============================================================================
+
Func.Ip4.Eth2P-Ethip4-Ip4Base-Copwhlistbase-Func :: *COP Security IPv4 Whit...
+
==============================================================================
+
TC01: DUT permits IPv4 pkts with COP whitelist set with IPv4 src-a... | PASS |
+
------------------------------------------------------------------------------
+
Func.Ip4.Eth2P-Ethip4-Ip4Base-Copwhlistbase-Func :: *COP Security ... | PASS |
+
1 critical test, 1 passed, 0 failed
+
1 test total, 1 passed, 0 failed
+
==============================================================================
+
Func.Ip4.Eth2P-Ethip4-Ip4Base-Func :: *IPv4 routing test cases*             
+
==============================================================================
+
TC01: DUT replies to ICMPv4 Echo Req to its ingress interface :: M... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
TC02: DUT routes IPv4 to its egress interface :: Make TG send ICMP... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
TC03: DUT1 routes IPv4 to DUT2 ingress interface :: Make TG send I... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
TC04: DUT1 routes IPv4 to DUT2 egress interface :: Make TG send IC... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
TC05: DUT1 and DUT2 route IPv4 between TG interfaces :: Make TG se... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
TC06: DUT replies to ICMPv4 Echo Reqs with size 64B-to-1500B-incr-... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
TC07: DUT replies to ICMPv4 Echo Reqs with size 1500B-to-9000B-inc... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
TC08: DUT replies to ARP request :: Make TG send ARP Request to DU... | FAIL |
+
Parent suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
------------------------------------------------------------------------------
+
Func.Ip4.Eth2P-Ethip4-Ip4Base-Func :: *IPv4 routing test cases*      | FAIL |
+
Suite setup failed:
+
TypeError: 'NoneType' object is not iterable
+
==============================================================================
+
 
+
8 critical tests, 0 passed, 8 failed
+
8 tests total, 0 passed, 8 failed
+
 
+
 
</pre>
 
</pre>
  

Revision as of 09:18, 19 January 2018

CSIT TOI

Based on CSIT committers availability, current proposal is to hold two separate 1hr sessions:

   TOI-1: Tue 23-Jan 07:00-08:00 PST / 15:00-16:00 UTC / 16:00-17:00 CET 
   TOI-1: Fri 31-Jan 07:00-08:00 PST / 15:00-16:00 UTC / 16:00-17:00 CET

Recent Patches

add new topology parameter: arch Merged 2018-01-10 https://gerrit.fd.io/r/#/c/9474/
update nodes dependency Merged 2018-01-02 https://gerrit.fd.io/r/#/c/9584/
update vagrant to use ubuntu 16.04 images https://gerrit.fd.io/r/#/c/8295/
warn against virtualenv --system-site-packages Merged 2017-12-20 https://gerrit.fd.io/r/#/c/9428/

TEST for 2-nodes topology

(gg) based on my tests, in order to reproduce issues more easily across the team.

  • set up 2 aarch64 VMs (TG + DUT)
    • give them 3 interfaces
    • one for management, the other two down for the test
    • compile igb_uio (see known issue #1) and load it
  • write topology file with `arch: aarch64`
    • fill pci_address, and mac_address according to your setup
  • create virtualenv following csit README
    • virtualenv env
    • source env/bin/activate
    • pip install -r requirements.txt
    • export PATH with vpp_api_test, and PYTHONPATH

cmdline to run functional tests only with debug logs. You need the "__init__.robot" present in the folder you give to pybot.

pybot --debugfile /tmp/debugfile --exitonerror -L TRACE -v TOPOLOGY_PATH:topologies/enabled/topology.yaml tests/vpp/func/

Functional test

Tests status:

  • CSIT: 54ad6efd342695d0a7dad5380cc989a8d846f518
  • 3-Nodes topology
  • On VM only
  • honeycomb disabled
  • vm_host disabled

No guarantee that the tests are failing because of arm or a bad setup of mine.

# path_to_testfile:PASS/FAIL
interfaces_api-crud-tap-func.out:1/0
interfaces_eth2p-eth-l2bdbasemaclrn-eth-2tap-func.out:1/0
ip4_eth2p-dot1q-ip4base-func.out:3/0/3
ip4_eth2p-ethip4-ip4base-ip4proxyarp-func.out:2/1
ip4_eth2p-ethip4-ip4base-ipolicemarkbase-func.out:3/1
ip4_eth2p-ethip4-ip4base-rpf-func.out:1/7
ip4_eth2p-ethip4-ip4basevrf-func.out:3/5
ip4_tunnels_gre_eth2p-ethip4gre-ip4base-func.out:1/0
ip4_tunnels_lisp_api-crud-lisp-func.out:5/0
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func.out:4/0
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremape-func.out:8/1
ip4_tunnels_softwire_eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func.out:1/0
ip4_tunnels_vxlan_eth2p-dot1qip4vxlan-l2bdbasemaclrn-func.out:1/0
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2bdbasemaclrn-func.out:1/0
ip4_tunnels_vxlan_eth2p-ethip4vxlan-l2xcbase-func.out:1/0
ip6_eth2p-ethip6-ip6base-func.out:4/3
ip6_eth2p-ethip6-ip6base-ip6ra-func.out:1/3
ip6_eth2p-ethip6-ip6base-ipolicemarkbase-func.out:4/0
ip6_eth2p-ethip6-ip6basevrf-func.out:5/5
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6base-func.out:1/0
ip6_tunnels_lisp_eth2p-ethip6lispgpe-ip6basevrf-func.out:1/0
l2bd_eth2p-dot1q--dot1ad-l2bdbasemaclrn-vlantrans12-func.out:1/7
l2bd_eth2p-dot1q-l2bdbasemaclrn-vlantrans11-func.out:2/2
l2bd_eth2p-eth-l2bdbasemaclrn-func.out:3/0
l2bd_eth2p-eth-l2bdbasemacstc-func.out:1/0
l2xc_eth2p-dot1ad-l2xcbase-func.out:1/0
l2xc_eth2p-dot1q--dot1ad-l2xcbase-vlantrans12-func.out:2/6
l2xc_eth2p-eth-l2xcbase-func.out:2/0
telemetry_eth2p-ethip4-ip4base-spanrx-func.out:2/0
telemetry_eth2p-ethip6-ip6base-spanrx-func.out:1/0

TODO

  • identify ARM64 hardware to replicate CSIT repo
  • make Jira EPIC for CSIT func
  • make Jira EPIC for CSIT performance


  • disk-image-builder scripts
  • write a bootstrap-* script for aarch64
  • add VPP_REPO_URL* VPP_STABLE_VER* files ... (could be any public server for now)

known issues

  1. dpdk does not compiles igb_uio on aarch64 (it seems to require a kernel patch introduced in kernel 4.12: f719582435afe9c7985206e42d804ea6aa315d33). it has been re-enabled in dpdk v17.11 (f1810113590373b157ebba555d6b51f38c8ca10f)