|
|
| Line 1: |
Line 1: |
| − | =DRAFT PROPOSAL: CSIT functional jobs re-organisation=
| |
| | | | |
| − | ==Background==
| |
| − |
| |
| − | CSIT functional job included too many test cases per VPP patch and took quite a long time to execute - vpp-csit-verify-virl-master job used to take 1 hour of execution. To address it, following actions have been taken:
| |
| − |
| |
| − | * identify test cases that can be skipped in case of vpp-csit-verify-virl-{branch} job execution:
| |
| − | ** tests covered by unittests in vpp (make test)
| |
| − | ** tests covering old/obsolete/unmaintained vpp code
| |
| − | ** tests covering features with limited use
| |
| − | ** scale tests / combined tests
| |
| − | * apply special tag to these test cases
| |
| − | * adapt corresponding bootstrap files to exclude these test from execution per VPP patch
| |
| − |
| |
| − | All test cases skipped per VPP patch are executed by csit weekly and semi-weekly jobs.
| |
| − |
| |
| − | ==tests covered by unittests==
| |
| − |
| |
| − | # dhcp
| |
| − | #* Eth2P-Ethip4-Ip4Base-Ip4Dhcpproxy-Func
| |
| − | #* Eth2P-Ethip6-Ip6Base-Ip6Dhcpproxy-Func
| |
| − | # gre
| |
| − | #* Eth2P-Ethip4Gre-Ip4Base-Func
| |
| − | # iacl
| |
| − | #* Eth2P-Eth-L2Xcbase-Iaclbase-Func
| |
| − | #* Eth2P-Ethip4-Ip4Base-Iaclbase-Func
| |
| − | # ip4
| |
| − | #* Eth2P-Ethip4-Ip4Base-Func
| |
| − | # ip6
| |
| − | #* Eth2P-Ethip6-Ip6Base-Func
| |
| − | # ipfix
| |
| − | #* Eth2P-Ethip4-Ip4Base-Ip4Ipfixscale-Func
| |
| − | # l2bd
| |
| − | #* Eth2P-Eth-L2Bdbasemaclrn-Func
| |
| − | #* Eth2P-Eth-L2Bdbasemacstc-Func
| |
| − | # l2xc
| |
| − | #* Eth2P-Eth-L2Xcbase-Func
| |
| − | # softwire
| |
| − | #* Eth2P-Ethip4--Ethip6Ip4-Ip4Base--Ip6Base-Swiremape-Func
| |
| − | # vrf
| |
| − | #* Eth2P-Ethip4-Ip4Basevrf-Func
| |
| − | #* Eth2P-Ethip6-Ip6Basevrf-Func
| |
| − |
| |
| − | ==tests covering old/obsolete/unmaintained vpp code==
| |
| − |
| |
| − | # ipsec
| |
| − | #* Eth2P-Ethip4Ipsectnl-Ip4Base-Func
| |
| − | #* Eth2P-Ethip4Ipsectpt-Ip4Base-Func
| |
| − | #* Eth2P-Ethip6Ipsectnl-Ip6Base-Func
| |
| − | #* Eth2P-Ethip6Ipsectpt-Ip6Base-Func
| |
| − | # ipsec+lisp
| |
| − | #* Eth2P-Ethip4Ipsectptlispgpe-Ip4Base-Eth-2Vhost-1Vm-Func
| |
| − | #* Eth2P-Ethip4Ipsectptlispgpe-Ip6Base-Eth-2Vhost-1Vm-Func
| |
| − | #* Eth2P-Ethip4Ipsectptlispgpe-Ip6Basevrf-Func
| |
| − | #* Eth2P-Ethip6Ipsectptlispgpe-Ip4Base-Eth-2Vhost-1Vm-Func
| |
| − | #* Eth2P-Ethip6Ipsectptlispgpe-Ip6Base-Eth-2Vhost-1Vm-Func
| |
| − | #* Eth2P-Ethip4Ipsectptlispgpe-Ip4Base-Func
| |
| − | #* Eth2P-Ethip4Ipsectptlispgpe-Ip6Base-Func
| |
| − | #* Eth2P-Ethip6Ipsectptlispgpe-Ip4Base-Func
| |
| − | #* Eth2P-Ethip6Ipsectptlispgpe-Ip6Base-Func
| |
| − |
| |
| − | ==tests covering features with limited use==
| |
| − |
| |
| − | # vlan
| |
| − | #* Eth2P-Dot1Ad--Dot1Q-L2Bdbasemaclrn-Vlantrans21-Func
| |
| − | #* Eth2P-Dot1Ad--Dot1Q-L2Xcbase-Vlantrans21-Func
| |
| − | #* Eth2P-Dot1Ad-L2Bdbasemaclrn-Vlantrans22-Func
| |
| − | #* Eth2P-Dot1Ad-L2Xcbase-Vlantrans22-Func
| |
| − | #* Eth2P-Dot1Q--Dot1Ad-L2Bdbasemaclrn-Vlantrans12-Func
| |
| − | #* Eth2P-Dot1Q--Dot1Ad-L2Xcbase-Vlantrans12-Func
| |
| − | #* Eth2P-Dot1Q-L2Bdbasemaclrn-Vlantrans11-Func
| |
| − | #* Eth2P-Dot1Q-L2Xcbase-Vlantrans11-Func
| |
| − |
| |
| − | ==scale tests / combined tests==
| |
| − |
| |
| − | # ipfix
| |
| − | #* Eth2P-Ethip6-Ip6Base-Ip6Dhcpproxy-Func
| |
| − | #* Eth2P-Ethip4-Ip4Base-Ip4Ipfixbase-Func - TC05
| |
| − | #* Eth2P-Ethip6-Ip6Base-Ip6Ipfixbase-Func - TC04
| |
| − |
| |
| − | ==test case tag==
| |
| − |
| |
| − | New tag SKIP_VPP_PATCH has been introduced to identify test cases to be excluded from execution in case of vpp-csit-verify job (i.e. per VPP patch) but not to be skipped in case of csit-vpp-verify job (i.e. per CSIT patch).
| |
| − |
| |
| − | ==bootstrap files==
| |
| − |
| |
| − | Bootstrap files have been adapted to exclude from execution test cases tagged with SKIP_PATCH tag only (CSIT patch) or test cases tagged with SKIP_PATCH or SKIP_VPP_PATCH tags (VPP patch).
| |
| − |
| |
| − | ==results==
| |
| − |
| |
| − | Overall execution time of vpp-csit-verify-virl-master job has been reduced to 40mins - it means 20 mins gain per VPP patch with current implementation.
| |
