Difference between revisions of "CSIT/FuncTestPlan"

From fd.io
Jump to: navigation, search
Line 44: Line 44:
 
Identified VPP working code availability dependencies are marked by VPP Jira ticket [TODO VPP-xxx].
 
Identified VPP working code availability dependencies are marked by VPP Jira ticket [TODO VPP-xxx].
  
−
==IPv4 data plane==
+
==IPv4 data plane (ip4dp)==
  
−
JIRA [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=39&quickFilter=37&quickFilter=64 board view], [https://jira.fd.io/issues/?filter=10631 list view] - '''ip4dp''' label for CSIT work tracking.
+
JIRA status for '''ip4dp''' tasks:
  
−
Indicative coverage = 50%
+
'''1609-p0'''
 +
 
 +
<jira jql='project = CSIT AND labels = ip4dp AND labels = func AND labels = 1609-p0 ORDER BY status DESC, priority DESC'></jira>
 +
 
 +
'''1609-p1'''
 +
 
 +
<jira jql='project = CSIT AND labels = ip4dp AND labels = func AND labels = 1609-p1 ORDER BY status DESC, priority DESC'></jira>
 +
 
 +
'''1609-p2'''
 +
 
 +
<jira jql='project = CSIT AND labels = ip4dp AND labels = func AND labels = 1609-p2 ORDER BY status DESC, priority DESC'></jira>
 +
 
 +
Original CSIT plan - need to update per JIRA ([https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=39&quickFilter=37&quickFilter=64 board view], [https://jira.fd.io/issues/?filter=10631 list view]):
  
 
# [1609-P1] qos - ingress policing - 1r2c-(cb,ca), 2r3c-(cb,ca) policers, RFC2475, RFC2697, RFC2698. [https://jira.fd.io/browse/CSIT-21 CSIT-21]
 
# [1609-P1] qos - ingress policing - 1r2c-(cb,ca), 2r3c-(cb,ca) policers, RFC2475, RFC2697, RFC2698. [https://jira.fd.io/browse/CSIT-21 CSIT-21]
Line 58: Line 70:
 
# [1606-DONE] security - iacl with n-tuple classification, ip4, ip6, mac.
 
# [1606-DONE] security - iacl with n-tuple classification, ip4, ip6, mac.
 
# [1606-DONE] security - cop white-/black-list ip4 src addr filtering.
 
# [1606-DONE] security - cop white-/black-list ip4 src addr filtering.
−
# [1609-DONE] VPN routed forwarding (VRF) - baseline. [https://jira.fd.io/browse/CSIT-183 CSIT-183]  
+
# [1609-DONE] VPN routed forwarding (VRF) - baseline. [https://jira.fd.io/browse/CSIT-183 CSIT-183]
 
# [1609-DONE] security - source IPv4 Reverse Path Forwarding (RPF). [https://jira.fd.io/browse/CSIT-188 CSIT-188]
 
# [1609-DONE] security - source IPv4 Reverse Path Forwarding (RPF). [https://jira.fd.io/browse/CSIT-188 CSIT-188]
  
−
==IPv4 control plane==
+
==IPv4 control plane (ip4cp)==
  
−
JIRA [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=39&quickFilter=64&quickFilter=35 board view], [https://jira.fd.io/issues/?filter=10632 list view] - '''ip4cp''' label for CSIT work tracking.
+
JIRA status for '''ip4cp''' tasks:
  
−
Indicative coverage = 83%
+
'''1609-p0'''
  
 +
<jira jql='project = CSIT AND labels = ip4cp AND labels = func AND labels = 1609-p0 ORDER BY status DESC, priority DESC'></jira>
 +
 +
'''1609-p1'''
 +
 +
<jira jql='project = CSIT AND labels = ip4cp AND labels = func AND labels = 1609-p1 ORDER BY status DESC, priority DESC'></jira>
 +
 +
'''1609-p2'''
 +
 +
<jira jql='project = CSIT AND labels = ip4cp AND labels = func AND labels = 1609-p2 ORDER BY status DESC, priority DESC'></jira>
 +
 +
 +
Original CSIT plan - need to update per JIRA ([https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=39&quickFilter=64&quickFilter=35 board view], [https://jira.fd.io/issues/?filter=10632 list view]):
  
 
# [1609-P0] IKEv2. [https://jira.fd.io/browse/CSIT-27 CSIT-27]
 
# [1609-P0] IKEv2. [https://jira.fd.io/browse/CSIT-27 CSIT-27]
Line 75: Line 99:
 
# [1609-DONE] DHCPv4 proxy. [https://jira.fd.io/browse/CSIT-25 CSIT-25]
 
# [1609-DONE] DHCPv4 proxy. [https://jira.fd.io/browse/CSIT-25 CSIT-25]
  
−
==IPv4 encapsulations==
 
  
−
JIRA [https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=39&quickFilter=64&quickFilter=38 board view], [https://jira.fd.io/issues/?filter=10633 list view] - '''ip4enc''' label for CSIT work tracking.
+
==IPv4 encapsulations (ip4enc)==
 +
 
 +
JIRA status for '''ip4enc''' tasks:
 +
 
 +
'''1609-p0'''
 +
 
 +
<jira jql='project = CSIT AND labels = ip4enc AND labels = func AND labels = 1609-p0 ORDER BY status DESC, priority DESC'></jira>
 +
 
 +
'''1609-p1'''
 +
 
 +
<jira jql='project = CSIT AND labels = ip4enc AND labels = func AND labels = 1609-p1 ORDER BY status DESC, priority DESC'></jira>
 +
 
 +
'''1609-p2'''
 +
 
 +
<jira jql='project = CSIT AND labels = ip4enc AND labels = func AND labels = 1609-p2 ORDER BY status DESC, priority DESC'></jira>
  
−
Indicative coverage = 50%
+
Original CSIT plan - need to update per JIRA ([https://jira.fd.io/secure/RapidBoard.jspa?rapidView=8&quickFilter=39&quickFilter=64&quickFilter=38 board view], [https://jira.fd.io/issues/?filter=10633 list view]):
  
 
# [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. [https://jira.fd.io/browse/CSIT-231 CSIT-231]
 
# [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. [https://jira.fd.io/browse/CSIT-231 CSIT-231]

Revision as of 20:34, 12 September 2016

WORK IN PROGRESS - PLEASE TREAT IT AS SUCH

CSIT development plan - background

CSIT test development priorities are aligned with VPP releases timeline, with relative dev work prioritization for each VPP release driven by perceived importance of test verification for VPP functionality and performance. Specific CSIT dev work priorities are driven by rough consensus within the project and contributors and committers availability and willingness to do work.

Following priority tagging scheme is used in this CSIT wiki and FD.io jira for CSIT:

  • [$VppRlsNo-$RelPrio]
    • VppRlsNo - aligned VPP release number - 16.09, 16.12, 17.03.
    • RelPrio - CSIT release priority - P0 for Must Have, P1 for Should Have, P2 for Could Have
  • CSIT dev work tagging examples:
    • [1609-P0] - CSIT Must Have for vpp-v1609
    • [1609-P1] - CSIT Should Have for vpp-v1609
    • [1609-P2] - CSIT Could Have for vpp-v1609

CSIT dev work plan execution tracked on CSIT jira, with priority tags encoded as Jira labels for easy of tracking and reporting.

CSIT functional tests - dev plan

For the purpose of tracking CSIT dev work, CSIT functional test cases are grouped into the following VPP functional areas:

  1. IPv4 data plane
  2. IPv4 control plane
  3. IPv4 encapsulations
  4. IPv4 telemetry
  5. IPv6 data plane
  6. IPv6 control plane
  7. IPv6 encapsulations
  8. IPv6 telemetry
  9. Ethernet L2 data plane
  10. Ethernet L2 control plane
  11. Ethernet L2 encapsulations
  12. Ethernet L2 management plane
  13. Ethernet L2 telemetry
  14. MPLS data plane
  15. NSH data plane
  16. OPNFV-FDS integration

Following sections provide a top-level breakdown of CSIT dev work items covering functional test cases per area, and their proposed priorities. Items completed to date are prefixed with [DONE]; remaining items are prefixed with their respective priority per target release as described earlier. Indicative VPP functionality coverage is listed at the beginning of each section.

Identified VPP working code availability dependencies are marked by VPP Jira ticket [TODO VPP-xxx].

IPv4 data plane (ip4dp)

JIRA status for ip4dp tasks:

1609-p0

CSIT-183 Vpn baseline routed forwarding (VRF) Done
CSIT-188 security - source IPv4 Reverse Path Forwarding (RPF) Done
view in JIRA

1609-p1

CSIT-227 Routing multipath (ECMP) - baseline Done
CSIT-171 IPv4 DP qos - ingress policing - 1r3c-(cb,ca), 2r3c-(cb,ca) policers Done
CSIT-379 IPv4 DP qos - ingress policing - 1r3c-(cb,ca), 2r3c-(cb,ca) policers II Done
view in JIRA

1609-p2

CSIT-21 IPv4 DP - QOS ingress policing - 1r2c-(cb,ca), 2r3c-(cb,ca) policers - part I Done
CSIT-214 Extranet VPN routed forwarding (VRF) - baseline. Done
CSIT-228 IP4 Security - stateful firewall Done
view in JIRA

Original CSIT plan - need to update per JIRA (board view, list view):

  1. [1609-P1] qos - ingress policing - 1r2c-(cb,ca), 2r3c-(cb,ca) policers, RFC2475, RFC2697, RFC2698. CSIT-21
  2. [1609-P1] routing multipath (ECMP, UECMP) - baseline. CSIT-227
  3. [1609-P1] [TODO VPP-xxx] carrier grade nat 44, CGN44, RFC6598, RFC6888. CSIT-22 - The feature is blocked at the moment (for more info please see Jira ticket).
  4. [1609-P2] extranet VPN routed forwarding (VRF) - baseline. CSIT-214
  5. [1609-P2] [TODO VPP-xxx] security - stateful firewall. CSIT-228
  6. [1606-DONE] IPv4 routing RFC791 - baseline.
  7. [1606-DONE] security - iacl with n-tuple classification, ip4, ip6, mac.
  8. [1606-DONE] security - cop white-/black-list ip4 src addr filtering.
  9. [1609-DONE] VPN routed forwarding (VRF) - baseline. CSIT-183
  10. [1609-DONE] security - source IPv4 Reverse Path Forwarding (RPF). CSIT-188

IPv4 control plane (ip4cp)

JIRA status for ip4cp tasks:

1609-p0

CSIT-27 IPv4 CP - IKEv2 Done
CSIT-24 [P1-R1] IPv4 - Proxy ARP Done
view in JIRA

1609-p1

CSIT-25 IPv4 CP - DHCPv4 proxy Done
CSIT-341 IPv4 CP - DHCPv4 proxy - refactor Done
view in JIRA

1609-p2

view in JIRA


Original CSIT plan - need to update per JIRA (board view, list view):

  1. [1609-P0] IKEv2. CSIT-27
  2. [1606-DONE] ARP.
  3. [1606-DONE] ICMPv4.
  4. [1606-DONE] DHCPv4 client.
  5. [1609-DONE] Proxy ARP. CSIT-24
  6. [1609-DONE] DHCPv4 proxy. CSIT-25


IPv4 encapsulations (ip4enc)

JIRA status for ip4enc tasks:

1609-p0

CSIT-28 IPv4 encaps - ip4-ipsec-ip4, part I Done
CSIT-231 ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp Done
CSIT-124 IPv4 encapsulations - IP4-GRE-IP4 part II Done
CSIT-229 ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp Done
view in JIRA

1609-p1

CSIT-456 IPv4 - ip4-ipsec-lispgpe-ip6 - main fib, vrf, phy2lisp, virt2lisp Done
CSIT-392 ip4-lispgpe-ip4 - main fib, vrf; virt2lisp Done
CSIT-393 ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf); virt2lisp Done
CSIT-232 ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp Done
CSIT-230 ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp Done
CSIT-459 IPv4 - ip6-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp Done
view in JIRA

1609-p2

view in JIRA

Original CSIT plan - need to update per JIRA (board view, list view):

  1. [1609-P0] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-231
  2. [1609-P1] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-230
  3. [1609-P1] ip4-ipsec-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-232
  4. [1606-DONE] ip4-gre-ip4 - baseline main fib.
  5. [1609-DONE] ip4-ipsec-ip4 - baseline main fib. CSIT-28
  6. [1609-DONE] ip4-lispgpe-ip4 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-229

IPv4 telemetry

JIRA board view, list view - ip4tel label for CSIT work tracking.

Indicative coverage = 50%

  1. [1609-P1] [Blocked-by VPP-185] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline. CSIT-235
  2. [1609-P1] [Blocked-by VPP-186] Lawful intercept - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline. CSIT-236
  3. [1609-DONE] IPv4 IPFIX - stats collection and export - baseline. CSIT-233
  4. [1609-DONE] IPv4 IPFIX - stats collection and export - functional scale. CSIT-234

IPv6 data plane

JIRA board view, list view - ip6dp label for CSIT work tracking.

Indicative coverage = 45%

  1. [1609-P1] routing multipath (ECMP, UECMP) - baseline. CSIT-240
  2. [1609-P2] [TODO VPP-194] security - source IPv6 Reverse Path Forwarding (RPF). CSIT-208
  3. [1609-P2] [Blocked-by VPP-224] extranet VPN routed forwarding (VRF) - baseline. CSIT-238
  4. [1609-P2] [TODO VPP-xxx] security - stateful firewall.
  5. [1609-P2] qos - ingress policing - 1r2c-(cb,ca), 2r3c-(cb,ca) policers, RFC2475, RFC2697, RFC2698. CSIT-239
  6. [1606-DONE] routing [rfc2460] - baseline.
  7. [1606-DONE] security - iacl with n-tuple classification.
  8. [1606-DONE] security - cop white-/black-list ip4 src addr filtering.
  9. [1609-DONE] VPN routed forwarding (VRF) - baseline. CSIT-237

IPv6 control plane

JIRA board view, JIRA list view - ip6cp label for CSIT work tracking.

Indicative coverage = 50%

  1. [1609-P1] DHCPv6 proxy. CSIT-31
  2. [1609-P2] [TODO VPP-xxx] ip6-ioam, in-band OAM.
  3. [1609-P2] DHCPv6 client. CSIT-242
  4. [1606-DONE] neighbor discovery, RFC4861.
  5. [1606-DONE] ICMPv6.
  6. [1609-DONE] router advertisement - baseline tests. CSIT-241

IPv6 encapsulations

JIRA board view, JIRA list view - ip6enc label for CSIT work tracking.

Indicative coverage = 30%

  1. [1609-P1] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-243
  2. [1609-P1] ip6-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-244
  3. [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-245
  4. [1609-P1] ip4-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-246
  5. [1609-P1] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-247
  6. [1609-P1] ip6-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-248
  7. [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), phy2lisp, virt2lisp. CSIT-249
  8. [1609-P1] ip4-ipsec-lispgpe-ip6 - main fib, vrf (gpe_vni-to-vrf), lisp2lisp. CSIT-250
  9. [1609-P1] [TODO VPP-xxx] ip6-segment-routing.
  10. [1609-DONE] ip4-o-ip6 softwire - lightweight46. CSIT-32
  11. [1609-DONE] ip4-o-ip6 softwire - map-e. CSIT-197
  12. [1609-DONE] ip4-o-ip6 softwire - map-t. CSIT-353
  13. [1609-DONE] ip6-ipsec-ip6. CSIT-33

IPv6 telemetry

JIRA board view, JIRA list view - ip6tel label for CSIT work tracking.

Indicative coverage = 50%

  1. [1609-P1] [Blocked-by VPP-185] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline. CSIT-253
  2. [1609-P1] [Blocked-by VPP-186] Lawful intercept - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline. CSIT-254
  3. [1609-DONE] [Blocked-by VPP-204] IPv6 IPFIX - stats collection and export - baseline. CSIT-251
  4. [1609-DONE] [Blocked-by VPP-204] IPv6 IPFIX - stats collection and export - functional scale. CSIT-252

Ethernet L2 data plane

JIRA board view, JIRA list view - l2dp label for CSIT work tracking.

Indicative coverage = 80%

  1. [1609-P1] Integrated Routing-Bridging (IRB) - baseline. CSIT-255
  2. [1606-DONE] L2 Cross-connec - baseline.
  3. [1606-DONE] L2 Cross-connect - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2.
  4. [1606-DONE] Bridge domain - baseline.
  5. [1609-DONE] Bridge domain - with VLAN tag acrobatics 1-to-1, 1-to-2, 2-to-1, 2-to-2. CSIT-35

Ethernet L2 control plane

JIRA board view, JIRA list view - l2cp label for CSIT work tracking.

Indicative coverage = 66%

  1. [1609-P2] [TODO VPP-xxx] CDP/LLDP.
  2. [1606-DONE] MAC learning.
  3. [1606-DONE] MAC split-horizon.

Ethernet L2 encapsulations

JIRA board view, JIRA list view - l2enc label for CSIT work tracking.

Indicative coverage = 33%

  1. [1609-P1] l2-eth-lispgpe-ip6 - L2BD. CSIT-256
  2. [1609-P1] l2-eth-lispgpe-ip4 - L2BD. CSIT-257
  3. [1609-P1] l2eth-gre-ip4-ipsec-ip4-eth - L2BD. CSIT-258
  4. [1609-P2] l2-eth-l2tpv3-ip4 - L2BD, L2XC. CSIT-259
  5. [1609-P2] l2-eth-l2tpv3-ip6 - L2BD, L2XC. CSIT-260
  6. [1609-P2] l2-eth-gre-ip4 - L2BD. CSIT-261
  7. [1609-P2] l2-eth-gre-ip6 - L2BD. CSIT-262
  8. [1606-DONE] l2-vxlan-ipv4 - with bridge-domain.
  9. [1606-DONE] l2-vxlan-ipv4 - with L2 cross-connect.

Ethernet L2 telemetry

JIRA board view, JIRA list view - l2tel label for CSIT work tracking.

Indicative coverage = 0%

  1. [1609-P1] L2 FIB - stats collection and export - baseline. CSIT-263
  2. [1609-P2] L2 FIB - stats collection and export - functional scale. CSIT-264
  3. [1609-P1] [Blocked-by VPP-185] Switched port analyzer mirroring SPAN - (flow,port) mirroring to local port - baseline. CSIT-265
  4. [1609-P1] [Blocked-by VPP-186] Lawful intercept - (flow,port) mirroring to remote locatio via LI UDP tunnel - baseline. CSIT-266

MPLS data plane

[ JIRA board view], [ JIRA list view] - mplsdp label for CSIT work tracking.

Indicative coverage = 0%

  1. [1609-P2] vpnv4-mpls-gre-ip4. CSIT-267
  2. [1609-P2] mpls-o-eth. CSIT-268

NSH data plane

[ JIRA board view], [ JIRA list view] - nshdp label for CSIT work tracking.

Indicative coverage = 0%

  1. [1609-P1] ip4-nsh-gre-ip4. CSIT-269
  2. [1609-P1] ip4-nsh-vxlan-gpe-ip4. CSIT-270

OPNFV-FDS integration

Indicative coverage = 100%

  1. [1606-DONE] FDS VM scenario with two L2 bridge domains – ICMPv4.
  2. [1606-DONE] FDS VM scenario with two L2 bridge domains with static L2FIB entries – ICMPv4.
  3. [1606-DONE] FDS VM scenario with two L2 bridge domains – ICMPv6.
  4. [1606-DONE] FDS VM scenario with two L2 bridge domains with static L2FIB entries – ICMPv6.
  5. [1606-DONE] FDS VM scenario with L2 cross-connect – ICMPv4.
  6. [1606-DONE] FDS VM scenario with L2 cross-connect – ICMPv6.
  7. [1606-DONE] FDS L2 test cases with tenant networks (VXLAN).
  8. [1606-DONE] FDS L2 test cases with provider physical networks (VLAN).
Retrieved from "https://wiki.fd.io/index.php?title=CSIT/FuncTestPlan&oldid=2951"