|
|
| (74 intermediate revisions by 8 users not shown) |
| Line 1: |
Line 1: |
| − | == FD.io CSIT Lab Extension - LLD Draft ==
| + | This page has been deprecated. Content has been moved to [[CSIT/Testbeds:_Xeon_Skx,_Arm,_Atom. | FD.io CSIT testbeds - Xeon Skylake, Arm, Atom]]. |
| − | | + | |
| − | This is a (proposed) low-level design for the extensions of FD.io CSIT lab to accommodate the new Intel Xeon Skylake, Arm AArch64 and Atom devices. The current version is available on [https://wiki.fd.io/view/CSIT/fdio_csit_lab_ext_lld_draft CSIT wiki]. | + | |
| − | | + | |
| − | == Server Specs ==
| + | |
| − | | + | |
| − | The new FD.io CSIT-CPL lab (is expected to contain|contains) following hardware server configurations:
| + | |
| − | | + | |
| − | 1. Type-1: Purpose - SUT for SW Data Plane Workload i.e. VPP, testpmd.
| + | |
| − | - Quantity: TBD based on testbed allocation.
| + | |
| − | - Physical connectivity:
| + | |
| − | - IPMI and host management ports.
| + | |
| − | - NIC ports connected into 2-node and 3-node topologies.
| + | |
| − | - Main HW configuration:
| + | |
| − | - Chassis: SuperMicro SYS-7049GP-TRT.
| + | |
| − | - Motherboard: SuperMicro X11DPG-QT.
| + | |
| − | - Processors: 2* Intel Platinum 8180 2.3 GHz.
| + | |
| − | - RAM Memory: 16* 16GB DDR4-2666MHz.
| + | |
| − | - Disks: 2* 1.6TB 6G SATA SSD.
| + | |
| − | - NICs configuration:
| + | |
| − | - Numa0: (x16, x16, x16 PCIe3.0 lanes)
| + | |
| − | - PCIe Slot2 18:00.xx: TBD e.g. x710-4p10GE Intel.
| + | |
| − | - PCIe Slot4 3b:00.xx: TBD e.g. (xxv710-DA2|xl710)-(2p25GE|2p40GE) Intel.
| + | |
| − | - PCIe Slot9 5e:00.xx: TBD e.g. ConnectX5-2p100GE Mellanox.
| + | |
| − | - Numa1: (x16, x16, x16 PCIe3.0 lanes)
| + | |
| − | - PCIe Slot6 86:00.xx: empty.
| + | |
| − | - PCIe Slot8 af:00.xx: empty.
| + | |
| − | - PCIe Slot10 d8:00.xx: empty.
| + | |
| − | 2. Type-2: Purpose - TG for T-Rex.
| + | |
| − | - Quantity: TBD based on testbed allocation.
| + | |
| − | - Physical connectivity:
| + | |
| − | - IPMI and host management ports.
| + | |
| − | - NIC ports connected into 2-node and 3-node topologies.
| + | |
| − | - Main HW configuration:
| + | |
| − | - Chassis: SuperMicro SYS-7049GP-TRT.
| + | |
| − | - Motherboard: SuperMicro X11DPG-QT.
| + | |
| − | - Processors: 2* Intel Platinum 8180 2.3 GHz.
| + | |
| − | - RAM Memory: 16* 16GB DDR4-2666MHz.
| + | |
| − | - Disks: 2* 1.6TB 6G SATA SSD.
| + | |
| − | - NICs configuration:
| + | |
| − | - Numa0: (x16, x16, x16 PCIe3.0 lanes)
| + | |
| − | - PCIe Slot2 18:00.xx: TBD e.g. x710-4p10GE Intel.
| + | |
| − | - PCIe Slot4 3b:00.xx: TBD e.g. xl710 2p40GE or xxv710-DA2 2p25GE Intel.
| + | |
| − | - PCIe Slot9 5e:00.xx: TBD e.g. ConnectX5-2p100GE Mellanox.
| + | |
| − | - Numa1: (x16, x16, x16 PCIe3.0 lanes)
| + | |
| − | - PCIe Slot6 86:00.xx: empty.
| + | |
| − | - PCIe Slot8 af:00.xx: empty.
| + | |
| − | - PCIe Slot10 d8:00.xx: TBD e.g. x710-4p10GE Intel.
| + | |
| − | 3. Type-3: Purpose - SUT for SW Data Plane Workload i.e. VPP, testpmd.
| + | |
| − | - Quantity: 2
| + | |
| − | - Physical connectivity:
| + | |
| − | - IPMI(?) and host management ports.
| + | |
| − | - NIC ports connected into 3-node topology.
| + | |
| − | - Main HW configuration:
| + | |
| − | - Chassis: Huawei TaiShan 2280.
| + | |
| − | - Processors: 1* hip07-d05 ~ 64* Arm Cortex-A72
| + | |
| − | - RAM Memory: 8* 16GB DDR4-2400MT/s
| + | |
| − | - Disks: 1* 4TB SATA HDD
| + | |
| − | - NICs configuration:
| + | |
| − | - PCIe Slot2 11:00.xx: 82599-2p10GE Intel.
| + | |
| − | - PCIe Slota e9:00.xx: connectx4-2p25GE Mellanox.
| + | |
| − | 4. Type-4: Purpose - SUT for SW Data Plane Workload i.e. VPP, testpmd.
| + | |
| − | - Quantity: 3
| + | |
| − | - Physical connectivity:
| + | |
| − | - Host management ports.
| + | |
| − | - NIC ports connected into 2-node and 3-node topologies.
| + | |
| − | - Main HW configuration:
| + | |
| − | - Chassis: MACCHIATObin.
| + | |
| − | - Processors: 1* Armada 8040 ~ 4* Arm Cortex-A72
| + | |
| − | - RAM Memory: 1* 16GB DDR4
| + | |
| − | - Disks: 1* 128GB(?) SATA SDD
| + | |
| − | - NICs configuration:
| + | |
| − | - pp2-2p10GE Marvell (on-chip Ethernet ports ; marvell plugin in VPP)
| + | |
| − | | + | |
| − | 5. Type-5: Purpose - SUT for SW Data Plane Workload i.e. VPP, testpmd.
| + | |
| − | - Quantity: TBD based on testbed allocation.
| + | |
| − | - Physical connectivity:
| + | |
| − | - IPMI(?) and host management ports.
| + | |
| − | - NIC ports connected into 2-node.
| + | |
| − | - Main HW configuration:
| + | |
| − | - Chassis: Netgate <..>.
| + | |
| − | - Processors: <..>.
| + | |
| − | - RAM Memory: <..>.
| + | |
| − | - Disks: <..>.
| + | |
| − | - NICs configuration:
| + | |
| − | - <..>
| + | |
| − | | + | |
| − | Current NIC equipment in storage includes:
| + | |
| − | - 2x XL710-QDA2
| + | |
| − | - 6x X710-DA2
| + | |
| − | - 6x X520-DA2
| + | |
| − | | + | |
| − | == Server Management Requirements ==
| + | |
| − | | + | |
| − | Total of 20 SM SYS-7049GP-TRT servers are made available for FD.IO CSIT testbed.
| + | |
| − | For management purposes, each server must have following two ports connected to the management network:
| + | |
| − | | + | |
| − | 1. 1GE IPMI port
| + | |
| − | - IPMI - Intelligent Platform Management Interface.
| + | |
| − | - Required for access to embedded server management with WebUI, CLI, SNMPv3, IPMIv2.0, for firmware (BIOS) and OS updates.
| + | |
| − | 2. 1GE/10GE management port
| + | |
| − | - hostOS management port for general system management.
| + | |
| − | | + | |
| − | == Server and Port Naming Convention ==
| + | |
| − | | + | |
| − | Following naming convention is used within this page to specify physical connectivity and wiring across defined CSIT testbeds:
| + | |
| − | | + | |
| − | - testbedname: testbedN.
| + | |
| − | - hostname:
| + | |
| − | - traffic-generator: tN-tgW.
| + | |
| − | - system-under-testX: tN-sutX.
| + | |
| − | - portnames:
| + | |
| − | - tN-tgW-cY/pZ.
| + | |
| − | - tN-sutX-cY/pZ.
| + | |
| − | - where:
| + | |
| − | - N - testbed number.
| + | |
| − | - tgW - server acts as traffic-generator with W index.
| + | |
| − | - sutX - server acts as system-under-test with X index.
| + | |
| − | - Y - PCIe slot number denoting a NIC card number within the host.
| + | |
| − | - Y=2,4,9 - slots connected to NUMA node 0.
| + | |
| − | - Y=6,8,10 - slots connected to NUMA node 1.
| + | |
| − | - Z - port number on the NIC card.
| + | |
| − | | + | |
| − | == Testbeds Overview ==
| + | |
| − | | + | |
| − | === Testbeds Type Breakdown ===
| + | |
| − | | + | |
| − | #. CSIT_tb. Purpose. SUT. TG. #tb. #SUTs. #TGs. #skx_node.
| + | |
| − | 1. 2-node Xeon. perf+func. skx. skx. 4. 4. 4. 8.
| + | |
| − | 2. 3-node Xeon. perf. skx. skx. 2. 4. 2. 6.
| + | |
| − | 3. tcp-l47. tcp-stack. skx. ps1. 2. 2. 1. 2.
| + | |
| − | 4. atom-netgate. perf+func. net. skx. >1. >1. 1. 1.
| + | |
| − | 5. aarch64. perf+func. arm. skx. >1. >1. 2. 2.
| + | |
| − | Total skx_node: 19.
| + | |
| − | | + | |
| − | === 2-Node Xeon Testbeds ===
| + | |
| − | | + | |
| − | Four 2-node Xeon testbeds (are expected to be built|are built), with each testbed using one SUTs (Type-1 server) and one TG (Type-2 server) connected back-to-back. NIC cards placement into slots and NIC ports connectivity is following the testbed specification included in next sections.
| + | |
| − | | + | |
| − | === 3-Node Xeon Testbeds ===
| + | |
| − | | + | |
| − | Two 3-node Xeon testbeds (are expected to be built|are built), with each testbed using two SUTs (Type-1 server) and one TG (Type-2 server) connected in full-mesh triangle. NIC cards placement into slots and NIC ports connectivity is following the testbed specification included in next sections.
| + | |
| − | | + | |
| − | === TCP/IP and L47 Testbeds ===
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | === AToM Testbeds ===
| + | |
| − | | + | |
| − | <Number> 2-node AToM (Netgate based) testbeds (are expected to be built|are built), with each testbed using one SUTs (Type-3 Netgate device) and one TG (Type-2 server) connected back-to-back. NIC cards placement into slots and NIC ports connectivity is following the testbed specification included in next sections.
| + | |
| − | | + | |
| − | === Arm Testbeds ===
| + | |
| − | | + | |
| − | One 3-node Huawei testbeds (are expected to be built|are built), with each testbed using two SUTs (Type-3 server) and one TG (Type-2 server) connected in full-mesh triangle.
| + | |
| − | | + | |
| − | One 3-node Marvell testbeds (are expected to be built|are built), with each testbed using two SUTs (Type-4 server) and one TG (Type-2 server) connected in full-mesh triangle.
| + | |
| − | | + | |
| − | == Server Management Addressing ==
| + | |
| − | | + | |
| − | === LOM (IPMI) and Management networks ===
| + | |
| − | | + | |
| − | Each server has a LOM (Lights-Out-Management e.g. SM IPMI) and a Management port, which are connected to two different VLANs.
| + | |
| − | | + | |
| − | 1. LOM (IPMI) VLAN:
| + | |
| − | - Subnet: <..>
| + | |
| − | - Gateway: <..>
| + | |
| − | - Broadcast: <..>
| + | |
| − | - DNS1: <..>
| + | |
| − | - DNS2: <..>
| + | |
| − | 2. Management Vlan:
| + | |
| − | - Subnet: <..>
| + | |
| − | - Gateway: <..>
| + | |
| − | - Broadcast: <..>
| + | |
| − | - DNS1: <..>
| + | |
| − | - DNS2: <..>
| + | |
| − | | + | |
| − | To access these hosts, an VPN connection is required.
| + | |
| − | | + | |
| − | === LOM (IPMI) VLAN IP Addresses allocation ===
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | === Management VLAN IP Addresses allocation ===
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | == Initial Build ==
| + | |
| − | | + | |
| − | Based on server hardware available on site as of 26-Apr-2018 (6 of Type-1/-2 servers), following are the initial testbeds to be built initially:
| + | |
| − | | + | |
| − | 1. 1 of 2-node Xeon ring3 40GE => totals: 2 * xeon servers, 2 * XL710-QDA2
| + | |
| − | 2. 1 of 3-node Xeon ring1+ring2 10GE => totals: 3 * xeon servers, 6 * X710-DA2
| + | |
| − | 3. 2 of arm ring1 10GE => totals: 1 * xeon servers, 2 * X520-DA2
| + | |
| − | | + | |
| − | === Server/Ports Naming, NIC Placement ===
| + | |
| − | | + | |
| − | ==== 2-Node Xeon ====
| + | |
| − | | + | |
| − | 1. Server1 of Type-1:
| + | |
| − | - testbedname: testbed21.
| + | |
| − | - hostname: s1-t21-sut1.
| + | |
| − | - IPMI IP: 10.30.50.41
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s1-t21-sut1-c2/p1 - empty.
| + | |
| − | - s1-t21-sut1-c2/p2 - empty.
| + | |
| − | - s1-t21-sut1-c2/p3 - empty.
| + | |
| − | - s1-t21-sut1-c2/p4 - empty.
| + | |
| − | - s1-t21-sut1-c4/p1 - 40GE-port1 xl710-2p40GE.
| + | |
| − | - s1-t21-sut1-c4/p2 - 40GE-port2 xl710-2p40GE.
| + | |
| − | - s1-t21-sut1-c9/p1 - empty.
| + | |
| − | - s1-t21-sut1-c9/p2 - empty.
| + | |
| − | 2. Server2 of Type-2:
| + | |
| − | - testbedname: testbed21.
| + | |
| − | - hostname: s2-t21-tg1.
| + | |
| − | - IPMI IP: 10.30.50.42
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s2-t21-tg1-c2/p1 - empty.
| + | |
| − | - s2-t21-tg1-c2/p2 - empty.
| + | |
| − | - s2-t21-tg1-c2/p3 - empty.
| + | |
| − | - s2-t21-tg1-c2/p4 - empty.
| + | |
| − | - s2-t21-tg1-c4/p1 - 40GE-port1 xl710-2p40GE.
| + | |
| − | - s2-t21-tg1-c4/p2 - 40GE-port2 xl710-2p40GE.
| + | |
| − | - s2-t21-tg1-c9/p1 - empty.
| + | |
| − | - s2-t21-tg1-c9/p2 - empty.
| + | |
| − | | + | |
| − | ==== 3-Node Xeon ====
| + | |
| − | | + | |
| − | 9. Server9 of Type-1:
| + | |
| − | - testbedname: testbed31.
| + | |
| − | - hostname: s9-t31-sut1.
| + | |
| − | - IPMI IP: 10.30.50.43
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s9-t31-sut1-c2/p1 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s9-t31-sut1-c2/p2 - 10GE-port2 x710-2p10GE.
| + | |
| − | - s9-t31-sut1-c2/p3 - empty.
| + | |
| − | - s9-t31-sut1-c2/p4 - empty.
| + | |
| − | - s9-t31-sut1-c4/p1 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s9-t31-sut1-c4/p2 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s9-t31-sut1-c9/p1 - empty.
| + | |
| − | - s9-t31-sut1-c9/p2 - empty.
| + | |
| − | 10. Server10 of Type-1:
| + | |
| − | - testbedname: testbed31.
| + | |
| − | - hostname: s10-t31-sut2.
| + | |
| − | - IPMI IP: 10.30.50.44
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s10-t31-sut2-c2/p1 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s10-t31-sut2-c2/p2 - 10GE-port2 x710-2p10GE.
| + | |
| − | - s10-t31-sut2-c2/p3 - empty.
| + | |
| − | - s10-t31-sut2-c2/p4 - empty.
| + | |
| − | - s10-t31-sut2-c4/p1 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s10-t31-sut2-c4/p2 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s10-t31-sut2-c9/p1 - empty.
| + | |
| − | - s10-t31-sut2-c9/p2 - empty.
| + | |
| − | 11. Server11 of Type-2:
| + | |
| − | - testbedname: testbed31.
| + | |
| − | - hostname: s5-t31-tg1.
| + | |
| − | - IPMI IP: 10.30.50.45
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s11-t31-tg1-c2/p1 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s11-t31-tg1-c2/p2 - 10GE-port2 x710-2p10GE.
| + | |
| − | - s11-t31-tg1-c2/p3 - empty.
| + | |
| − | - s11-t31-tg1-c2/p4 - empty.
| + | |
| − | - s11-t31-tg1-c4/p1 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s11-t31-tg1-c4/p2 - 10GE-port1 x710-2p10GE.
| + | |
| − | - s11-t31-tg1-c9/p1 - empty.
| + | |
| − | - s11-t31-tg1-c9/p2 - empty.
| + | |
| − | | + | |
| − | ==== 3-Node Arm ====
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | === Physical Connectivity within Testbeds ===
| + | |
| − | | + | |
| − | ==== 2-Node Xeon ====
| + | |
| − | | + | |
| − | One 2-Node testbeds are constructed by connecting 8 Xeon servers using below specification:
| + | |
| − | | + | |
| − | 1. testbed21:
| + | |
| − | - ring3 40GE-ports xl710-2p40GE on SUT
| + | |
| − | - t21-tg1-c4/p1 to t21-sut1-c4/p1.
| + | |
| − | - t21-sut1-c4/p2 to t21-tg1-c4/p2.
| + | |
| − | | + | |
| − | ==== 3-Node Xeon ====
| + | |
| − | | + | |
| − | 1. testbed31:
| + | |
| − | - ring1 10GE-ports x710-2p10GE on SUTs:
| + | |
| − | - t31-tg1-c2/p1 to t31-sut1-c2/p1.
| + | |
| − | - t31-sut1-c2/p2 to t31-sut2-c2/p2.
| + | |
| − | - t31-sut2-c2/p1 to t31-tg1-c2/p2.
| + | |
| − | - ring2 10GE-ports x710-2p10GE on SUT:
| + | |
| − | - t31-tg1-c4/p1 to t31-sut1-c4/p1.
| + | |
| − | - t31-sut1-c4/p2 to t31-sut2-c4/p2.
| + | |
| − | - t31-sut2-c4/p1 to t31-tg1-c4/p2.
| + | |
| − | | + | |
| − | ==== 3-Node Arm ====
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | == Target Build ==
| + | |
| − | | + | |
| − | === Server/Ports Naming, NIC Placement ===
| + | |
| − | | + | |
| − | ==== 2-Node Xeon ====
| + | |
| − | | + | |
| − | Each server in 2-node Xeon topology has its NIC cards placed, and NIC cards and ports indexed per following specification:
| + | |
| − | | + | |
| − | 1. Server1 of Type-1:
| + | |
| − | - testbedname: testbed21.
| + | |
| − | - hostname: s1-t21-sut1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s1-t21-sut1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s1-t21-sut1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s1-t21-sut1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s1-t21-sut1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s1-t21-sut1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s1-t21-sut1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s1-t21-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s1-t21-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 2. Server2 of Type-2:
| + | |
| − | - testbedname: testbed21.
| + | |
| − | - hostname: s2-t21-tg1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s2-t21-tg1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s2-t21-tg1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s2-t21-tg1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s2-t21-tg1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s2-t21-tg1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s2-t21-tg1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s2-t21-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s2-t21-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 3. Server3 of Type-1:
| + | |
| − | - testbedname: testbed22.
| + | |
| − | - hostname: s3-t22-sut1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s3-t22-sut1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s3-t22-sut1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s3-t22-sut1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s3-t22-sut1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s3-t22-sut1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s3-t22-sut1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s3-t22-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s3-t22-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 4. Server4 of Type-2:
| + | |
| − | - testbedname: testbed22.
| + | |
| − | - hostname: s4-t22-tg1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s4-t22-tg1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s4-t22-tg1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s4-t22-tg1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s4-t22-tg1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s4-t22-tg1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s4-t22-tg1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s4-t22-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s4-t22-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 5. Server5 of Type-1:
| + | |
| − | - testbedname: testbed23.
| + | |
| − | - hostname: s5-t23-sut1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s5-t23-sut1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s5-t23-sut1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s5-t23-sut1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s5-t23-sut1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s5-t23-sut1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s5-t23-sut1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s5-t23-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s5-t23-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 6. Server6 of Type-2:
| + | |
| − | - testbedname: testbed23.
| + | |
| − | - hostname: s6-t23-tg1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s6-t23-tg1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s6-t23-tg1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s6-t23-tg1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s6-t23-tg1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s6-t23-tg1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s6-t23-tg1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s6-t23-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s6-t23-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 7. Server7 of Type-1:
| + | |
| − | - testbedname: testbed24.
| + | |
| − | - hostname: s7-t24-sut1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s7-t24-sut1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s7-t24-sut1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s7-t24-sut1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s7-t24-sut1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s7-t24-sut1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s7-t24-sut1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s7-t24-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s7-t24-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 8. Server8 of Type-2:
| + | |
| − | - testbedname: testbed21.
| + | |
| − | - hostname: s8-t24-tg1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s8-t24-tg1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s8-t24-tg1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s8-t24-tg1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s8-t24-tg1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s8-t24-tg1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s8-t24-tg1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s8-t24-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s8-t24-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | | + | |
| − | ==== 3-Node Xeon ====
| + | |
| − | | + | |
| − | Each server in 3-node Xeon topology has its NIC cards placed, and NIC cards and ports indexed per following specification:
| + | |
| − | | + | |
| − | 9. Server9 of Type-1:
| + | |
| − | - testbedname: testbed31.
| + | |
| − | - hostname: s9-t31-sut1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s9-t31-sut1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s9-t31-sut1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s9-t31-sut1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s9-t31-sut1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s9-t31-sut1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s9-t31-sut1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s9-t31-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s9-t31-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 10. Server10 of Type-1:
| + | |
| − | - testbedname: testbed31.
| + | |
| − | - hostname: s10-t31-sut2.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s10-t31-sut2-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s10-t31-sut2-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s10-t31-sut2-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s10-t31-sut2-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s10-t31-sut2-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s10-t31-sut2-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s10-t31-sut2-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s10-t31-sut2-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 11. Server11 of Type-2:
| + | |
| − | - testbedname: testbed31.
| + | |
| − | - hostname: s5-t31-tg1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s11-t31-tg1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s11-t31-tg1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s11-t31-tg1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s11-t31-tg1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s11-t31-tg1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s11-t31-tg1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s11-t31-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s11-t31-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 12. Server12 of Type-1:
| + | |
| − | - testbedname: testbed32.
| + | |
| − | - hostname: s12-t32-sut1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s12-t32-sut1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s12-t32-sut1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s12-t32-sut1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s12-t32-sut1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s12-t32-sut1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s12-t32-sut1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s12-t32-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s12-t32-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 13. Server13 of Type-1:
| + | |
| − | - testbedname: testbed32.
| + | |
| − | - hostname: s13-t32-sut2.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s13-t32-sut2-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s13-t32-sut2-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s13-t32-sut2-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s13-t32-sut2-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s13-t32-sut2-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s13-t32-sut2-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s13-t32-sut2-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s13-t32-sut2-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | 14. Server14 of Type-2:
| + | |
| − | - testbedname: testbed32.
| + | |
| − | - hostname: s-14-t32-tg1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s14-t32-tg1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s14-t32-tg1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s14-t32-tg1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s14-t32-tg1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s14-t32-tg1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s14-t32-tg1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s14-t32-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s14-t32-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | | + | |
| − | ==== TCP/IP and L47 ====
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | ==== AToM ====
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | ==== Arm ====
| + | |
| − | | + | |
| − | 15. Server15 of Type-3:
| + | |
| − | - testbedname: testbed33.
| + | |
| − | - hostname: s15-t33-sut1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <10.30.51.36>
| + | |
| − | - portnames:
| + | |
| − | - s15-t33-sut1-ca/p1 - 10GE-port1 82599-2p10GE.
| + | |
| − | - s15-t33-sut1-ca/p2 - 10GE-port2 82599-2p10GE.
| + | |
| − | - s15-t33-sut1-c2/p1 - 25GE-port1 cx4-2p10GE.
| + | |
| − | - s15-t33-sut1-c2/p2 - 25GE-port2 cx4-2p10GE.
| + | |
| − | 16. Server16 of Type-3:
| + | |
| − | - testbedname: testbed33.
| + | |
| − | - hostname: s16-t33-sut2.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <10.30.51.37>
| + | |
| − | - portnames:
| + | |
| − | - s16-t33-sut2-ca/p1 - 10GE-port1 82599-2p10GE.
| + | |
| − | - s16-t33-sut2-ca/p2 - 10GE-port2 82599-2p10GE.
| + | |
| − | - s16-t33-sut2-c2/p1 - 25GE-port1 cx4-2p10GE.
| + | |
| − | - s16-t33-sut2-c2/p2 - 25GE-port2 cx4-2p10GE.
| + | |
| − | 17. Server17 of Type-2:
| + | |
| − | - testbedname: testbed33.
| + | |
| − | - hostname: s17-t33-tg1.
| + | |
| − | - IPMI IP: <..>
| + | |
| − | - Host IP: <..>
| + | |
| − | - portnames:
| + | |
| − | - s17-t33-tg1-c2/p1 - 10GE-port1 x710-4p10GE.
| + | |
| − | - s17-t33-tg1-c2/p2 - 10GE-port2 x710-4p10GE.
| + | |
| − | - s17-t33-tg1-c2/p3 - 10GE-port3 x710-4p10GE.
| + | |
| − | - s17-t33-tg1-c2/p4 - 10GE-port4 x710-4p10GE.
| + | |
| − | - s17-t33-tg1-c4/p1 - (25GE|40GE)-port1 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s17-t33-tg1-c4/p2 - (25GE|40GE)-port2 (xxv710-DA2|xl710)-(2p25GE|2p40GE).
| + | |
| − | - s17-t33-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE.
| + | |
| − | - s17-t33-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE.
| + | |
| − | | + | |
| − | === Physical Connectivity within Testbeds ===
| + | |
| − | | + | |
| − | ==== 2-Node Xeon ====
| + | |
| − | | + | |
| − | Four 2-Node testbeds are constructed by connecting 8 Xeon servers using below specification:
| + | |
| − | | + | |
| − | 1. testbed21:
| + | |
| − | - ring1 10GE-ports x710-4p10GE on SUT:
| + | |
| − | - t21-tg1-c2/p1 to t21-sut1-c2/p1.
| + | |
| − | - t21-sut1-c2/p2 to t21-tg1-c2/p2.
| + | |
| − | - ring2 10GE-ports x710-4p10GE on SUT:
| + | |
| − | - t21-tg1-c2/p3 to t21-sut1-c2/p3.
| + | |
| − | - t21-sut1-c2/p4 to t21-tg1-c2/p4.
| + | |
| − | - ring3 (25GE|40GE)-ports (xxv710-DA2|xl710)-(2p25GE|2p40GE) on SUT
| + | |
| − | - t21-tg1-c4/p1 to t21-sut1-c4/p1.
| + | |
| − | - t21-sut1-c4/p2 to t21-tg1-c4/p2.
| + | |
| − | - ring4 100GE-ports ConnectX5-2p100GE on SUT:
| + | |
| − | - t21-tg1-c9/p3 to t21-sut1-c9/p3.
| + | |
| − | - t21-sut1-c9/p4 to t21-tg1-c9/p4.
| + | |
| − | - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests:
| + | |
| − | - t21-tg1-c10/p1 to t21-tg1-c10/p2.
| + | |
| − | - t21-tg1-c10/p3 to t21-tg1-c10/p4.
| + | |
| − | | + | |
| − | 2. testbed22:
| + | |
| − | - <..>
| + | |
| − | | + | |
| − | 3. testbed23:
| + | |
| − | - <..>
| + | |
| − | | + | |
| − | 4. testbed24:
| + | |
| − | - <..>
| + | |
| − | | + | |
| − | ==== 3-Node Xeon ====
| + | |
| − | | + | |
| − | 1. testbed31:
| + | |
| − | - ring1 10GE-ports x710-4p10GE on SUTs:
| + | |
| − | - t31-tg1-c2/p1 to t31-sut1-c2/p1.
| + | |
| − | - t31-sut1-c2/p2 to t31-sut2-c2/p2.
| + | |
| − | - t31-sut2-c2/p1 to t31-tg1-c2/p2.
| + | |
| − | - ring2 10GE-ports x710-4p10GE on SUT:
| + | |
| − | - t31-tg1-c2/p3 to t31-sut1-c2/p3.
| + | |
| − | - t31-sut1-c2/p4 to t31-sut2-c2/p4.
| + | |
| − | - t31-sut2-c2/p3 to t31-tg1-c2/p4.
| + | |
| − | - ring3 (25GE|40GE)-ports (xxv710-DA2|xl710)-(2p25GE|2p40GE) on SUT
| + | |
| − | - t31-tg1-c4/p1 to t31-sut1-c4/p1.
| + | |
| − | - t31-sut1-c4/p2 to t31-sut2-c4/p2.
| + | |
| − | - t31-sut2-c4/p1 to t31-tg1-c4/p2.
| + | |
| − | - ring4 100GE-ports ConnectX5-2p100GE on SUT:
| + | |
| − | - t31-tg1-c9/p1 to t31-sut1-c9/p1.
| + | |
| − | - t31-sut1-c9/p2 to t31-sut2-c9/p2.
| + | |
| − | - t31-sut2-c9/p1 to t31-tg1-c9/p2.
| + | |
| − | - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests:
| + | |
| − | - t21-tg1-c10/p1 to t21-tg1-c10/p2.
| + | |
| − | - t21-tg1-c10/p3 to t21-tg1-c10/p4.
| + | |
| − | | + | |
| − | 2. testbed32:
| + | |
| − | - <..>
| + | |
| − | | + | |
| − | ==== TCP/IP and L47 ====
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | ==== AToM ====
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | ==== Arm ====
| + | |
| − | | + | |
| − | 1. testbed33:
| + | |
| − | - ring1 10GE-ports 82599-2p10GE on SUTs:
| + | |
| − | - s17-t33-tg1-c2/p1 - s15-t33-sut1-c?/p1.
| + | |
| − | - s15-t33-sut1-c?/p2 - s16-t33-sut2-c?/p1.
| + | |
| − | - s16-t33-sut2-c?/p2 - s17-t33-tg1-c2/p2
| + | |
| − | | + | |
| − | === Host OS and configuration ===
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | ==== SUTs ====
| + | |
| − | | + | |
| − | <..>
| + | |
| − | | + | |
| − | ==== TG ====
| + | |
| − | | + | |
| − | <..>
| + | |
