Open Fabrics BOF Supercomputing 2008 Tziporet Koren, Gilad Shainer, Yiftah Shahar, Bob Woodruff, Betsy Zeller

2 Agenda  Introduction – 5 minutes  OFA objectives and goals  Open Fabrics Linux Update (15 – minutes)  OFED 1.3, 1.3.1, 1.4 releases  OFED 1.5 plans and roadmap  Open Fabrics Windows Update (15 – minutes)  WinOF 2.0 release  WinOF 2.1 plans and roadmap  Open Discussion – 25 minutes

3 Open Fabrics BOF - Introduction  Open Fabrics “The mission of the OpenFabrics Alliance (OFA) is to develop, distribute and promote a unified, transport-independent, open-source software stack for RDMA-capable fabrics and networks, including InfiniBand and Ethernet “  Support for both Microsoft Windows and Linux  Development happens using a typical open source development model Developer’s Working Group code developed and reviewed using lists  Validation and release by the Enterprise Working Group and the Windows Working Group OFED – for Linux WinOF – for Windows

4 Open Fabrics BOF - Introduction  Open Fabrics – working groups (cont.)  XWG – Executive Working group overall steering committee  IWG – Interoperability Working Group Open Fabrics logo program  LWG – Legal Working Group handles code licensing and other legal issues  MWG – Marketing Working Group Promotes the work of the alliance and recruits new members  UWG – Users Working Group define end user requirements

5 Agenda  Introduction – 5 minutes  OFA objectives and goals  Open Fabrics Linux Update (15 – minutes)  OFED 1.3, 1.3.1, 1.4 releases  OFED 1.5 plans and roadmap  Open Fabrics Windows Update (15 – minutes)  WinOF 2.0 release  WinOF 2.1 plans and roadmap  Open Discussion – 25 minutes

6 Linux: OFED Components  HCA/NIC Drivers  IB: IBM, Mellanox, QLogic  iWARP: Chelsio, NetEffects (now Intel)  Core: Verbs, mad, SMA, CMA, SA cache  IPoIB  SDP  SRP and SRP Target  iSER  RDS  Qlogic_VNIC  uDAPL  OSM  Diagnostic tools OFED 1.4 additions:  iSER Target  NFS-RDMA  Bonding module  Open iSCSI  MPI Components  MVAPICH  Open MPI  MVAPICH2  Benchmark tests  Proprietary MPIs: Intel, HP, Platform mpi  Proprietary SMs: Cisco, Voltaire, Qlogic OFA DevelopmentAdd on Tested with

7 OFED Development Process Update  add Betsy’s slide on development process changes since Sonoma here

8 OFED 1.3 and  OFED 1.3 released on Feb  Improved performance and scalability  Quality of Service (QOS) support  New OpenSM routing algorithms  IPoIB connect mode – improved performance  MPI enhancements to improve scalability and reduce memory footprint  Enhanced diagnostic tools to improve network management  Support for additional hardware, added additional iWarp NIC drivers  Expanded interoperability with Storage and Networking hardware  uDAPL 2.0  OFED release on June  Added support for RedHat EL 5.2 and SLES 10 SP2  fixed several critical bugs  Distro integration:  Red Hat AS 4.7 and RHEL 5.2, SLES10 SP2  Used in Intel Cluster Ready Solutions

9 OFED 1.4  General Info  Released in Nov 2008  Used in the Interoperability event in Nov 2008  Features:  Kernel base  VPI support: Eth and IB for ConnectX  NFS-RDMA – as technology preview  ISer Target  New BMME verbs (fast memory thru send queue (FRWR); Local invalidate send work requests; Read with invalidate)  IPoIB – LRO  RDS – GA with RDMA API  SDP – GA  uDAPL (socket CM for scalability; UD extensions)

10 OFED 1.4 (cont.)  Features (cont)  OpenSM: Cashed routing; APM - disjoint paths; Path balancing for LMC + console diagnostics; OpenSM configuration unification; MGID to MLID mapping for IPv6 SNM; Routing engines chain; IBA additions; Failover/Handover improvements  Congestion Control in ibutils  MPI: MVAPICH 1.1; MVAPICH2 1.2; OpenMPI  Likely Distro Integration  Red Hat EL 5.4, SLES ??  Used in Intel Cluster Ready Solutions

11 OFED 1.4 OS Matrix  kernel.org: kernel and  Novell  SLES 10  SLES 10 SP1 (up1)  SLES 10 SP2  Redhat  RHEL 4 (up4, up5, up6, up7)  RHEL 5 (up1, up2)  OEL  OEL 5  Free distros (with limited QA):  Open SuSE 10.3  Fedora Core 9  Ubuntu 6.06 (with RPM package installed)  CentOS 5.2 * new for OFED 1.4 in bold

12 OFED 1.5 Plans  Preliminary Schedule  Feature Freeze: 3/1/09  Alpha Release: 3/1/09  Beta Release: 4/1/09  RC1: 5/1/09  RC2-RCx: About every 2 weeks as needed  Release: June 2009  Features:  Kernel.org: and  Multiple Event Queues to support Multi-core CPUs  NFS/RDMA – GA  RDS support for iWarp  new versions of openmpi ? mvapich/mvapich2 ?  Add support/backports for RedHat EL 5.3  more TBD…

13 Agenda  Introduction – 5 minutes  OFA objectives and goals  Open Fabrics Linux Update (15 – minutes)  OFED 1.3, 1.3.1, 1.4 releases  OFED 1.5 plans and roadmap  Open Fabrics Windows Update (15 – minutes)  WinOF 2.0 release  WinOF 2.1 plans and roadmap  Open Discussion – 25 minutes

14 Open Fabrics Windows Update  placeholder for WinOF 2.0 and future plans slides

15 Agenda  Introduction – 5 minutes  OFA objectives and goals  Open Fabrics Linux Update (15 – minutes)  OFED 1.3, 1.3.1, 1.4 releases  OFED 1.5 plans and roadmap  Open Fabrics Windows Update (15 – minutes)  WinOF 2.0 release  WinOF 2.1 plans and roadmap  Open Discussion – 25 minutes