© 2002 IBM Corporation IPoIB IETF-60 May 16, 2015 IPoIB Vivek Kashyap

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 2 IPoIB Draft Status

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 3  IP over InfiniBand Architecture –draft-ietf-ipoib-architecture-04.txt –Approved to be published as an Informational RFC.  Transmission of IP over InfiniBand –draft-ietf-ipoib-over-infiniband-06.txt –Issue: EUI64 -> GUID -> Interface Identifier –Sense of ‘u’ bit –Awaiting IB version 1.2 specification –Resubmit for ‘last-call’  DHCP over InfiniBand –draft-ietf-ipoib-dhcp-over-infiniband-06.txt –IESG Review

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 4 IPoIB Connected Mode

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 5 IP over InfiniBand Modes  Unreliable Datagram  Connected –Reliable –Unreliable  Raw

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 6 IPoIB Connected Mode Large MTU Automatic Path Migration Reliability RDMA  RC  2^31 bytes  2-4K links  Yes  UC  2^31 bytes  2-4K links  Yes  No  RDMA Write

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 7 IPoIB Subnet  IPoIB-Connected subnet is formed over a Non-Broadcast Multiple Access network IPoIB-UD is over an abstract broadcast link Node D IPd IPs Node A A IPa IPp IPr IPc Node C IPq IPb Node B IB switched network IPoIB UD broadcast domain IPoIB Connected network

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 8 Address Resolution IPbIPa IPoIB UD broadcast/multicast domain - IPb broadcasts query - IPa responds UD unicast IB switched network - CM REQ message - CM REP message - CM RTU message IPoIB Connected network - IP communication over connected mode

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 9 Address Resolution  IPb: Who Has IPa? –Address resolution over a UD QP –May reuse IPoIB-UD QP –Use multicast-GID –FF1x: : :: –P_Key of connected network –[RC|UC]: :GIDb –RC|UC indicate the requesters view of the subnet –IPa receives the packet –Determine PathRecord to GIDb –Send reply over UD-QP with GIDa [RC|UC]: :GIDa  The UD QPN is used to create the Service ID for setting IB connection

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 10 Connection Setup  Connection is setup using IB MADs  Service ID  Private Data field –Desired MTU –Minimum/Accepted MTU 0x1Resvd 3-octet QPN Resvd

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 11 MTU Negotiation per IB link  A requests MTU in private data of CM REQ message to peer B –Desired MTU in private data  B accepts value in CM REP message to peer A –Accepts value in the range ‘Minimum MTU’ and ‘Desired MTU’ –Respond with ‘Accepted MTU’  A accepts or, Rejects and tries again Allows a fallback to ‘Minimum MTU’ which is the MTU derived from Broadcast-GID’s MTU

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 12 Encapsulation  IP packets encapsulated with the ‘Ethertype’ indication as in IPoIB- UD  Address resolution over UD QP needs to follow IPoIB-UD encapsulation

IETF-60: IPoIB Connected Mode Vivek Kashyap, IBM 13 Ongoing discussions  Multiple connected mode links between peers –Draft says ‘SHOULD NOT’ –Proposal:  Add more detail to draft  Multiple IB connects are allowed. But ‘user beware’.  Mixed UC/RC interconnects –Link type determined based on advertisement –Issue: interoperability

© 2002 IBM Corporation IPoIB IETF-60 May 16, 2015 Questions? Vivek Kashyap