August 04, 2004John Carrier, Adaptec1 One-Shot STags John Carrier Adaptec

August 04, 2004John Carrier, Adaptec2 Agenda Problem Statement Proposals Recommendations

August 04, 2004John Carrier, Adaptec3 Problem Statement STags may be valid for more than one RDMA access, which may leave the ULP open to attack from the remote peer. The ULP may be especially vulnerable from attacks through RDMA Writes –The remote peer indicates completion of the RDMA through an RDMA Send type message. –When the local ULP receives this completion message, it may assume that it again has ownership of its sink buffer. –However, unless the STag itself is invalidated, the remote peer may overwrite the buffer even as the local ULP is processing its contents. This remote access may be inadvertent or deliberate. IPsec cannot address this problem because the error is generated by the connection peer, not a man in the middle.

August 04, 2004John Carrier, Adaptec4 RDMA Write with Explicit ULP Invalidation

August 04, 2004John Carrier, Adaptec5 Peer Attack of Data Sink Buffer

August 04, 2004John Carrier, Adaptec6 Proposed Solutions Caitlin Bestler proposed ‘one-shot’ STags as a means of limiting the scope of STags to a single RDMA transaction. Current architectures, however, enable persistent STags that may be used for multiple RDMA events. –Some protocols such as SCSI, will rely on persistent STags to enable targets to DMA data to initiator buffers in the way targets find most efficient (ie as it becomes available) Reflector discussions focused on mechanisms where the RNIC could invalidate the STag before the ULP processed the completion –Send & Invalidate –Receive & Invalidate –RDMA Write & Invalidate

August 04, 2004John Carrier, Adaptec7 Send & Invalidate DDP already includes a Send & Invalidate message that allows the remote peer to include an STag in a Send Message which the local RNIC can invalidate before the local ULP processes the completion. The original intent of this message was to reduce a roundtrip to the RNIC during completion processing. If the RNIC invalidated the STag, then the local peer would not have to write to the RNIC after receiving the completion. The mechanism, however, effectively closes remote access to the STag and removes the vulnerability to further attack from the remote peer.

August 04, 2004John Carrier, Adaptec8 Implicit Invalidation: Send & Invalidate

August 04, 2004John Carrier, Adaptec9 Receive & Invalidate Caitlin suggested that the local peer should be in control of invalidating its own STags. –Send & Invalidate leaves that control with the remote peer. –Unless the ULP explicitly describes the use of Send & Invalidate (as iSER does), it is possible that the message will not be used correctly and sink buffers will be vulnerable. For ULPs that know which untagged receive buffer will be used to complete an RDMA Write, Caitlin proposed a new mechanism where the ULP included an STag when it posted the receive buffer. When the RNIC consumes the untagged buffer, it retrieves the STag associated with it, and then invalidates the STag for the Data Sink buffer.

August 04, 2004John Carrier, Adaptec10 Implicit Invalidation: Receive & Invalidate

August 04, 2004John Carrier, Adaptec11 RDMA Write & Invalidate In both Send & Invalidate and Receive & Invalidate the RNIC invalidates the STag following an RDMA Send Message. It is possible to have the RNIC invalidate the STag following the RDMA Write Message. –The RNIC “knows” that a Write Message is complete –After ACKing the last byte of the message, it could invalidate the STag associated with the RDMA write. The problem with this proposal is that the remote peer may require multiple RDMA Writes to fill the Data Sink buffer. –iSER, for example, could not use this mechanism

August 04, 2004John Carrier, Adaptec12 Multiple Use of a Data Sink Stag

August 04, 2004John Carrier, Adaptec13 Recommendations Send & Invalidate is the only general purpose solution that can be validated on the wire Receive & Invalidate may be useful to specific classes of applications. –Two well known protocols (SDP, iSER), for example, do not have such tight control of receive resources as to be able to associate STags for an RDMA transaction with a specific untagged buffer to hold its completion. –Furthermore, enabling this mechanism requires a non-trivial change to the RNIC Interface in order for the RNIC to store a STags in a queue of STag buffer pointers. Write & Invalidate restricts use of RDMA Write

August 04, 2004John Carrier, Adaptec14 Discussion Are one-shot STags really necessary? Would it suffice to describe the pitfalls of not using Send&Invalidate?