SCSI over PCI Express (SOP) use cases T10/11-111r0 by Robert Elliott, HP (elliott@hp.com) 6 March 2011

Use cases SOP is not just for SSDs 1. PCIe CEM attached SSDs 3. PCIe attached RAID controllers 5. Internal interface for external targets HP P2000 MSA external RAID controller (SAS, iSCSI, or FC front-end; SAS back-end) HP PCIe IO Accelerator HP Smart Array P800 controller 2. PCIe Storage FF attached SSDs 4. PCIe attached HBAs 6. Other PCIe fabrics HP 2.5" SSD (SATA shown) HP NC373F iSCSI HBA and NIC 6 March 2011 T10/11-111r0 SCSI over PCI Express (SOP) use cases

#1: PCIe CEM attached SSDs Common Electromechanical (CEM) form factor cards Custom form factor cards (e.g., blade server mezzanine cards) App client SCSI initiator port SCSI target port Logical unit LUN 0 Device server SOP queues SOP registers CEM connector Simple SOP IUs only support 2-byte LUN CPU PCIe Root Complex SSD PCIe Direct-attached, or behind a PCIe switch fabric SSD PCIe PCIe switch PCIe Root Complex PCIe SSD PCIe 6 March 2011 T10/11-111r0 SCSI over PCI Express (SOP) use cases

#2: PCIe Storage FF attached SSDs New PCIe Storage hot-plug form factor with Multi-Protocol Multi-Link connector App client SCSI initiator port SCSI target port Logical unit LUN 0 Device server SOP queues SOP registers MultiLink connector Simple SOP IUs only support 2-byte LUN CPU PCIe Root Complex SSD PCIe Direct-attached, or behind a PCIe switch fabric SSD PCIe PCIe switch PCIe Root Complex PCIe SSD PCIe 6 March 2011 T10/11-111r0 SCSI over PCI Express (SOP) use cases

#3: PCIe attached RAID controllers RAID controller presents multiple LUNs, made from storage on the back-end SCSI domain(s) Logical unit LUN 0 (RAID configuration) App client SCSI initiator port SCSI target port Logical units LUNs 1..n (Logical drives) SOP queues SOP registers App client SCSI initiator port SCSI target port(s) Complex SOP IUs support 8-byte LUN Back-end SCSI domain(s) (SAS, FC, Ethernet, USB, PCIe) HDD CEM connector CPU RAID Other interconnect(s) PCIe Root Complex PCIe SSD Direct-attached, or behind a PCIe switch fabric PCIe switch RAID PCIe PCIe Root Complex PCIe PCIe RAID 6 March 2011 T10/11-111r0 SCSI over PCI Express (SOP) use cases

#4: PCIe attached HBAs HBA CPU HBA PCIe switch HBA HBAs bridge to back-end SCSI domains Each back-end SCSI domain target port is addressable as a target port. Maintain back-end target port:LUN relationships SCSI target port 0 LUN App client SCSI initiator port LUN SCSI target port N LUN LUN SOP queues SOP registers App client SCSI initiator port SCSI target port(s) LUN Complex SOP IUs support remote fabric target port identifier and 8-byte LUN LUN Back-end SCSI domain(s) (SAS, FC, Ethernet, USB, PCIe) Other HDD, SSD CEM connector CPU HBA Other interconnect(s) Other PCIe Root Complex PCIe RAID Other Tape, Library Direct-attached, or behind a PCIe switch fabric Other PCIe switch PCIe HBA PCIe Root Complex PCIe PCIe HBA Other 6 March 2011 T10/11-111r0 SCSI over PCI Express (SOP) use cases

#5: Internal interface for external targets External RAID controllers, tape libraries HBAs run backwards Receive unsolicited SCSI commands, transmit responses Provide multiple LUNs SCSI initiator port(s) SCSI target port LUN Device server App client LUN LUN Device server SCSI initiator port SCSI target port 0 LUN SCSI target port N LUN SOP registers SOP queues SOP queues SOP registers LUN SCSI initiator port SCSI target port(s) CPU Front-end SCSI domain(s) (SAS, FC, iSCSI, SOP) HBA Other PCIe PCIe Root Complex HDD Other PCIe switch PCIe HBA SSD PCIe Root Complex PCIe Back-end SCSI domain(s) HBA HDD PCIe SSD 6 March 2011 T10/11-111r0 SCSI over PCI Express (SOP) use cases

T10/11-111r0 SCSI over PCI Express (SOP) use cases Other requirements SR-IOV (Single Root I/O Virtualization) Virtual machines are mapped to virtual PCI functions Both initiator and target use cases Support SOP devices in Serial Attached SCSI (SAS) fabrics If SOP drives using the Multifunction MultiLink connector become ubiquitous, will want to plug them in anywhere PCIe Tunneling Protocol (PTP) Include hints in SOP to facilitate tunneling Advise when opening/closing connections may be prudent Affiliation model – only allow access by one initiator MR-IOV support to allow multiple initiators – may be too complex 6 March 2011 T10/11-111r0 SCSI over PCI Express (SOP) use cases