1. SCSI over PCI Express (SOP) use cases
T10/11-111r0
by Robert Elliott, HP
6 March 2011

2. Use cases SOP is not just for SSDs 1. PCIe CEM attached SSDs
3. PCIe attached RAID controllers
5. Internal interface for external targets
HP P2000 MSA external RAID controller
(SAS, iSCSI, or FC front-end; SAS back-end)
HP PCIe IO Accelerator
HP Smart Array P800 controller
2. PCIe Storage FF attached SSDs
4. PCIe attached HBAs
6. Other PCIe fabrics
HP 2.5" SSD (SATA shown)
HP NC373F iSCSI HBA and NIC
6 March 2011
T10/11-111r0 SCSI over PCI Express (SOP) use cases

3. #1: PCIe CEM attached SSDs
Common Electromechanical (CEM) form factor cards
Custom form factor cards (e.g., blade server mezzanine cards)
App client
SCSI initiator port
SCSI target port
Logical unit
LUN 0
Device server
SOP queues
SOP registers
CEM connector
Simple SOP IUs only support 2-byte LUN
CPU
PCIe Root Complex
SSD
PCIe
Direct-attached, or behind a PCIe switch fabric
SSD
PCIe
PCIe switch
PCIe Root Complex
PCIe
SSD
PCIe
6 March 2011
T10/11-111r0 SCSI over PCI Express (SOP) use cases

4. #2: PCIe Storage FF attached SSDs
New PCIe Storage hot-plug form factor with Multi-Protocol Multi-Link connector
App client
SCSI initiator port
SCSI target port
Logical unit
LUN 0
Device server
SOP queues
SOP registers
MultiLink connector
Simple SOP IUs only support 2-byte LUN
CPU
PCIe Root Complex
SSD
PCIe
Direct-attached, or behind a PCIe switch fabric
SSD
PCIe
PCIe switch
PCIe Root Complex
PCIe
SSD
PCIe
6 March 2011
T10/11-111r0 SCSI over PCI Express (SOP) use cases

5. #3: PCIe attached RAID controllers
RAID controller presents multiple LUNs, made from storage on the back-end SCSI domain(s)
Logical unit LUN 0 (RAID configuration)
App client
SCSI initiator port
SCSI target port
Logical units LUNs 1..n (Logical drives)
SOP queues
SOP registers
App client
SCSI initiator port
SCSI target port(s)
Complex SOP IUs support 8-byte LUN
Back-end SCSI domain(s)
(SAS, FC, Ethernet, USB, PCIe)
HDD
CEM connector
CPU
RAID
Other interconnect(s)
PCIe Root Complex
PCIe
SSD
Direct-attached, or behind a PCIe switch fabric
PCIe switch
RAID
PCIe
PCIe Root Complex
PCIe
PCIe
RAID
6 March 2011
T10/11-111r0 SCSI over PCI Express (SOP) use cases

6. #4: PCIe attached HBAs HBA CPU HBA PCIe switch HBA
HBAs bridge to back-end SCSI domains
Each back-end SCSI domain target port is addressable as a target port. Maintain back-end target port:LUN relationships
SCSI target port 0
LUN
App client
SCSI initiator port
LUN
SCSI target port N
LUN
LUN
SOP queues
SOP registers
App client
SCSI initiator port
SCSI target port(s)
LUN
Complex SOP IUs support remote fabric target port identifier and 8-byte LUN
LUN
Back-end SCSI domain(s)
(SAS, FC, Ethernet, USB, PCIe)
Other
HDD, SSD
CEM connector
CPU
HBA
Other interconnect(s)
Other
PCIe Root Complex
PCIe
RAID
Other
Tape, Library
Direct-attached, or behind a PCIe switch fabric
Other
PCIe switch
PCIe
HBA
PCIe Root Complex
PCIe
PCIe
HBA
Other
6 March 2011
T10/11-111r0 SCSI over PCI Express (SOP) use cases

7. #5: Internal interface for external targets
External RAID controllers, tape libraries
HBAs run backwards
Receive unsolicited SCSI commands, transmit responses
Provide multiple LUNs
SCSI initiator port(s)
SCSI target port
LUN
Device server
App client
LUN
LUN
Device server
SCSI initiator port
SCSI target port 0
LUN
SCSI target port N
LUN
SOP registers
SOP queues
SOP queues
SOP registers
LUN
SCSI initiator port
SCSI target port(s)
CPU
Front-end SCSI domain(s)
(SAS, FC, iSCSI, SOP)
HBA
Other
PCIe
PCIe Root Complex
HDD
Other
PCIe switch
PCIe
HBA
SSD
PCIe Root Complex
PCIe
Back-end SCSI domain(s)
HBA
HDD
PCIe
SSD
6 March 2011
T10/11-111r0 SCSI over PCI Express (SOP) use cases

8. T10/11-111r0 SCSI over PCI Express (SOP) use cases
Other requirements
SR-IOV (Single Root I/O Virtualization)
Virtual machines are mapped to virtual PCI functions
Both initiator and target use cases
Support SOP devices in Serial Attached SCSI (SAS) fabrics
If SOP drives using the Multifunction MultiLink connector become ubiquitous, will want to plug them in anywhere
PCIe Tunneling Protocol (PTP)
Include hints in SOP to facilitate tunneling
Advise when opening/closing connections may be prudent
Affiliation model – only allow access by one initiator
MR-IOV support to allow multiple initiators – may be too complex
6 March 2011
T10/11-111r0 SCSI over PCI Express (SOP) use cases