1. Last updated: 15 Jan 2002Developed by GUSTECH1 Fibre Channel with SCSI A few notes about ‘Fibre Channel’ Some “FINE” Print: DISCLAIMER: This information is intended ONLY as an overview and introduction to Fibre Channel with SCSI and is NOT a specification of any kind, nor is it a substitution for official course material prepared by Solution Technology. GUSTECH, Solution Technology Inc., and the instructor make no warranty and assume no liability arising from the application or use of any information described herein, and assume no responsibility for errors appearing in this document or for any claims that the concepts or details discussed or disclosed in these materials are proprietary to any person or company. Students are strongly urged to clear any designs proposed for products with their patent and copyright council. Note: company, brand, product, trade, and other names used in this document are trademarks of their respective holders.

2. Last updated: 15 Jan 2002Developed by GUSTECH2 Fibre Channel with SCSI Brief Synopsis n Fibre Channel Topologies and General Characteristics n Fibre Channel Architectural Overview (classes & layers) n Fibre Channel “Payloads” n SCSI “over” Fibre Channel

3. Last updated: 15 Jan 2002Developed by GUSTECH3 Topologies & Characteristics Point-to- Point: N-Port TxRx N-Port TxRx ex.: Host-to-Host, or Host-to-single Disk or Disk Array. Bandwidth = 100% Full Duplex Fabric (switch) General View: N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx F-Port Tx Rx F-Port Tx Rx F-Port Tx Rx F-Port Rx Tx F-Port Rx Tx F-Port Rx Tx Fabric Communications Models can be arranged as one-to-one, one-to-many, and one-to-all.

4. Last updated: 15 Jan 2002Developed by GUSTECH4 Topologies & Characteristics Bandwidth = 100% Full Duplex Fabric (switch), Dedicated Connection Mode: N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx F-Port Tx Rx F-Port Tx Rx F-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx F-Port Rx Tx F-Port Rx Tx F-Port Rx Tx F_BSY Pair of Point-to-Point connections, deterministic and in-order delivery due to dedicated links; shown with full-resource reservation (non-virtual circuit). Connections made before communications start, and need to be dismantled when finished. ex.: Class 1 Service, like a computer I/O Channel

5. Last updated: 15 Jan 2002Developed by GUSTECH5 Topologies & Characteristics Packets of information (Frames) are routed based on the destination address in the frame header. No paths or resources are reserved. Successive frames may be routed over different paths and may be received out of order. Non-deterministic. Bandwidth: as available Fabric (switch), Connectionless Mode: N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx Frame Note: Mixed Mode Fabric is a switch that can simultaneously handle Connection and Connectionless Modes. ex.: Class 2 & 3 Service, like resource multiplexing and Internet non-acknowledgement

6. Last updated: 15 Jan 2002Developed by GUSTECH6 Topologies & Characteristics Bandwidth = Fixed % as needed Fabric (switch), Virtual Circuit (fractional bandwidth) Mode: N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx F-Port Tx Rx F-Port Tx Rx F-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx F-Port Rx Tx F-Port Rx Tx F-Port Rx Tx Point-to-Point connections, deterministic and in-order delivery due to dedicated links; shown with dedicated, partial-resource reservations, or virtual circuits. Connections made before communications start, and need to be dismantled when finished. These circuits are managed by the Quality of Service Facilitator 10% 15% 25% 60%40% ex.: Class 4 Service, like Data acquisition & video

7. Last updated: 15 Jan 2002Developed by GUSTECH7 Topologies & Characteristics NL-Port: NL-Port TxRx ex.: Arrays of I/O devices on a Controller Bandwidth = 100% Full Duplex Arbitrated Loop: Private loop shown: Up to 126 nodes. Only two communicating (virtual point-to-point) at a time. Others “repeat”. Repeater NL-Port Tx Rx NL-Port Rx Tx NL-Port Rx Tx NL-Port Rx Tx NL-Port Tx Rx NL-Port Tx Rx Not shown: By-Pass Circuitry HUBs are often used to develop loops and to enable by-pass and reconfiguration capabilities. The Loop Port State Machine performs initialization, arbitration, opening and closing loops, and includes the repeater

8. Last updated: 15 Jan 2002Developed by GUSTECH8 Topologies & Characteristics Arbitrated Loop: Public loop shown: Up to 127 nodes. Only two communicating (virtual point-to-point) at a time. Others “repeat”. Fabric node enables connection of loop to the “outside world” NL-Port Tx Rx NL-Port Rx Tx NL-Port Rx Tx NL-Port Tx Rx NL-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx N-Port Tx Rx F-Port Tx Rx F-Port Tx Rx F-Port Tx Rx F-Port Rx Tx F-Port Rx Tx FL-Port Rx Tx LOOPFABRIC NOTE: Only one FL-port per Arbitrated Loop

9. Last updated: 15 Jan 2002Developed by GUSTECH9 Architectural Overview Layers and Classes Physical Layer: Copper, Single Mode Fiber, Multimode Fiber 133 Mb/s, 256 Mb/s, 531 Mb/s, 1.06 Gb/s, 2.12 Gb/s, 4.25 Gb/s FC Layer:OSI Layer: FC-01 Encode/Decode Layer: “Link” transmission protocol, (8B/10B) Serializing, (10B/8B) Deserializing, and Error Detection FC-1 2 Framing Protocol Layer: “Transport” signaling including: Flow control, frame structure, and byte sequences FC-2 4 Common Services: “Session” Management Services, Name Server, Alias Server, and Key Server FC-3 5 Application: “Channels”: SCSI, SBCCS, HIPPI, IPI; “Networks”: IP, 802.2; “Multimedia”: Audio, Video FC-4 7 The five layers of Fibre Channel

10. Last updated: 15 Jan 2002Developed by GUSTECH10 Architectural Overview Layers and Classes 6 Fractional Bandwidth Connection: Same as Class 1 except latency and partial bandwidth usage are managed by a Quality of Service Facilitator (FFFFF9 ); “Virtual Circuit” 4 3 2 Acknowledged Connection: Established Connection; 100% Bandwidth; In-Order Delivery; Frame Receipt Acknowledged; End-to-End & Link-Level Flow Control 1 The five classes of service of Fibre Channel Acknowledged Connectionless: NO Established Connection; Bandwidth, Latency, and In-Order Delivery are NOT guaranteed; Frame Receipt is Acknowledged; does use End-to-End & Link-Level Flow Control Unacknowledged Connectionless: NO Established Connection, NO Bandwidth or In-Order Delivery guarantees, NO Frame Receipt Acknowledgement, NO End-to-End Flow Control; Does use Link-Level Flow Control; often called “Ship & Pray” Unidirectional Connection: Same as Class1modified for reliable one-to- many multicast and pre-emption; uses a Multicast Server (FFFFF5)

11. Last updated: 15 Jan 2002Developed by GUSTECH11 Fibre Channel “Payloads” a SCSI kind of view An exchange is composed of one or more non- concurrent sequences for a single operation. Each transfer is accomplished with a separate sequence or Information Unit. Each sequence consists of one or more frames, which are the smallest indivisible packet of data that is sent over the link. EXCHANGEEXCHANGE FRAME Like a SCSI Read CDB, Data, and Status FRAME First Sequence: CDB Frame Second Sequence: Data IN Frames End of Second Sequence: “Status” Frame Unique “Exchange” ID is identical to a Queue Tag EXCHANGEEXCHANGE FRAME Like a SCSI Test Unit Ready CDB, and Status FRAME First Sequence: CDB Frame Last Sequence: “Status” Frame Unique “Exchange” ID is identical to a Queue Tag INITIATORTARGET

12. Last updated: 15 Jan 2002Developed by GUSTECH12 Fibre Channel “Payloads” details on FRAMES FILLWORDFILLWORD Start of Frame 1 word Frame Header 6 words Optional Headers 0-64 bytes Payload Information Units Fill 0-3 bytes CRC 1 word End of Frame 1 word FILLWORDFILLWORD DATA FIELD (0-2112 Bytes) F R A M E R_CTLD estination A ddress (D_ID) (Target) CS_CTLSource Address (S_ID) (Initiator) TYPEFrame Control (F_CTL) SEQ_IDDF_CTLSEQ_CNT OX_ID (Q_tag#) RX_ID Parameter Field (PARAM) Routing Control Flags, Info types Class-specific control information Protocol (FC-4) Type in this frame SCSI-FCP = 8 Sequence # this frame belongs to Originator’s Exchange ID # Multi-purpose parameter field, like: Relative Offset or Task Retry Identifier Destination N_port Identifier Source N_port Identifier Frame Control field Sequential count of frames Responder’s Exchange ID #, for an exchange established by an Originator and identified by an OX_ID

13. Last updated: 15 Jan 2002Developed by GUSTECH13 SCSI over Fibre Channel Details on PAYLOADS: SCSI Information Units FILLWORDFILLWORD Start of Frame 1 word Frame Header 6 words Optional Headers 0-64 bytes Payload Information Units Fill 0-3 bytes CRC 1 word End of Frame 1 word FILLWORDFILLWORD DATA FIELD (0-2112 Bytes) F R A M E FC Name: Cate- gory SCSI Function # of Words Function # of Words Data NOTES: FCP_CMND6 Command (& embedded messages) 80 Messages in Task Codes & Flags FCP_XFR_ RDY 5(no direct equivalent)30 “Transfer Ready” is optional for reads FCP_DATA1Data (in and out)0Up to 528optional FCP_RESP7Status (& sense data)8Variable Sense data length per Word 4 FCP_CONF3(no direct equivalent)No payload0Confirm Status

14. Last updated: 15 Jan 2002Developed by GUSTECH14 SCSI over Fibre Channel Details on FCP_CMND PAYLOAD, Information Category = 6 FC Label WORD BYTE0Byte1Byte2Byte3 FCP_LUN 0First Level LUN Addressing (Byte 1)Second Level LUN Addressing 1Third Level LUN AddressingFourth Level LUN Addressing FCP_CNTL2CMD Ref. NumTask CodesTask Mgmt FlagsExec Mgmt Codes FCP_CDB 31 st byte of CDB2 nd byte of CDB3 rd byte of CDB4 th byte of CDB 45 th byte of CDB6 th byte of CDB7 th byte of CDB8 th byte of CDB 59 th byte of CDB10 th byte of CDB11 th byte of CDB12 th byte of CDB 613 th byte of CDB14 th byte of CDB15 th byte of CDB16 th byte of CDB Add_CDB7+?Additional CDB Bytes for “variable” (Group 3) CDB’s FCP_DLSize of data buffer allocated by Initiator BITS: 2 1 0 0 0 0 0 0 1 0 1 0 1 0 0 1 0 1 others Simple Q Head of Q Ordered Q ACA Q Untagged reserved Read Data = Bit 1 Write Data = Bit 0 Additional CDB Length = Bits 7..2 Bit 7 Bit 6 Bit 5 Bit 4 Bit 3 Bit 2 Bit 1 Bit 0 Terminate Task Clear ACA Target Reset Reset LUN reserved Clear Task Set Abort Task Set reserved Command Reference Number = Initiator assigned “order” numbers enabling “Precise Order of Delivery” capabilities: “1,+1 -> 255 ->, no 0” Now obsolete

15. Last updated: 15 Jan 2002Developed by GUSTECH15 SCSI over Fibre Channel Details on FCP_XFER_RDY PAYLOAD, Information Category = 5 FC Label WORD BYTE 0Byte 1Byte 2Byte 3 DATA_RO0Relative Offset for following data sequence BURST_LEN1Burst Length = Number of bytes in following data sequence reserved2 Optional for Read Commands Not to exceed “Maximum Burst Size” as set by Initiator using MODE PAGE management on Disconnect-Reconnect Mode Page

16. Last updated: 15 Jan 2002Developed by GUSTECH16 SCSI over Fibre Channel Details on FCP_DATA PAYLOAD, Information Category = 1 WORD BYTE 0Byte 1Byte 2Byte 3 0Data … n Three different types of SCSI data: Command Parameter (select) Command Response (sense) Read/write “information”

17. Last updated: 15 Jan 2002Developed by GUSTECH17 SCSI over Fibre Channel Details on FCP_RESP PAYLOAD, Information Category = 7 FC Label WORD BYTE 0Byte 1Byte 2Byte 3 reserved 0 1 FCP_STATUS 2reserved Validity FlagsSCSI Status Byte FCP_RESID 3Number of residual data bytes not transferred in the FCP_DATA FCP_SNS_LEN 4Sense Data Length FCP_RSP_LEN 5Response Information length: 8 bytes per FCP2 FCP_RSP_INFO 6reserved RSP_CODE 7reserved FCP_SNS_INFO 8 Sense Information: variable length per word 4: FCP_SNS_LEN n Bit Function: 0 RSP_LEN_VALID 1 SNS_LEN_VALID 2 RESID_OVER Bit Function: 3 RESID_UNDER 4 CONF_REQ 7-5 reserved Response to Command’s Task Management Flags: 00 = no failure or function complete 01 = Data length different than Burst_Len 02 = FCP_CMND Fields invalid 03 = DATA RO mismatch with XFER_RDY 04 = function not supported 05 = function failed 06-FF reserved

18. Last updated: 15 Jan 2002Developed by GUSTECH18 Fibre Channel with SCSI NOT PRESENTED: n Tons of details n About 4 days, 7 hours, and 30 minutes of rest of the data n The importance of the rest of the information

19. Last updated: 15 Jan 2002Developed by GUSTECH19 Fibre Channel with SCSI Recommended Study: Courses from Solution Technology: n Fibre Channel Concepts and Solutions n Comprehensive Introduction to Fibre Channel n Hands-ON Fibre Channel