Cisco Storage Networking Fibre Channel James Long Storage Networking Systems Engineer jalong@cisco.com

Comprehensive Fibre Channel Support Login server Name server Zone Server (hardware-enforced) Fabric Configuration Server Registered State Change Notification (RSCN) Private loop, Public loop, Translative loop In-order delivery Fabric Shortest Path First (FSPF) IP over Fibre Channel Classes of service: 2, 3, F Standard port types: G, E, F, FL Enhanced port types: SD, ST, TE, TL Persistent FCIDs not bound to physical port

Director Class Port Density Lots of switches means lots of ISLs and wasted ports ISL over-subscription without congestion control Principal Switch Selection, Domain ID assignment, FSPF convergence, RSCN propagation takes longer Management is burdensome Troubleshooting is challenging Only one instance of fabric services No traffic isolation 512 Port SAN Built w/ 64-Port Directors 512 Port SAN Built w/ 256-Port Directors No ISLs means no wasted ports No oversubscription within the network Convergence is instantaneous Only 2 switches to manage Troubleshooting is simplified Virtual SANs provide service and traffic isolation MDS9509 is the first product to offer scalability from 16 ports to 208 ports in a single director

Clarification of PortChannel vs. Trunk An Inter-Switch Link (ISL) is an E_Port A PortChannel is a bundle of ISLs An Enhanced ISL (EISL) is an ISL that trunks VSANs An EISL is also called a Trunking E_Port (TE_Port) ISL = E_Port 3rd Party Switch Cisco MDS 9000 Logical E_Port Logical TE_Port ISL EISL ISL EISL ISL EISL MDS Switch MDS Switch MDS Switch MDS Switch PortChannel PortChannel + Trunking

Industry’s Most Scalable Port Channel A Port Channel aggregates links into a virtual ISL A virtual ISL has a logical interface name and aggregate bandwidth MDS can bundle up to 16 links for aggregate of 32 Gbps Up to 128 Port Channels per switch Any port in any slot, no restrictions Much faster recovery than FSPF-based load balancing Optimizes use of ISLs (fewer FSPF neighbors) Hardware-based intelligent load distribution algorithms Mode 1: based on src/dst FCID Mode 2: based on src/dst FCID + OXID (assures in order delivery) Port Channel Best practice is to use the same Port Channel interface number on both ends of the Port Channel Use “in-order-guarantee” command if applications cannot tolerate out-of-order I/Os or frames during fabric reconvergence; this feature takes precedence over Port Channel and FSPF load balancing features

PortChannel Protocol B A B A C po10 po5 1 1 1 2 2 2 3 3 3 Channel Group10 Channel Group 5 B A po10 po5 B 1 A 1 1 2 2 2 3 C 3 3 Individual link With PortChannel Protocol, Port A3 will be operational as Individual Link The plug-and-play functionality of PortChannel Protocol ensures the A3-B3 link is automatically added to PortChannel 5 Benefits similar to Ethernet 802.3ad Exchanges configuration information between switches to automatically configure and maintain PortChannels Provides consistency check of configuration parameters at both ends

FC Write Acceleration (WA) Extend Distances for DR/BC Applications Problem Performance of DR/BC application drops over extended distances Solution FC Write Acceleration minimizes end-to-end latency to preserve application IOPS Primary Applications Synchronous replication Host access to storage across campus or MAN Primary Data Center DR Data Center FC WA SSM SSM Assume an app can tolerate a latency of 1 ms. From a distance perspective this translates to a distance of 50 KM 50 KM * 4 One way delay * 5 us/km With Write Acceleration, for the same latency of 1 ms the distance can be increased to 100 km. Here is how: 100 KM * 2 one way delay * 5 us/km A major financial services company saw a 30% improvement in performance over 125 km distance

Without FC Write Acceleration With FC Write Acceleration SAN-OS 2.0(1) Feature FC Write Acceleration WRITE XFER_RDY DATA STATUS MDS w/SSM Without FC Write Acceleration With FC Write Acceleration I/O latency Reduction FC MAN EMC SRDF: yes EMC Mirrorview: no HDS Truecopy: yes HP CA-XP: yes HP CA-MA: yes HP CA-EVA: no IBM PPRC: no IBM PPRC-XD: no IBM XRC: no IBM FlashCopy: yes IBM FastT: yes Requirements for FC Write Acceleration Requires Storage Services Module (SSM) Initiator and Target must be directly attached to the SSM Module Benefits of FC Write Acceleration Improves response time for the storage application Extended distance for DR and BC apps

Allow Orange, Red, Blue VSANs Traffic Management with VSANs & Port Channels for Enhanced Application Performance App 1 Storage Array FC FC FC Allow only Green VSAN PortChannel 1 Bandwidth Allocation: Green VSAN 100% of PortChannel 1 Orange, Red, Blue VSANs share PortChannel 2 PortChannel 2 FC FC FC FC FC FC Allow Orange, Red, Blue VSANs Apps 2, 3, 4

FSPF Multipath Forwarding Optimizes use of fabric paths Utilizes up to 16 paths Reduces cost of adding more bandwidth to fabric Uses only equal-cost paths Supported over PortChannels FSPF Load Balancing The utilization of each path will not be taken into account—only the cost of the paths are considered The following formula can be used to calculate FSPF link cost: S * (1.0625e12 / Baud Rate) S represents an administratively defined weighting factor (default value = 1.0) R is the bit rate of the link The default link cost of a 1Gb link is 1000; the default link cost of a 2 Gb link is 500 On an MDS, you actually specify the total link cost, instead of specifying the S variable

Zone Server Enhancements Zone server now supports 2 modes Basic mode complies with GS3/SW2 standards (like previous SANOS releases) Enhanced mode complies with GS4/SW3 standards Consistent full-zone database across fabric Support for attributes in the standard Consistent zoning policy across fabric via lock mechanism Unique vendor type Reduced payload for activation request with distribute option set to full

Fabric-Based LUN Zoning Zone 1 WWN 1 WWN 3, LUN 1 LUN1 Extends fabric zoning granularity Provides an additional level of LUN security Hardware enforced Centralized, common interface for LUN masking across all storage arrays Promotes heterogeneous storage deployments LUN 1 WWN 1 Zone 1 WWN 3 LUN 2 FC LUN 2 FC Zone 2 MDS 9000 WWN 2 LUN1 LUN 1 WWN 4 Zone 2 WWN 2 WWN 3, LUN 2 WWN 4, LUN 1

Read-Only Zones Zone can be target level or LUN level Enables a volume to be shared across different application servers Video Editing Data Warehousing Backup Zoning based on SCSI I/O operation type: Read only Read and Write WWN, Port, and LUN zoning all done in hardware Zone 1 WWN 1 WWN 3, LUN 1 WWN 1 WWN 3 FC Zone 1 Read/Write LUN1 Read Only FC Zone 2 MDS 9000 WWN 2 Zone 2 WWN 2 WWN 3, LUN 1, Read Only

The Problem With FC Aliases FC Alias was invented to simplify zone configuration FC Alias: ERP_Servers FC Alias: ERP_Disks PWWN1 -PWWN4 PWWN2 PWWN3 ZS: Production_ZSv3 => Zone1: ERP_Servers, ERP_Disks Customers started to use FC Alias as substitute for WWNs FC Alias : ERP_ServerA  PWWN1 ERP_ServerC  PWWN3 ERP_ServerB  PWWN2 ERP_Disk  PWWN4 Shortcomings Alias distribution is tied to Zone Service (ZS distribution full) Alias is only visible in a given VSAN Alias name can’t be used by any other applications

Distributed Device Alias Services Cryptic WWNs Independent of FC Alias Independent of Zone Service Independent of VSANs 1:1 correspondence with WWN Unique aliases minimize zone merge issues Can be used by all applications (Zone, IVR, FC Ping, etc.) Uses CFS for SAN-wide distribution Simplifies SAN configuration and management User-friendly CLI/GUI commands and outputs WWN1 = 12:22:67:92:86:92:15:34 WWN2 = 02:12:35:86:93:08:64:43 WWN1 WWN2 Human friendly names Alias1 = Server-Oracle-ERP Alias2 = Array-OLTP Alias1 Alias2

N_Port ID Virtualization (NPIV) Email LUN 1 N_Port_ID 1 F_Port Web LUN 2 N_Port_ID 2 Application Server Print LUN 3 N_Port_ID 3 N_Port Controller MDS Switch Node Each application (email, web, print, etc.) connects to the fabric using its own FCID

NPIV Details Mechanism to assign multiple N_Port IDs to a single N_Port Multiple applications on the server can use different N_Port IDs on the same HBA Allows access control, zoning and port security to be implemented per application Multiple N_Port IDs are allocated in the same VSAN

MDS 9000 Interoperability Modes Interop Mode 1 – Standard mode that requires all other vendors in the fabric to be in interop mode; disruptive to implement on Brocade and McData fabrics if they are currently running in native mode Interop Mode 2 – Brocade native mode Core PID 0; non-disruptive to implement Interop Mode 3 – Brocade native mode Core PID 1; non-disruptive to implement