1. Cisco Storage Networking Fibre Channel
James Long
Storage Networking Systems Engineer

2. Comprehensive Fibre Channel Support
Login server
Name server
Zone Server (hardware-enforced)
Fabric Configuration Server
Registered State Change Notification (RSCN)
Private loop, Public loop, Translative loop
In-order delivery
Fabric Shortest Path First (FSPF)
IP over Fibre Channel
Classes of service: 2, 3, F
Standard port types: G, E, F, FL
Enhanced port types: SD, ST, TE, TL
Persistent FCIDs not bound to physical port

3. Director Class Port Density
Lots of switches means lots of ISLs and wasted ports
ISL over-subscription without congestion control
Principal Switch Selection, Domain ID assignment, FSPF convergence, RSCN propagation takes longer
Management is burdensome
Troubleshooting is challenging
Only one instance of fabric services
No traffic isolation
512 Port SAN Built w/ 64-Port Directors
512 Port SAN Built w/ 256-Port Directors
No ISLs means no wasted ports
No oversubscription within the network
Convergence is instantaneous
Only 2 switches to manage
Troubleshooting is simplified
Virtual SANs provide service and traffic isolation
MDS9509 is the first product to offer scalability from 16 ports to 208 ports in a single director

4. Clarification of PortChannel vs. Trunk
An Inter-Switch Link (ISL) is an E_Port
A PortChannel is a bundle of ISLs
An Enhanced ISL (EISL) is an ISL that trunks VSANs
An EISL is also called a Trunking E_Port (TE_Port)
ISL = E_Port
3rd Party Switch
Cisco MDS 9000
Logical E_Port
Logical TE_Port
ISL
EISL
ISL
EISL
ISL
EISL
MDS Switch
MDS Switch
MDS Switch
MDS Switch
PortChannel
PortChannel + Trunking

5. Industry’s Most Scalable Port Channel
A Port Channel aggregates links into a virtual ISL
A virtual ISL has a logical interface name and aggregate bandwidth
MDS can bundle up to 16 links for aggregate of 32 Gbps
Up to 128 Port Channels per switch
Any port in any slot, no restrictions
Much faster recovery than FSPF-based load balancing
Optimizes use of ISLs (fewer FSPF neighbors)
Hardware-based intelligent load distribution algorithms
Mode 1: based on src/dst FCID
Mode 2: based on src/dst FCID + OXID (assures in order delivery)
Port Channel
Best practice is to use the same Port Channel interface number on both ends of the Port Channel
Use “in-order-guarantee” command if applications cannot tolerate out-of-order I/Os or frames during fabric reconvergence; this feature takes precedence over Port Channel and FSPF load balancing features

6. PortChannel Protocol B A B A C po10 po5 1 1 1 2 2 2 3 3 3
Channel Group10
Channel Group 5 B A
po10
po5 B 1 A 1 1 2 2 2 3 C 3 3
Individual link
With PortChannel Protocol, Port A3 will be operational as Individual Link
The plug-and-play functionality of PortChannel Protocol ensures the A3-B3 link is automatically added to PortChannel 5
Benefits similar to Ethernet 802.3ad
Exchanges configuration information between switches to automatically configure and maintain PortChannels
Provides consistency check of configuration parameters at both ends

7. FC Write Acceleration (WA)
Extend Distances for DR/BC
Applications
Problem
Performance of DR/BC application drops over extended distances
Solution
FC Write Acceleration minimizes end-to-end latency to preserve application IOPS
Primary Applications
Synchronous replication
Host access to storage across campus or MAN
Primary Data Center
DR Data Center
FC WA
SSM
SSM
Assume an app can tolerate a latency of 1 ms.
From a distance perspective this translates to a distance of 50 KM
50 KM * 4 One way delay * 5 us/km
With Write Acceleration, for the same latency of 1 ms the distance can be increased to 100 km. Here is how:
100 KM * 2 one way delay * 5 us/km
A major financial services company saw a
30% improvement in performance over
125 km distance

8. Without FC Write Acceleration With FC Write Acceleration
SAN-OS 2.0(1) Feature
FC Write Acceleration
WRITE
XFER_RDY
DATA
STATUS
MDS w/SSM
Without FC Write Acceleration
With FC Write Acceleration
I/O latency Reduction FC
MAN
EMC SRDF: yes
EMC Mirrorview: no
HDS Truecopy: yes
HP CA-XP: yes
HP CA-MA: yes
HP CA-EVA: no
IBM PPRC: no
IBM PPRC-XD: no
IBM XRC: no
IBM FlashCopy: yes
IBM FastT: yes
Requirements for FC Write Acceleration
Requires Storage Services Module (SSM)
Initiator and Target must be directly attached to the SSM Module
Benefits of FC Write Acceleration
Improves response time for the storage application
Extended distance for DR and BC apps

9. Allow Orange, Red, Blue VSANs
Traffic Management with VSANs & Port Channels for Enhanced Application Performance
App 1
Storage Array FC FC FC
Allow only Green VSAN
PortChannel 1
Bandwidth Allocation:
Green VSAN 100% of PortChannel 1
Orange, Red, Blue VSANs share PortChannel 2
PortChannel 2 FC FC FC FC FC FC
Allow Orange, Red, Blue VSANs
Apps 2, 3, 4

10. FSPF Multipath Forwarding
Optimizes use of fabric paths
Utilizes up to 16 paths
Reduces cost of adding more bandwidth to fabric
Uses only equal-cost paths
Supported over PortChannels
FSPF Load Balancing
The utilization of each path will not be taken into account—only the cost of the paths are considered
The following formula can be used to calculate FSPF link cost:
S * (1.0625e12 / Baud Rate)
S represents an administratively defined weighting factor (default value = 1.0)
R is the bit rate of the link
The default link cost of a 1Gb link is 1000; the default link cost of a 2 Gb link is 500
On an MDS, you actually specify the total link cost, instead of specifying the S variable

11. Zone Server Enhancements
Zone server now supports 2 modes
Basic mode complies with GS3/SW2 standards (like previous SANOS releases)
Enhanced mode complies with GS4/SW3 standards
Consistent full-zone database across fabric
Support for attributes in the standard
Consistent zoning policy across fabric via lock mechanism
Unique vendor type
Reduced payload for activation request with distribute option set to full

12. Fabric-Based LUN Zoning
Zone 1
WWN 1
WWN 3, LUN 1
LUN1
Extends fabric zoning granularity
Provides an additional level of LUN security
Hardware enforced
Centralized, common interface for LUN masking across all storage arrays
Promotes heterogeneous storage deployments
LUN 1
WWN 1
Zone 1
WWN 3
LUN 2 FC
LUN 2 FC
Zone 2
MDS 9000
WWN 2
LUN1
LUN 1
WWN 4
Zone 2
WWN 2
WWN 3, LUN 2
WWN 4, LUN 1

13. Read-Only Zones Zone can be target level or LUN level
Enables a volume to be shared across different application servers
Video Editing
Data Warehousing
Backup
Zoning based on SCSI I/O operation type:
Read only
Read and Write
WWN, Port, and LUN zoning all done in hardware
Zone 1
WWN 1
WWN 3, LUN 1
WWN 1
WWN 3 FC
Zone 1
Read/Write
LUN1
Read Only FC
Zone 2
MDS 9000
WWN 2
Zone 2
WWN 2
WWN 3, LUN 1, Read Only

14. The Problem With FC Aliases
FC Alias was invented to simplify zone configuration
FC Alias: ERP_Servers FC Alias: ERP_Disks
PWWN1 -PWWN4
PWWN2
PWWN3
ZS: Production_ZSv3 => Zone1: ERP_Servers, ERP_Disks
Customers started to use FC Alias as substitute for WWNs
FC Alias : ERP_ServerA  PWWN1 ERP_ServerC  PWWN3
ERP_ServerB  PWWN2 ERP_Disk  PWWN4
Shortcomings
Alias distribution is tied to Zone Service (ZS distribution full)
Alias is only visible in a given VSAN
Alias name can’t be used by any other applications

15. Distributed Device Alias Services
Cryptic WWNs
Independent of FC Alias
Independent of Zone Service
Independent of VSANs
1:1 correspondence with WWN
Unique aliases minimize zone merge issues
Can be used by all applications (Zone, IVR, FC Ping, etc.)
Uses CFS for SAN-wide distribution
Simplifies SAN configuration and management
User-friendly CLI/GUI commands and outputs
WWN1 = 12:22:67:92:86:92:15:34
WWN2 = 02:12:35:86:93:08:64:43
WWN1
WWN2
Human friendly names
Alias1 = Server-Oracle-ERP
Alias2 = Array-OLTP
Alias1
Alias2

16. N_Port ID Virtualization (NPIV)
LUN 1
N_Port_ID 1
F_Port
Web
LUN 2
N_Port_ID 2
Application Server
Print
LUN 3
N_Port_ID 3
N_Port Controller
MDS Switch
Node
Each application ( , web, print, etc.) connects to the fabric using its own FCID

17. NPIV Details
Mechanism to assign multiple N_Port IDs to a single N_Port
Multiple applications on the server can use different N_Port IDs on the same HBA
Allows access control, zoning and port security to be implemented per application
Multiple N_Port IDs are allocated in the same VSAN

18. MDS 9000 Interoperability Modes
Interop Mode 1 – Standard mode that requires all other vendors in the fabric to be in interop mode; disruptive to implement on Brocade and McData fabrics if they are currently running in native mode
Interop Mode 2 – Brocade native mode Core PID 0; non-disruptive to implement
Interop Mode 3 – Brocade native mode Core PID 1; non-disruptive to implement