1. IP SAN Storage Overview
Introduction
This lesson discusses the basics of IP Storage Area Network (SAN). It covers IP Protocol Encapsulation, the Cisco 542x Storage Router, Internet Small Computer Systems Interface (iSCSI) host implementations, Fibre Channel over IP (FCIP), Internet Fibre Channel Protocol (iFCP). Additionally, the lesson discusses a dedicated IP storage network, security for IP storage, and IP storage advantages and disadvantages.
Importance
Understanding the basics of iSCSI use in SAN is crucial given Cisco's continued movement into the SAN markets.
© 2003, Cisco Systems, Inc. All rights reserved. 1

2. Lesson Objective
After completion of this lesson, you will explain the role of IP Storage, including iSCSI, FCIP, iFCP and security issues.
Performance Objective
After completion of this lesson, you will explain the role of IP Storage, including iSCSI, FCIP, iFCP and security issues.
Enabling Objectives
Explain the basic protocol encapsulation concepts of IP Storage
Explain the purpose of the 542x Storage Router in iSCSI storage solutions
Explain the three architectures used to support iSCSI Host implementations
Explain the basic concepts of Fibre Channel over IP (FCIP)
Explain the basic concepts of iFCP
Explain the purpose of dedicated IP Storage Networks
Explain IP Security (IPSec), Virtual LANs (VLANs), Access Control Lists (ACLs), Authentication, Authorization, Accounting, (AAA) and Firewalls as they relate to the management of storage traffic
Explain the advantages of IP storage solutions
Explain the disadvantages of IP storage solutions

3. Lesson Outline IP Protocol Encapsulation 542x Storage Router
iSCSI Host Implementations
What is FCIP (Fibre Channel over IP)?
iFCP: An Alternative to iSCSI and FCIP
Dedicated IP Storage Network
Security for IP Storage
IP Storage Advantages
IP Storage Disadvantages
Summary
Prerequisites
Curriculum Unit 1, Module 1 lesson "Storage Protocols."

4. IP Protocol Encapsulation
IP SANs carry block I/O traffic on top of IP
Leverage Gigabit Ethernet performance for local traffic
Use TCP: A reliable transport for delivery in MAN/WANs
Two primary protocols:
iSCSI—”IP-SCSI” IP-native transport of SCSI CDBs and data within TCP/IP connections
FCIP—”Fibre-Channel-over-IP”— Tunneling of Fibre Channel frames within TCP/IP connections, including FC fabric management frames IP
TCP
iSCSI
SCSI CDBs
IP Protocol Encapsulation
Objective
Explain the basic protocol encapsulation concepts of IP Storage.
Introduction
This section introduces the basics of IP SANs.
Facts
While Fibre Channel (FC) was the first network technology to see common use for transporting SAN traffic, IP networks can also be used. This includes Gigabit Ethernet for local storage as well as leveraging IP Metropolitan Area Networks (MANs) and Wide Area Networks (WANs) for business continuity applications.
There are two flavors of IP SAN Technology, one that tunnels Fibre Channel traffic in TCP/IP sessions and another that transports serial Small Computer Systems Interface (SCSI) as a native storage protocol. Native in this case means that another networking technology, such as FC, does not need to be used on the host.
The protocol diagrams illustrate both iSCSI and FCIP carrying SCSI command descriptor blocks (CDBs). IP
TCP
FCIP FC
SCSI CDBs

5. 542x Storage Router
Purpose is to link iSCSI-enabled servers on IP networks with Fibre Channel based storage subsystems
The 542X router terminates iSCSI CDBs and re-initiates Fibre Channel FCP CDBs
iSCSI Router
542x Storage Router
Objective
Explain the purpose of the 542x Storage router in iSCSI storage solutions.
Introduction
This section introduces the function of the Cisco 542x Storage router.
Facts
Cisco 5420 iSCSI routers connect IP-resident servers with FC-resident storage subsystems. The router terminates SCSI CDBs on the iSCSI side and re-initiates them on the FC side. Acknowledgements (ACKs) flow back through the same path. The SN 5420 Storage router includes:
1U high stackable appliance
iSCSI technology
FC port
Gigabit Ethernet port –1000Base-SX
High Availability
Graphical User Interface (GUI)/Command Line Interface (CLI)/Simple Network Management Protocol (SNMP) Management
FC point-to-point, loop and fabric
Access Control Lists (ACLs) for secure access to storage
Logical Unit Number (LUN) mapping
iSCSI Target
FC Initiator
FC Target
iSCSI Initiator
Ethernet
Switch
FC Switch

6. 542x Storage Router (cont.)
Host
iSCSI Solution Architecture
GigE
NIC
SCSI Adapter
Applications
File System
Block Device
SCSI Generic
TCP/IP Stack
NIC Driver
Adapter Driver
iSCSI
542X Storage Router
iSCSI Host Driver
SCSI Translation
FC Driver
TCP/IP Driver
FC HBA
GigE NIC
TCP/IP Network
542x Storage Router (cont.)
Facts
iSCSI traffic is initiated on the host and processed through the iSCSI protocol stack.
The target address and LUN seen by the host is exported on the Ethernet/IP network by the router.
The router terminates the SCSI session, unloads the iSCSI payload and creates a new FCP payload for transmission of the FC network.
The router creates a new FC frame and sends it over the FC SAN to the target:LUN address on the FC side.
It is worth noting that iSCSI and the Fibre Channel Protocol (FCP) serial SCSI are similar, but different implementations of the serial SCSI standard.
Fibre Channel
Fibre Channel Storage Array
542x Storage Router closeup

7. iSCSI Host Implementations
GigE
NIC
SCSI Adapter
Applications
File System
Block Device
SCSI Generic
TCP/IP Stack
NIC Driver
Adapter Driver
iSCSI
iSCSI device driver
Legacy NIC
Legacy software based TCP/IP stack
iSCSI Host Implementations
Objective
Explain the three architectures used to support iSCSI Host implementations
Introduction
This section introduces the three architectures used to support iSCSI Host implementations.
Facts
iSCSI Host Implementation (1)
The simplest implementation of iSCSI on the server involves adding an iSCSI device driver and using the legacy Network Interface Card (NIC) and software based TCP/IP stack. The problem with this implementation is that TCP/IP processing overhead can become a burden on the server resources.
iSCSI Host Implementation (1)

8. iSCSI Host Implementations (cont.)
iSCSI device driver
TCP Offload Engine (TOE)
Processing implemented in NIC
CPUs used to be faster than networks, but not anymore
Fewer interrupts
Eliminates memory copies
Applications
File System
Block Device
SCSI Generic
iSCSI
TOE Driver
Adapter Driver
TCP/IP Stack
SCSI Adapter
iSCSI Host Implementations (cont.)
Facts
iSCSI Host Implementation (2)
TCP algorithms can use significant CPU power at Gigahertz and greater speeds. That is why several companies are developing TCP offload technology.
When TCP was developed, processor performance far exceeded network performance and so it made sense to implement TCP in algorithms running as system processes. However, networking performance has progressed faster than CPU performance to the point where TCP algorithm processing can take a high percentage of CPU cycles.
Heavy traffic at Gigabit speed creates many interruptions: TCP processing requires many memory to memory copies—and memory has not kept pace with CPU performance, so CPUs have to wait for memory copies to complete. As CPU cycles are spent on TCP, there are fewer cycles left for applications. Hence the need for a TCP Offload Engine (TOE) card.
Heavy TCP gigabit speed requires ~ a gigahertz of CPU.
SCSI protocols can also be processed in the Host Bus Adapter (HBA)
It is important to note that iSCSI does not have a huge impact; the real benefit is TCP offloading.
iSCSI Host Implementation (2)

9. iSCSI Host Implementations (cont.)
Applications
iSCSI and TOE both implemented in NIC
File System
Block Device
SCSI Generic
TOE Driver
Adapter Driver
TCP/IP Stack
iSCSI Host Implementations (cont.)
Facts
iSCSI Host Implementation (3)
The only difference between this and the previous implementation is that the iSCSI logic has been moved into hardware ASICs (Application Specific Integrated Circuit, a chip designed for a specific application) along with the TOE.
Example
An example of such a product is the Alacritech 100x4 Quad-Port Server Accelerator.
iSCSI
SCSI Adapter
iSCSI Host Implementation (3)

10. iSCSI Host Implementation Practice
GigE
NIC
SCSI Adapter
Applications
File System
Block Device
SCSI Generic
TCP/IP Stack
NIC Driver
Adapter Driver
iSCSI
Applications
Applications
File System
File System
Block Device
Block Device
SCSI Generic
SCSI Generic
iSCSI
TOE Driver
Adapter Driver
TOE Driver
Adapter Driver
SCSI Adapter
Practice - iSCSI Host Implementation
Instructions
The three diagrams illustrate three iSCSI host implementations. Explain the differences between the three diagrams and indicate why one might adopt one implementation over another.
TCP/IP Stack
iSCSI
SCSI Adapter
TCP/IP Stack
iSCSI Host Implementation (1)
iSCSI Host Implementation (2)
iSCSI Host Implementation (3)

11. What is FCIP (Fibre Channel over IP)?
IT creates one logical fabric between remote SANS, and the switches think they are connected. IP is only used for tunneling through the WAN. FC FC
IP Network
Fibre Channel Fabric
Fibre Channel Fabric
What is FCIP (Fibre Channel over IP)?
Objective
Explain the basic concepts of Fibre Channel over IP (FCIP)
Introduction
This section introduces the basics of FCIP.
Facts
FCIP tunnels FC traffic over an IP network
FC frames are segmented and transmitted over IP LANS, MANS and WANS
The main reason for FCIP is to achieve remote storage for business continuity applications
Applications of FCIP include data mirroring, remote backup and remote storage copies
FCIP
Gateway
FCIP
Gateway FC FC

12. What is FCIP (Fibre Channel over IP)? (cont.)
FCIP creates a Virtual FC Inter-Switch Link (ISL)
FCIP creates a transparent method of extending Fabric ISLs over TCP/IP
FC Switch
FC Switch
ISL
What is FCIP (Fibre Channel over IP)? (cont.)
Facts
Architecturally, FCIP creates a virtual e-port connection through a pair of FCIP gateways
FC fabrics are formed through ISLs (Inter-Switch Link) that use e-ports in FC switches and directors (a director is a big fault tolerant switch). The idea is that all IP network details will be transparent to FC nodes and switches. Obviously, the faster the IP network is, and the less retries and errors there are, the better this works.
Network latencies can cause problems with respect to SCSI timeouts, which may range from tens to hundreds of milliseconds.
IP Network
FCIP
Gateway
FCIP
Gateway
FC Switch
FC Switch

13. What is FCIP (Fibre Channel over IP)? (cont.)
FCIP Extended SANs
Remote FC resources are viewed as local
Fabric service information is extended across the FCIP ISLs FC FC
Fibre Channel Fabric
IP Network
Fibre Channel Fabric
FCIP
Gateway
FCIP
Gateway FC FC
What is FCIP (Fibre Channel over IP)? (cont.)
Facts
Another way to view FCIP is that it extends the FC fabric across remote connections. The concept of an FC fabric is based on the notion that all switches will have the exact same service information available for all users of the network
With FCIP, all local and remote resources appear as local, even if the performance may not be the same.
In the top part of the diagram, there are two SAN islands, that is, two separate fabrics, connected by FCIP gateways across an IP network. These FCIP gateways could be separate devices, or an additional blade on a FC switch. The bottom half of the diagram shows the resulting logical unified fabric. FC FC
Fibre Channel Fabric FC FC

14. iFCP: An Alternative to iSCSI and FCIP
Mapping of Fibre Channel’s FCP protocol over UDP and IP
Only used by Nishan Systems
Limited industry support
iSCSI and FCIP are more widely supported
No chipsets or “native” HBAs and controllers
iFCP: An Alternative to iSCSI and FCIP
Objective
Explain the basic concepts of iFCP.
Introduction
This section introduces iFCP, which is a competitive technology that can be utilized as an alternative to iSCSI and FCIP.
Facts
iFCP is a competitive technology to iSCSI and FCIP. It uses FC’s FCP serial SCSI implementation, instead of iSCSI’s serial SCSI implementation. There is not likely going to be too much of it in the field, but it is already in use.
The technology was pioneered by Nishan systems, and has elements of both iSCSI and FCIP. Theoretically, it could be used for native transport, but silicon chip sets do not exist and are not being specified, unlike iSCSI and TOE. Thus, it is usually considered to be a gateway for FC tunneling.
Typically, with iFCP the addressing at the gateway is terminated. FC FC
IP Network
iFCP Gateway
iFCP Gateway

15. Dedicated IP Storage Network
Front-side IP Network
Clients
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI-enabled
Hosts
This diagram illustrates a representative dedicated IP storage network
Catalyst
Switches
Dedicated IP Network
Dedicated IP Storage Network
Objective
Explain the purpose of dedicated IP storage networks.
Introduction
This section introduces a representative dedicated IP storage network.
Facts
This diagram illustrates a representative dedicated IP storage network.
The IP storage network is isolated behind application hosts
There is minimized potential for bandwidth contention
It allows for the use of a separate VLAN in an existing Ethernet network
The use of dedicated Ethernet interfaces on a host for attachment to storage network is recommended
iSCSI Routers
FC Fabric
FC Attached Hosts with HBAs
Storage Pool

16. Security for IP Storage
Numerous services exist within IP to secure IP storage traffic, many of which are not available in Fibre Channel
IPSec
VLANs
Access Control Lists (ACLs)
Authentication, Authorization and Accounting
Firewalls
Security for IP Storage
Objective
Explain IPSec, VLANs, ACLs, Authentication, Authorization, and Accounting (AAA), and Firewalls as they relate to the management of storage traffic.
Introduction
This section introduces the management of SAN storage traffic as Cisco's IP leverage.
Facts
Many services exist within IP to secure IP storage traffic, many of which are not available in FC.
IPSec
FCIP uses IPSec hardware encryption to encrypt FCIP tunnels across the WAN/MAN. iSCSI standard calls for IPSec support. IPSec requires hardware acceleration in the client and is very vendor dependent.
IPSec is a set of protocols developed by the Internet Engineering Task Force (IETF) to support secure exchange of packets at the IP layer. IPSec has been deployed widely to implement Virtual Private Networks (VPNs).
IPSec supports two encryption modes: Transport and Tunnel. Transport mode encrypts only the data portion (payload) of each packet, but leaves the header untouched. The more secure Tunnel mode encrypts both the header and the payload. On the receiving side, an IPSec-compliant device decrypts each packet. For IPSec to work, the sending and receiving devices must share a public key. This is accomplished through a protocol known as Internet Security Association and Key Management Protocol/Oakley (ISAKMP/Oakley), which allows the receiver to obtain a public key and authenticate the sender using digital certificates. (see
Continued …

17. IP Storage Advantages
Many services exist within IP to ensure performance in an IP SAN as well as protect IP storage traffic from potential bottlenecks
QoS
EtherChannel
Gigabit Ethernet
Multi-Protocol Label Switching (MPLS)
IP Storage Advantages
Objective
Explain the advantages of IP storage solutions.
Introduction
This section provides a brief description of the advantages of IP Storage.
Facts
Many services exist within IP to ensure performance in an IP SAN as well as to protect IP storage traffic from potential bottlenecks.
Quality of Service (QoS): iSCSI uses QoS to protect iSCSI traffic and prioritize it within the LAN for higher priority queuing and switching. FCIP uses QoS to throttle non-IP storage traffic to protect FCIP traffic in a WAN/MAN.
EtherChannel: Use EtherChannel to bundle up to 16Gbps of bandwidth into one logical link within the LAN.
Gigabit Ethernet: Potentially use 100Mbps built-in host NIC for applications with lower performance requirements.
Multi-Protocol Label Switching (MPLS): Network administrators can utilize the traffic engineering capability of MPLS to intelligently move IP storage traffic across the WAN/MAN while optimizing bandwidth utilization.
© 2000, Cisco Systems, Inc.

18. IP Storage Disadvantages
IP Storage is an emerging technology
Extended distance may affect application performance
Security needs to be considered
IP Storage Disadvantages
Objective
Explain the disadvantages of IP storage solutions.
Introduction
This section briefly describes the potential disadvantages of IP Storage.
Facts
It should be noted that this is new technology and there are likely to be problems along the way. For example, storage interoperability should not be taken for granted and testing should be done to make sure everything will work as drawn.
While business continuity is desired, long distance connections are much slower than local connections— this can severely impact the performance of high throughput application systems such as Oracle transaction processing. These complexities are discussed in Module 2, Practical Applications of Storage Networking, Lesson 3, Replication.
Security of I/O traffic in IP SANs must be considered. Advancements such as IPSec may provide the security needed, but at a performance price for storage.

19. FCIP Practice Practice - FCIP Instructions
Label each of the components in the diagram and explain their function. Be sure to explain the problem to which FCIP is a solution.

20. Security for IP Storage Practice
Break up into groups of two. In one or two sentences, explain each of the following as they relate to the management and security of storage traffic:
IPSec
VLANs
Access Control Lists (ACLs)
Authentication, Authorization and Accounting
Firewalls
Practice - Security for IP Storage
Instructions
Explain each of the following as they relate to management and security of storage traffic:
IPSec
VLANs
ACLs
AAA
Firewalls

21. Advantages and Disadvantages Practice
Instructions
On a piece of paper, list as many advantages and disadvantages of IP storage as you can
Check your lists against the class notes
Practice - IP Storage Advantages and Disadvantages
Instructions
On a piece of paper, list as many advantages and disadvantages of IP storage as you can
Check your lists against the class notes

22. 542x Storage Router Practice
Practice - 542x Storage Router
Instructions
Label each of the components in the diagram and explain their function.

23. Lesson practice
A student was asked to write a summary paragraph about IP storage advantages and disadvantages. This is what he wrote:
There is one primary protocol, namely iSCSI, in, which we have native transport of SCSI CDBs and data within TCP/IP connections. There are two iSCSI host implementations, in which processing is implemented in the NIC and in which iSCSI and TOE are both implemented in NIC. There is another alternative to iSCSI, iFCP, but it has limited industry support. With regard to security, there are numerous services that exist within IP to secure IP storage traffic, most of which are available from all FC vendors. Thus, security for IP storage is very robust.
Instructions:
1. Break up into groups of two or three
2. Analyze the above paragraph, noting both areas in which it is correct, as well as errors and omissions.
3. Be prepared to share your answers with the class.
Practice - IP SAN Storage Overview

24. Dedicated IP Storage Network Practice
Practice - Dedicated IP Storage Network
Instructions
Label each of the components in the diagram and explain their function..

25. Lesson practice (cont.)
Answer the following questions. Be prepared to share your answers with the class.
Compare and contrast FCIP and iFCP
Compare and contrast a pure iSCSI SAN solution and one that uses FCIP
What is the main advantage of a TOE?
You are making a sales call on a customer who is considering purchasing a SAN. They want your advice on whether to implement a pure iSCSI backend network or a fibre channel backend network. Characterize the advantages and disadvantages of both. Before making a recommendation what information would you want to obtain from the customer?
Practice - IP SAN Storage Overview

26. Summary This lesson presented these key points:
There are two main protocols used in IP storage:
iSCSI
FCIP
The third, iFCP, does not have wide industry support.
There are a variety of iSCSI host implementations.
Numerous services exist within IP to secure IP storage traffic, many of which are not available in Fibre Channel
Advantagse of IP Storage include QoS, EtherChannel, Gigabit Ethernet, and Multi-Protocol Label Switching (MPLS)
Summary: IP Storage
Review
This lesson discussed the basics of IP Storage Area Network (SAN). It covered IP Protocol Encapsulation, the Cisco 542x Storage Router, Internet Small Computer Systems Interface (iSCSI) host implementations, Fibre Channel over IP (FCIP), Internet Fibre Channel Protocol (iFCP). Additionally, the lesson discussed a dedicated IP storage network, security for IP storage, and IP storage advantages and disadvantages.