1. iSCSI Storage Dennis Martin Demartek May 2007 RMWTUG Meetings

2. © 2007 Demartek 2 Introduction ♦Dennis Martin, President, Demartek Demartek ♦27 years in I.T. –Software Engineer –Software Engineering Manager –Data Center Manager –Marketing –Industry Analyst –Mainframe, UNIX, Windows ♦Founded RMWNTUG in Denver in 1994 ♦Microsoft Storage MVP 2005, 2006, 2007

3. © 2007 Demartek 3 Pre-Presentation Survey ♦How many are using iSCSI storage today? ♦How many are planning to use iSCSI storage? ♦How many have heard of iSCSI but want to know more?

4. © 2007 Demartek 4 iSCSI Storage – Agenda ♦Storage: DAS, NAS and SAN ♦iSCSI Concepts ♦Network Technology Items ♦Storage Technology Items ♦Solutions Available Today ♦Microsoft iSCSI Best Practices ♦Performance ♦Futures ♦Recommendations ♦Reference links

5. © 2007 Demartek 5 Direct Access Storage (DAS) ♦Storage is directly connected to CPU ♦Short distance between CPU and storage (inches or small number of meters) ♦Storage is “owned” by one host computer only ♦Limited number of storage devices can be connected ♦Common interfaces are ATA/IDE, SCSI, SATA, SAS

6. © 2007 Demartek 6 DAS Diagram Due to cable length limitations, the storage devices often reside in the same cabinet as the CPU, or in a separate enclosure physically near the CPU cabinet. Host CPU performs “block” I/O directly to device. CPUMemory Storage Interface Storage Device Inside the Host Computer Cabinet © 2007 Demartek

7. 7 Network Attached Storage (NAS) ♦One host server shares its resources with other clients on the network ♦Clients make requests by filename ♦NAS server has its own storage, clients only see “share” names ♦Large number of “shares” can be created ♦NAS server can be a long distance from clients ♦Common protocols are NFS and CIFS

8. © 2007 Demartek 8 NAS Diagram NAS Server Local Area Network (LAN) Network clients NAS Private Storage Network Attached Storage (NAS) Diagram Network clients request files from file server (NAS device) over the LAN. NAS device receives file requests from network clients and translates those requests to “block” I/O commands to its own private storage. It then formats its data into a “file” format and responds to the network clients. © 2007 Demartek

9. 9 Storage Area Network (SAN) ♦Infrastructure allows computers to appear to have direct connection to storage ♦Storage can be great distance from computers requiring access ♦Clients make “block” I/O requests ♦Can be millions of storage devices in a SAN ♦Two technologies: Fibre Channel and Ethernet

10. © 2007 Demartek 10 SAN Diagram Local Area Network (LAN) Network clients Storage Devices Storage Area Network (SAN) Diagram Storage Area Network (SAN) Application Servers Storage traffic for SAN storage generally flows over a separate network from the LAN traffic. © 2007 Demartek

11. 11 SAN Components ♦Server ♦SAN Interface Card ♦SAN Switch ♦Storage Device (with Interface)

12. © 2007 Demartek 12 SAN Component Diagram Storage Device Storage Area Network (SAN) Component Diagram Application Server Interface card Switch © 2007 Demartek

13. 13 Unified Storage ♦Combination of NAS and SAN into single, integrated storage solution ♦Often includes: –Fibre Channel SAN technology –iSCSI SAN technology –NAS technology

14. © 2007 Demartek 14 iSCSI SAN Components For iSCSI today, Ethernet components should be 1-Gigabit or better. Cabling and interfaces can be copper or fiber-optic. Ethernet adapters can be on the motherboard or in a PCI/PCI-X/PCI-E slot. iSCSI Target Storage Device iSCSI SAN Components Application Server Ethernet adapter Ethernet Switch Ethernet cabling © 2007 Demartek

15. 15 iSCSI Initiators ♦Same basic function as SCSI initiators ♦Available in software or hardware adapters ♦Microsoft provides free software initiator –Free download for : Windows Server 2003, Windows XP Pro SP1, Windows 2000 SP3 –Included in Control Panel in Windows Vista, Longhorn Server –Available in 32-bit and 64-bit ♦Third-parties sell iSCSI Host Bus Adapters (HBA) ♦Initiators have iSCSI Qualified Name (IQN)

16. © 2007 Demartek 16 iSCSI Targets ♦Same basic function as SCSI targets ♦Available in software or hardware ♦Wide variety of hardware targets available today that can fit price, capacity or performance needs –SATA disk arrays –SAS disk arrays –Fibre Channel disk arrays ♦Targets have iSCSI Qualified Name (IQN)

17. © 2007 Demartek 17 Microsoft iSCSI Initiator Install 1 2 3 4 MPIO not checked by default

18. © 2007 Demartek 18 Microsoft iSCSI Initiator Notice the IQNs Three targets (volumes) visible to this initiator Two target portals visible to this initiator

19. © 2007 Demartek 19 Multi-path I/O (MPIO) ♦MPIO is supported for iSCSI ♦Same MPIO as Fiber Channel ♦Allows multiple network connections from the same host server to function as data paths to the target ♦Different load balance policies are available

20. © 2007 Demartek 20 iSCSI Security ♦iSCSI security should be considered in addition to other network security that may be implemented ♦Challenge Handshake Authentication Protocol (CHAP) –Predefined shared secret between initiators and targets –Only allows logon if the other party knows the secret –Can be one-way or mutual authentication ♦IPSec –If enabled, all data transfer packets are encrypted and authenticated

21. © 2007 Demartek 21 Network Technology Items ♦These affect iSCSI AND your regular network ♦Server-class NICs ♦Scalable Networking Pack ♦Receive-side Scaling ♦TCP Offload Engines (TOE) ♦Switches ♦Jumbo Frames

22. © 2007 Demartek 22 Server-class NICs ♦There’s a big difference between a low-cost desktop NIC and a server-class NIC –Server-class NICs typically have some degree of offload function such as TCP Checksum Offload, Large Send Offload and other similar functions –Server-class NICs have equal or better performance with lower CPU utilization on the server –Server-class NICs are more expensive than low-cost NICs

23. © 2007 Demartek 23 Scalable Networking Pack ♦Microsoft Scalable Networking Pack (SNP) –Free download for Windows 2003 Server and Windows XP 64-bit –Included in Windows Server 2003 R2 SP2 ♦Provides network acceleration and compatibility with hardware-based offload technologies –Receive-side Scaling –TCP Offload Engines –NetDMA ♦Improves all network traffic, not just iSCSI

24. © 2007 Demartek 24 Receive-side Scaling ♦Especially important with multi-core and multi- processor systems ♦Without SNP and Receive-side Scaling, all incoming network traffic interrupts are routed to exactly one core ♦With SNP and Receive-side Scaling, incoming network traffic interrupts are distributed across the cores and processors ♦Requires a NIC that supports Receive-side Scaling ♦Provides very good performance * Also known as “Scalable I/O” on Linux systems

25. © 2007 Demartek 25 Receive-side Scaling

26. © 2007 Demartek 26 TCP Offload Engine (TOE) ♦TCP Offload NIC handles all TCP/IP processing ♦Reduces CPU utilization on host server ♦Provides overall best performance ♦More expensive than server-class NICs with Receive-side Scaling

27. © 2007 Demartek 27 CPU Utilization Graphs Percent CPU Utilization Affect of NIC type on CPU Utilization

28. © 2007 Demartek 28 Switches, Jumbo Frames ♦Switches used for iSCSI should be Gigabit or better ♦When jumbo frames are enabled, must be enabled in the switch and all NICs connected to that switch, including iSCSI initiators and targets –Make sure all components are using the same jumbo frame size

29. © 2007 Demartek 29 Storage Technology Items ♦RAID type ♦Stripe sizes ♦Disk drive type ♦MPIO in the target ♦Caching

30. © 2007 Demartek 30 RAID Definitions ♦RAID – Redundant Array of Inexpensive Disks –RAID 0: Interleaving or “striping” data across two or more disks –RAID 1: Disk mirroring – same data written on two different disks –RAID 5: Data striping with parity across multiple disks –RAID 6: Data striping with double parity across multiple disks –RAID 1+0 or RAID 10: combine RAID 1 and RAID 0 Two variations: stripe first or mirror firstTwo variations: stripe first or mirror first

31. © 2007 Demartek 31 RAID Types Data: “ABCD” RAID 0 B DA C RAID 1 ABCD RAID 10 A C B D RAID 5 P1CB A D RAID 6 P1P2CBA D

32. © 2007 Demartek 32 Stripe Sizes ♦Size of data block that is striped across disk drives in a RAID group ♦I/O of larger than the stripe size causes the target to perform extra work to complete the I/O

33. © 2007 Demartek 33 Disk Drive Type ♦Enterprise class (SAS, SCSI, Fibre Channel) –Designed to be operated 7 x 24 x 365 –Dual internal processors (data transfer, physical motion) ♦Desktop class (SATA, ATA/IDE) –Largest capacities, best price/capacity ♦Notebook class –Low power consumption, slower ♦Consumer class –Very low power consumption and weight

34. © 2007 Demartek 34 Drive Type Chart DeviceEnterpriseDesktopNotebookConsumer Seek time 3 – 5 ms 8 – 10 ms 10 – 12 ms ~12 ms Xfer rate (MB/s)* 70 – 120 60 – 80 30 – 60 10 – 30 RPM (K) 10, 15 5.4, 7.2, 10 4.2, 5.4, 7.2 3.6, 4.2 CapacitiesLarge Very large MediumSmall Processors2111 Cmd. Queuing TCQNCQ-- Power need HighMediumLow Very low Warranty 5 years 3 – 5 years 1 – 3 years 1 year * Maximum device transfer rate from drive surface to buffer. Sustained rates are lower. This is not the same as the interface transfer rate.

35. © 2007 Demartek 35 Caching ♦Different iSCSI target solutions use different caching –Some are based on Windows Server platform –Some are based on Linux platform –Some use proprietary platform

36. © 2007 Demartek 36 Solutions Available Today ♦Most of the major disk storage vendors provide iSCSI solutions ♦Several iSCSI-only vendors are producing very good solutions ♦Wide variety of price, capacity, performance and advanced feature selection is available ♦Generally speaking, iSCSI storage vendors have done a good job of making tasks such as LUN provisioning very easy to accomplish

37. © 2007 Demartek 37 Microsoft iSCSI Best Practices ♦Use true server-class NIC ♦Separate iSCSI traffic from all other network traffic ♦Use enterprise-class managed switches

38. © 2007 Demartek 38 Performance ♦I have personally used iSCSI target solutions that: –Achieve greater than 30,000 IOPs (with two paths) –Achieve greater than 200 MB/sec (with two paths) ♦You can have great performance with iSCSI

39. © 2007 Demartek 39 Futures ♦10-Gigabit Ethernet –Currently used primarily in clustering and network trunking applications –Expect prices to drop, especially as PCI-Express becomes the dominant bus type in servers ♦Storage Management Software –Expect iSCSI target vendors to make more information available to centralized storage management software solutions ♦Microsoft iSCSI Deployment Paper – soon!

40. © 2007 Demartek 40 Recommendations ♦Make sure you have Scalable Networking Pack installed and use either Receive-side Scaling capable NICs or TCP Offload Engines ♦Take advantage of MPIO

41. © 2007 Demartek 41 References ♦Microsoft Storage www.microsoft.com/storage/ www.microsoft.com/storage/ ♦Microsoft iSCSI Storage www.microsoft.com/WindowsServer2003/technologies/storage/iscsi /default.mspx www.microsoft.com/WindowsServer2003/technologies/storage/iscsi /default.mspx www.microsoft.com/WindowsServer2003/technologies/storage/iscsi /default.mspx ♦Microsoft Storage Technical Articles and White Papers www.microsoft.com/windowsserversystem/storage/indextecharticle. mspx www.microsoft.com/windowsserversystem/storage/indextecharticle. mspx www.microsoft.com/windowsserversystem/storage/indextecharticle. mspx ♦Microsoft Scalable Networking Pack www.microsoft.com/technet/network/snp/default.mspx www.microsoft.com/technet/network/snp/default.mspx ♦Microsoft Exchange Solution Reviewed Program – Storage technet.microsoft.com/en-us/exchange/bb412165.aspx technet.microsoft.com/en-us/exchange/bb412165.aspx

42. © 2007 Demartek 42 Post-Presentation Survey ♦How many would be interested in a day-long, vendor-neutral, storage class? ♦What topics would be interesting to you in such a class?

43. © 2007 Demartek 43 Thank You Dennis Martin Demartek (303) 940-7575 Dennis @ demartek.com Dennis @ demartek.com www.demartek.com