Enabling Local and Remote High Performance Backups Shyamsundar R Software Engineer, SMS Novell, Inc. Vijai Babu Madhavan Software Engineer, SMS Novell, Inc

© March 9, 2004 Novell Inc. 2 one Net: Information without boundaries…where the right people are connected with the right information at the right time to make the right decisions. The one Net vision Novell exteNd ™ Novell Nsure ™ Novell Nterprise ™ Novell Ngage SM : : : :

© March 9, 2004 Novell Inc. 3 The one Net vision Novell Nterprise is an innovative family of products which gives you the power to enable and manage the constant interaction of people with your business systems — regardless of who they are or where they are. Novell Nterprise ™ Novell exteNd ™ Novell Nsure ™ Novell Nterprise ™ Novell Ngage SM : : : :

© March 9, 2004 Novell Inc. 4 Agenda Technology overview SMS technology advancements Troubleshooting backup on NetWare Making the most out of your storage Related technology

© March 9, 2004 Novell Inc. 5 Backups Backup is a second class citizen…

© March 9, 2004 Novell Inc. 6 Backup data flow TSAFS * Backup Engine File Systems Local backup data flow

© March 9, 2004 Novell Inc. 7 Backup data flow TSAFS Backup Engine File Systems * Local backup data flow

© March 9, 2004 Novell Inc. 8 Backup data flow TSAFS Backup Engine File Systems * Local backup data flow

© March 9, 2004 Novell Inc. 9 Backup data flow TSAFS Backup Engine File Systems * Local backup data flow

© March 9, 2004 Novell Inc. 10 Backup data flow TSAFS Backup Engine File Systems * Local backup data flow

© March 9, 2004 Novell Inc. 11 Backup data flow SMDR Backup Engine SMDR Network * TSAFS Backup Engine File Systems Remote backup data flow TSAFS File Systems

© March 9, 2004 Novell Inc. 12 Backup data flow SMDR Backup Engine SMDR Network * Remote backup data flow TSAFS File Systems TSAFS Backup Engine File Systems

© March 9, 2004 Novell Inc. 13 Backup data flow SMDR Backup Engine SMDR Network * Remote backup data flow TSAFS File Systems TSAFS Backup Engine File Systems

© March 9, 2004 Novell Inc. 14 Backup data flow SMDR Backup Engine SMDR Network * Remote backup data flow TSAFS File Systems TSAFS Backup Engine File Systems

© March 9, 2004 Novell Inc. 15 Backup data flow SMDR Backup Engine SMDR Network * Remote backup data flow TSAFS File Systems Backup Engine TSAFS File Systems

16 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus Disk I/O (PCI) bus HBA Memory *

17 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

18 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

19 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

20 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

21 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

22 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

23 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

24 Local backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge Storage (SCSI) Bus I/O (PCI) bus HBA Memory * Disk

25 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory Network Network Controller Storage (SCSI) Bus Disk *

26 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus Disk

27 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus Disk

28 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus Disk

29 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus Disk

30 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus Disk

31 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus Disk

32 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus

33 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory Network Network Controller Storage (SCSI) Bus *

34 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus

35 Remote backup data flow: Systemic view CPU-memory bus Cac he CP U Bridge I/O (PCI) bus HBA Memory * Network Network Controller Storage (SCSI) Bus

© March 9, 2004 Novell Inc. 36 Available network bandwidth Ethernet wire capacity in theory 100 Mbps = Mb/s ~= 676 MB/min – Accommodating space loss due to protocol headers – Accommodating time loss due to inter-frame gap 1000 Mbps ~= 6755 MB/min Practical dedicated Ethernet wire capacity 100 Mbps – ~656 MB/min – 97% 1000 Mbps – ~6417 MB/min – 95% “Ideal bulk transfer applications should be bandwidth bound”

© March 9, 2004 Novell Inc. 37 Available network bandwidth Ethernet wire capacity in theory 100 Mbps = Mb/s ~= 676 MB/min – Accommodating space loss due to protocol headers – Accommodating time loss due to interframegap 1000 Mbps ~= 6755 MB/min Practical dedicated Ethernet wire capacity 100 Mbps – ~656 MB/min – 97% 1000 Mbps – ~6417 MB/min – 95% “Ideal bulk transfer applications should be bandwidth bound”

© March 9, 2004 Novell Inc. 38 Agenda Technology overview SMS technology advancements Troubleshooting backup on NetWare Making the most out of your storage Related technology

© March 9, 2004 Novell Inc. 39 How fast can remote backups go? We can do 1TB in 3.5 Hours on the wire at 4.8GB/Min, with the streaming SMDR prototype

© March 9, 2004 Novell Inc. 40 Streaming SMDR prototype results “Remote matching local performance...” Setup#1 Setup#2 Setup# Streaming Prototype Local Streaming Prototype Local % Difference Setup #1: 2GB SYS volume with an average file size of 27KB Setup #2: 2GB SYS volume with an average file size of 40KB Setup #3: 6GB volume with 6000 files each being 1MB in size

© March 9, 2004 Novell Inc. 41 Streaming SMDR prototype results “Remote matching local performance...” Setup#1 Setup#2 Setup# Streaming Prototype Local Streaming Prototype Local % Difference Setup #1: 2GB SYS volume with an average file size of 27KB Setup #2: 2GB SYS volume with an average file size of 40KB Setup #3: 6GB volume with 6000 files each being 1MB in size

© March 9, 2004 Novell Inc. 42 Streaming SMDR prototype results “Remote matching local performance...” Setup#1 Setup#2 Setup# Streaming Prototype Local Streaming Prototype Local % Difference Setup #1: 2GB SYS volume with an average file size of 27KB Setup #2: 2GB SYS volume with an average file size of 40KB Setup #3: 6GB volume with 6000 files each being 1MB in size

© March 9, 2004 Novell Inc. 43 Streaming SMDR prototype results Improvements over existing SMDR

© March 9, 2004 Novell Inc. 44 Streaming SMDR prototype results “Improved remote restore performance”

© March 9, 2004 Novell Inc. 45 How Streaming SMDR works Alleviates latency effects on throughput by anticipation Keeps the network pipe full Better network bandwidth utilization Read ahead, lazy write Decouples the interface from the implementation

© March 9, 2004 Novell Inc. 46 Who gets the benefit Applications that use SMDR for remote backups Data movement applications Novell migration wizard Novell server consolidation utility Novell DFS volume move/volume split Commercial copy applications

© March 9, 2004 Novell Inc. 47 SMDR: Features Existing Features Provides location transparency Abstracts communication details, supports both TCP/IP and IPX/SPX Advertisement/Name resolution Synchronous RPC model Recent Features Multiple IP address support DNS support for name resolution Policy based discovery/name resolution mechanism order iManager and server console based configuration

© March 9, 2004 Novell Inc. 48 SMDR: Features Existing Features Provides location transparency Abstracts communication details, supports both TCP/IP and IPX/SPX Advertisement/Name resolution Synchronous RPC model Recent Features Multiple IP address support DNS support for name resolution Policy based discovery/name resolution mechanism order iManager and server console based configuration

© March 9, 2004 Novell Inc. 49 TSAFS: Features Scalability Serial nature of backup does not limit scalability Re-architected TSA to scale with disk configurations Manageability Improved manageability, helps identify performance sweet spots iManager and server console based configuration Availability Cluster enabled, supports continuation of backup after a fail- over/fail-back Not just with NCS, even with server restarts Ships with NetWare ® 6.5, NetWare ® 6.0 SP4

© March 9, 2004 Novell Inc. 50 TSAFS: Features Scalability Serial nature of backup does not limit scalability Re-architected TSA to scale with disk configurations Manageability Improved manageability, helps identify performance sweet spots iManager and server console based configuration Availability Cluster enabled, supports continuation of backup after a fail- over/fail-back Not just with NCS, even with server restarts Ships with NetWare ® 6.5, NetWare ® 6.0 SP4

© March 9, 2004 Novell Inc. 51 TSAFS: Features Scalability Serial nature of backup does not limit scalability Re-architected TSA to scale with disk configurations Manageability Improved manageability, helps identify performance sweet spots iManager and server console based configuration Availability Cluster enabled, supports continuation of backup after a fail- over/fail-back Not just with NCS, even with server restarts Ships with NetWare ® 6.5, NetWare ® 6.0 SP4

© March 9, 2004 Novell Inc. 52 Sample TSAFS test results Setup #1: Ultra 160 controller with 4 disks using RAID 5, with PCI-X 64bit 100 MHz Setup #2: Ultra 160 controller with 6 disks using RAID 0, with PCI-X 64bit 100 MHz TSAFS improvements and tweaking effects

© March 9, 2004 Novell Inc. 53 Sample TSAFS test results TSAFS improvements and tweaking effects Setup #1: Ultra 160 controller with 4 disks using RAID 5, with PCI-X 64bit 100 MHz Setup #2: Ultra 160 controller with 6 disks using RAID 0, with PCI-X 64bit 100 MHz

© March 9, 2004 Novell Inc. 54 Agenda Technology overview SMS technology advancements Troubleshooting backup on NetWare Making the most out of your storage Related technology

© March 9, 2004 Novell Inc. 55 General Guidelines Rules of Thumb Eliminate one component at a time Try and eliminate broader sub systems first – For e.g., target the disk or network first Use tools DPMETERM TSATEST NetWare Remote Manager (NRM) iManager and Server Console

© March 9, 2004 Novell Inc. 56 General Guidelines Rules of Thumb Eliminate one component at a time Try and eliminate broader sub systems first – For e.g., target the disk or network first Use tools DPMETERM TSATEST NetWare Remote Manager (NRM) iManager and Server Console

© March 9, 2004 Novell Inc. 57 Troubleshooting demo Establish a baseline TSATEST Backup application Can it be improved? DPMETERM Can we go beyond this? Using NRM Using TSATEST Tweaking TSAFS, ReadThreadsPerJob and ReadBufferSize

© March 9, 2004 Novell Inc. 58 Troubleshooting demo Establish a baseline TSATEST Backup application Can it be improved? DPMETERM Can we go beyond this? Using NRM Using TSATEST Tweaking TSAFS, ReadThreadsPerJob and ReadBufferSize

© March 9, 2004 Novell Inc. 59 Troubleshooting demo Establish a baseline TSATEST Backup application Can it be improved? DPMETERM Can we go beyond this? Using NRM Using TSATEST Tweaking TSAFS, ReadThreadsPerJob and ReadBufferSize

© March 9, 2004 Novell Inc. 60 Troubleshooting SMDR issues Use tools for performance issues PERFTEST NETMON Other packet capture/analysis utilities Discovery/Name Resolution issues SLP SMDR NEW SMDR CONN “Ensure simultaneous remote backups do not flood your network”

© March 9, 2004 Novell Inc. 61 Troubleshooting SMDR issues Use tools for performance issues PERFTEST NETMON Other packet capture/analysis utilities Discovery/Name Resolution issues SLP SMDR NEW SMDR CONN “Ensure simultaneous remote backups do not flood your network”

© March 9, 2004 Novell Inc. 62 Troubleshooting SMDR issues Use tools for performance issues PERFTEST NETMON Other packet capture/analysis utilities Discovery/Name Resolution issues SLP SMDR NEW SMDR CONN “Ensure simultaneous remote backups do not flood your network”

© March 9, 2004 Novell Inc. 63 Troubleshooting TSAFS issues TSAFS debug log TSAFS /SmsDebug= /SmsDebug2= Server memory usage TSAFS /CacheMemoryThreshold= Other factors Competing software (Compression/Anti-Virus) CPU usage/Busy threads, NRM I/O statistics

© March 9, 2004 Novell Inc. 64 Troubleshooting TSAFS issues TSAFS debug log TSAFS /SmsDebug= /SmsDebug2= Server memory usage TSAFS /CacheMemoryThreshold= Other factors Competing software (Compression/Anti-Virus) CPU usage/Busy threads, NRM I/O statistics

© March 9, 2004 Novell Inc. 65 Troubleshooting TSAFS issues TSAFS debug log TSAFS /SmsDebug= /SmsDebug2= Server memory usage TSAFS /CacheMemoryThreshold= Other factors Competing software (Compression/Anti-Virus) CPU usage/Busy threads, NRM I/O statistics

© March 9, 2004 Novell Inc. 66 Agenda Technology overview SMS technology advancements Troubleshooting backup on NetWare Making the most out of your storage Related technology

67 Power of aggregation CPU-memory bus D PCI bus DDDD DDDDD DDDDD DDDDD DDDDD DDDDD DDDDD DDDDD DDDDD DDDDD DDDDD DDDDD SCSI/RAID Controller SCSI Controller PCI bus... PCI bus... Fibre Channel HBA

© March 9, 2004 Novell Inc. 68 Suggestions on exploiting bandwidth Create parallel I/O paths for high performance Load balance peer-to-peer PCI buses – Distribute your storage and network controllers Use multiple channels per HBA and stripe across them Use technology to suit performance needs 1Gbps networks are becoming common, 4Gbps cards are out in the market, 10Gbps is close to reality Mix and match with care Do not use Ultra 320 disks with an Ultra 160 SCSI controller Performance is limited by the lowest bandwidth component in your data pipe Design your storage configuration for high availability as well

© March 9, 2004 Novell Inc. 69 Suggestions on exploiting bandwidth Create parallel I/O paths for high performance Load balance peer-to-peer PCI buses – Distribute your storage and network controllers Use multiple channels per HBA and stripe across them Use technology to suit performance needs 1Gbps networks are becoming common, 4Gbps cards are out in the market, 10Gbps is close to reality Mix and match with care Do not use Ultra 320 disks with an Ultra 160 SCSI controller Performance is limited by the lowest bandwidth component in your data pipe Design your storage configuration for high availability as well

© March 9, 2004 Novell Inc. 70 Suggestions on exploiting bandwidth Create parallel I/O paths for high performance Load balance peer-to-peer PCI buses – Distribute your storage and network controllers Use multiple channels per HBA and stripe across them Use technology to suit performance needs 1Gbps networks are becoming common, 4Gbps cards are out in the market, 10Gbps is close to reality Mix and match with care Do not use Ultra 320 disks with an Ultra 160 SCSI controller Performance is limited by the lowest bandwidth component in your data pipe Design your storage configuration for high availability as well

© March 9, 2004 Novell Inc. 71 Suggestions on exploiting bandwidth Create parallel I/O paths for high performance Load balance peer-to-peer PCI buses – Distribute your storage and network controllers Use multiple channels per HBA and stripe across them Use technology to suit performance needs 1Gbps networks are becoming common, 4Gbps cards are out in the market, 10Gbps is close to reality Mix and match with care Do not use Ultra 320 disks with an Ultra 160 SCSI controller Performance is limited by the lowest bandwidth component in your data pipe Design your storage configuration for high availability as well

© March 9, 2004 Novell Inc. 72 Agenda Technology overview SMS technology advancements Troubleshooting backup on NetWare Making the most out of your storage Related technology

© March 9, 2004 Novell Inc. 73 Related Technology File Copy On Write Permits open file backup Pool Snapshots Point in time data snapshots Freeze/Thaw Permits data consistency across compliant applications Versioning User managed versioning of files, easing restores

© March 9, 2004 Novell Inc. 74 Related Technology File Copy On Write Permits open file backup Pool Snapshots Point in time data snapshots Freeze/Thaw Permits data consistency across compliant applications Versioning User managed versioning of files, easing restores

© March 9, 2004 Novell Inc. 75 Related Technology File Copy On Write Permits open file backup Pool Snapshots Point in time data snapshots Freeze/Thaw Permits data consistency across compliant applications Versioning User managed versioning of files, easing restores

© March 9, 2004 Novell Inc. 76 Related Technology File Copy On Write Permits open file backup Pool Snapshots Point in time data snapshots Freeze/Thaw Permits data consistency across compliant applications Versioning User managed versioning of files, easing restores

© March 9, 2004 Novell Inc. 77 SMS on Linux Location Transparency Enterprise scale, logical backup Multiple file system support Rich meta-data support

© March 9, 2004 Novell Inc. 78 SMS: In Summary One-stop Network wide Storage Management Solution High Performance, Scalable Extensible solution Multi-Latency/Bandwidth environments True backup virtualization

© March 9, 2004 Novell Inc. 79 Backups Backup is a second class citizen… Or is it?

General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. Novell, Inc., makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, Novell, Inc., reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.