Good DS SVC Best Practices January 11, 2011

Good DS8000 - SVC Best Practices January 11, 2011
Dominic Pruitt Brian Smith Jim Sedgwick Joanne will do an introduction here 9/18/2018

Introduction about SVC and DS8000 Reference configuration
Agenda Introduction about SVC and DS8000 Reference configuration DS8800 Standard Class SVC CF8 Cluster Configuration scaling considerations Some Changing Practices Brian And some alternative configurations where appropriate

SVC and DS8K DS8000 SVC SVC in Front of DS8000
IBM’s premier enterprise storage product Highest Performance, Enterprise-class reliability SVC IBM’s premier enterprise storage virtualization product Complete set of advanced function SVC in Front of DS8000 The best of both! With considerations for each! Brian We will be discussing DS8000 DS8100, DS8300, DS8700, DS8800(the newest model). And SVC

Best practices for performance and availability
Fewer, larger Mdisk Groups (MDGs / Storage Pools) Use more Mdisks in an MDG, not fewer. No single Mdisk MDGs! Spread Vdisks across all SVC Nodes Keep many, many arrays active! Use a lot of Ports into DS8K… usually 16 Use dual fabric! Use sophisticated multi-pathing software in the servers! Don’t overlook LVM Striping in the server OS! Don’t overlook High Availability and Disaster Recover Brian – here are some high level objectives for SVC and DS8K resources.. Spread the loads, use al the resources.. Our reference configuration doesn’t really deal with HA, DR...

Assumptions & Audience
Assumes some knowledge of SVC and DS8K VDisks (Volumes), MDisks, Managed Disk Groups (MDG/Storage Pool) Raid arrays, extent pools, host adapters Assumes some knowledge of SAN fabrics Dual fabrics Zoning concepts For IBMers, Business Partners, and Customers Those interested in SVC with DS8K Curious about the latest configuration recommendations SAN Volume Controller Best Practices and Performance Guidelines Redbook SG Published December 2008 Think of this webinar as an update Brian This is not a primer on SVC nor DS8K. It assumes some familiarity with both, and with SANs (Storage Area Networks – switches) There are other publications on this topic (SVC and DS8K best practice)… Much of it, primarily from SVC perspective, has been published in Redbook format: IBM Redbook publication SG247521:SAN Volume Controller Best Practices and Performance Guidelines”

Storage Pools, Extent Pools, MDGs
These are similar concepts… where volumes are allocated Storage Pools – SMI-S Extent Pools – DS8K Managed Disk Group (MDG) or Storage Pool – SVC VDisks (Volumes) are allocated in an SVC IO group, backed by an MDG MDisks are allocated in a DS8K Extent Pools. Two or more. MDisks are just DS8K LUNs (or DS8K Volumes) Fabrics and Zoning are the Entry Ports into all Storage Remember, the SVC is unaware of the physical configuration of the DS8000 Brian – but here is a high level introduction to some key terms. Volumes are units of storage that can be assigned to servers (hosts).. Like AIX, Windows, etc. Vdisks are changing terminology to “volumes”... Next, we will look at our reference configuration.. We aren’t saying it is the best or recommended configuration, But a starting point to tie together some principles for configuring SVC with any DS8K. Take it away, Jim!

Reference Configuration - High performance
DS full base frame Model 951 standard class 10 Gigapacks - 30 Arrays GB 15K RPM SAS (2.5” form factor) RAID 5 = 24.4 TiB SVC sees binary 64 GB cache (4x) 4-port Host Adapters SVC 2145-CF8 running v6.1 4-node cluster 24 GB cache per node No SSD TPC for Disk Monitor performance Jim

DS8800 Reference Configuration - Cache
64 GB cache Main reason: 2 GB write cache Experience - we often see low-level NVS-full conditions on demanding workloads on DS8000s with 1 GB write cache behind a SVC Speculation - DS8800 has faster plumbing (HAs, DAs, bus, etc.) SAS Array performance is similar to DS K RPM Arrays, give or take…. Conclusion - we think we’re going to see a greater need for DS8800 write cache 2 GB to 4 GB cache per TB - Rule of Thumb for Tier 1 DS8000 Performance & Tuning Redbook Jim

DS8800 Reference Logical Configuration
7+P 900GB 6+P+S 772GB 6+P+S 772GB DA2 DA3 DA1 DA0 Even Extent Pools DS8800 Logical Configuration for SVC One Rank per Extent Pool CEC balance 15 ‘even’ extent pool Ranks 15 ‘odd’ extent pool Ranks Host Adapters (HAs) Disk Magic 4 DS8800 HAs can go lots faster than a DS8800 base frame Never allow HAs to limit the performance of a enterprise-class storage subsystem solution HMC Ethernet CEC 3a1 3b1 1a1 1b1 2a2 2b2 0a1 0b1 2a1 2b1 1 3 2 Enclosure 0 (XI1) Slot 0 Slot 1 Slot 2 Device Adapter Slot 3 Slot 4 Slot 5 1 Enclosure 1 (XI2) Enclosure 2 (XI3) 2 3 Enclosure 3 (XI4) C1 C2 C3 C4 C5 C6 Port 0 (T0) Port 1 (T1) Port 2 (T2) Port 3 (T3) I0030 I0031 I0032 I0033 I0100 I0101 I0102 I0103 I0230 I0231 I0232 I0233 I0300 I0301 I0302 I0303 Jim Odd Extent Pools

DS8800 Reference Configuration - Volumes
7+P 900GB 6+P+S 772GB 6+P+S 772GB Managed Disk Group (MDG) Strategy SVC Managed Disk is ≤ 2 TB Volume size - One volume per Array 2 Managed Disk Groups (SVC admin decision) 14 volumes 900 GB each  MDG A 16 volumes 772 GB each  MDG B Major Consideration: Both managed disk groups have enough capacity to handle the member applications Two types of capacity: Performance & Data Storage Do not proceed until this is satisfied Plan for growth Each MDG has member MDs from each DS8800 DA Both MDGs have the same number of spindles - equal performance This is more an observation than a major consideration Each DS8800 volume is assigned to all 16 HA ports SVC can see every DS8800 Volume from each HA DA2 DA1 DA3 DA0 DA2

DS8000 Reference Configuration - Volumes
7+P 900GB 6+P+S 772GB 6+P+S 772GB DA2 SVC Managed Disk Group All Volumes in MDG are created on Arrays that are the same technology Same disk size (146 GB) Same disk speed (15K RPM SAS) Same RAID type (RAID-5) No exceptions How many Arrays in an SVC MDG Enough to satisfy the application workload requirements Disk Magic Enough to physically hold the application Minimum of 8 (allows greatest port workload spread in a IO Group) 4 is too few (old message - let’s get away from this) Balance across DA Pairs Balance across CECs Typically an application would reside in a single MDG OK for mult. applications to be in the same MDG Applications in multiple MDGs have availability considerations DA1 DA3 DA0 Jim DA2

SVC Reference Configuration – Performance & Data Placement
SVC CF8 nodes selected, current & fastest HW available for SVC software 4 node SVC cluster selected for availability (eliminate single point of failure) SSD not selected due to support in v6.1 (Easy Tier) and limited use in v5.1 4 Node SVC Cluster plus DS8800 is a nice reference configuration that is current Group hosts/applications By Performance objectives By Availability objectives Align applications to a single MDG / Storage Pool (unit of failure) Isolate critical applications from less critical applications By IO Group & MDG boundary Logic errors triggered by administration activity Configuration updates Storage provisioning Microcode updates Hardware topology changes Administration errors Dom – svc cpu is the performance bottleneck, dedicate svc compute resources for application workloads

DS8800/SVC Reference Configuration - Zones
SVC Cluster Zone (for intra-node comm) SVC to Storage Zones (goal to prevent ISL traffic) Can use VSAN or Traffic isolation in switch OS Dom Host zoning more complex, zone to port #s in SVC IO groups

SVC Reference Configuration - SVC Port to LUN Path Assignment
The SVC has simple multi-pathing from the each NODE in the cluster to the DS8000, this simple multi-pathing allows the SVC to talk to many disk controllers Upon LUN Discovery each SVC node will see all paths and pick a preferred port & path to use for a given LUN/MDisk Each subsequent LUN found in discovery will be assigned the next port & path, the SVC will pick the next port with the least number of MDisk paths assigned to it. For example SVC Node A Port 0 will see DS8000 LUN0 on HA0 port 0 and will use this path (it’s not the only one), repeat & overlap for each LUN/MDisk assigned to the SVC. SVC node port 1 -> LUN1 on HA0 port 1 … SVC node port 3 -> LUN15 on HA3 port 4 This simple multi-pathing algorithm spreads workload among as many resources as possible. Dom

MDisk Preferred Path & Port distributed across per Node resources at MDisk discovery NodeA - Port 0 NodeA - Port 1 NodeA - Port 2 NodeA - Port 3 Dom HA0P0 HA0P1 HA0P2 HA0P3 HA1P0 HA1P1 HA1P2 HA1P3 HA2P0 HA2P1 HA2P2 HA2P3 HA3P0 HA3P1 HA3P2 HA3P3 All LUNs presented through all ports on all HAs

MDisk0 MDisk1 MDisk2 MDisk3 MDisk9 MDisk14 NodeA - Port 0 NodeA - Port 1 NodeA - Port 2 NodeA - Port 3 Dom HA0P0 HA0P1 HA0P2 HA0P3 HA1P0 HA1P1 HA1P2 HA1P3 HA2P0 HA2P1 HA2P2 HA2P3 HA3P0 HA3P1 HA3P2 HA3P3 LUN0 LUN3 LUN9 LUN14

Multi-Rank Extent Pools in DS8K
Some Venerable “Best Practice” are being Challenged. In particular, the recommendation for single rank extent pools Reference configuration for DS8100 or DS8300 could be done with two Extent Pools One even, one odd pool Or four extent pools DS8800 does not yet support EasyTier. Let’s look at DS8100 example: Brian Having see the reference configuration and recommended “best practices” associated with it, We want to emphasize that no one configuration or set of practices works everywhere for all time. There are a lot of SVC/DS8100 and SVC/DS8300 and SVC/DS8700 configurations already out there. Most have single rank extent pools on the DS8K. But !!! Virtualization is not just the province of SVC… it comes in DS8K as well. Consider the possibilities in multi-rank extent pools. Let’s look at an example… it works for all models of DS8K. But let’s look at DS8100

Brian There are 6+P and 7+P in each pool. Ranks Balanced across DA Pairs Ranks Balanced across Servers. Volumes (SVC mdisks) will be allocated in a round-robin fashion across all the arrays in an extent pool.

Multi-Rank Extent Pools in DS8K Behind SVC
Advantages Virtual Mdisks all the same size, or… DS8K EasyTier – SSD mixed with FC or SATA drive Required for EasyTier !!! Automatic Exploitation of SSDs! See EasyTier Redpaper “Best Practice” is no alternative for thoughtful design Think about the workloads Think about the configuration Monitor both SVC and the DS8K SVC and DS8K virtualization make changes easier, less disruptive. Evolution of storage capabilities changing longstanding practices Brian Why might we depart from the traditional “single rank extent pool”? Let’s talk some more about EasyTier

EasyTier for SVC and for DS8000
There is EasyTier for SVC Identify external Mdisks to SVC as “SSDs” SVC manages the use of SSDs. There is EasyTier for DS8K SSD and FC drives in the same extent pools (even and odd pools) DS8K keeps appropriate extents (1GB each) on the SSDs. EasyTier manages extents Not necessarily the “HOT” extents… Rather, the appropriate extents… Not too much sequential activity, not too much write (writes exploit write cache) And, certainly not extents with great cache read hit percentage Brian You can see than manually managing an expensive resource like SSDs could be a challenge. Let EasyTier do it… available in some flavors now, and more to come. Watch this space! But let’s get back to some more traditional “best practice” discussion.

DS8000 Host Adapter Considerations
Never allow HAs to limit the performance of an enterprise-class storage subsystem solution Ports dedicated to SVC Monitor performance Balanced I/O across all ports serving SVC Response times around 1 ms. DS8100, DS8300, DS8700 (DS8000) HAs One HA can do around 350 MB/sec Typically, we end up using 2 ports per card Use one of top 2 ports Use one of bottom 2 ports DS8800 HAs are really fast One HA can do around 2000 MB/sec No considerations about what port to use How many HAs do I need? Use Disk Magic Consider the following table as a place to start for single SVC cluster configurations Jim Storage ≤16 Arrays 17-32 Arrays 33-48 Arrays >48 Arrays High Performance - most demanding DS port adapters 2 HAs - 8 ports 4 HAs - 16 ports 8 HAs - 16 ports DS8000 (DS DS8700) 4 HAs - 8 ports 16 HAs - 16 ports

Larger Drive Considerations - Volumes & MDGs
600 GB Drives 102.5 TB Still the ‘One Rank per Extent Pool’ discussion Create multiple volumes per Array Each set of volumes goes into different MDGs For now, SVC has a MD size limit of 2 TB Solution #1 - 3 MDGs Create 2TB volume on each array  MDG A (61.4 TB) Create 1173 GB Volumes on the 6+P Arrays  MDG B (18 TB) Create 1698 GB Volumes on the 7+P Arrays  MDG C (23.1 TB) Solution #2 - 4 MDGs Create 2 equal sized Volumes on each Array All of these messages apply for tiered storage e.g. 30 Arrays of 15K RPM & 30 Arrays of 10K RPM Deal with each tier separately 7+P 3698GB 7+P 3698GB 7+P 3698GB DA2 7+P 3698GB 7+P 3698GB 7+P 3698GB 7+P 3698GB 7+P 3698GB 6+P+S 3173GB DA1 6+P+S 3173GB 6+P+S 3173GB 6+P+S 3173GB 7+P 3698GB 7+P 3698GB 6+P+S 3173GB DA3 6+P+S 3173GB 6+P+S 3173GB 6+P+S 3173GB 7+P 3698GB Jim 7+P 3698GB 6+P+S 3173GB DA0 6+P+S 3173GB 6+P+S 3173GB 6+P+S 3173GB 7+P 3698GB 7+P 3698GB 6+P+S 3173GB DA2 6+P+S 3173GB 6+P+S 3173GB 6+P+S 3173GB

Disk Magic - DS8000 Utilization Considerations
How many Arrays do I need to meet my application performance requirements? How many HAs do I need for a full DS880 base frame? Customers Know this tool exists as a resource to your storage technical sales team For sale For IBMers Select 'Tools' Select 'Disk Magic' Follow instructions to download and install For Business Partners Search on 'Disk Magic' Follow instructions to download and install point Start Disk Magic Select ‘Storage Virtualization Wizard Project’ Hit ‘OK’ Jim Great to answer ‘what if?’ questions Customers - it’s for sale, contact me for additional information on all applicable fees, contracts, titles, taxes and additional red tape It is time for all tech sales to have this tool in their back pocket Complex projects or customer presentations - specialized Techline team - 2 week turnaround

SVC Performance Considerations
VDisk / Volume creation, round robin in IO group by default Number of MDisks in the MDG Most important attribute influencing performance Minimum of 8 MDisks in MDG to use all 8 ports in a IO group More MDisks in the MDG is better for transactional workloads Note: Increasing performance “potential” adversely increases impact boundary Cannot be avoided up to minimum performance requirements Rotational speed of the disks More significant than access density SSD considerations Measurement of benefit can be analyzed w/o SSD using SVC STAT tool Can use SSD today from backend storage Dom – Need to describe where the SSDs are – in the SVC or the DS8K… and how they are identified to the storage system as a whole .(bjs)

SVC Additional Considerations
Pay attention to VDisk / Volume extent distribution over time Make sure all MDisks in MDG / Storage Pool have an equal number of extents Group hosts/applications storage growth Rebalance extents after adding additional capacity (MDisks to MDGs) Script is available to accomplish this on Alphaworks Don’t overlook LVM striping! Use it to balance IO across SVC nodes Use fined grained LVM striping to compensate for larger SVC Extent sizes, especially for writes. Small MDG’s (<8 arrays) may limit IO performance Dom

SVC Scaling Considerations
Number of SVC Nodes Monitor CPU utilization on all nodes - less than 50% sustained Use TPC for Disk Number of DS8K Arrays Can add the number of LUNs (MDisks) to cluster anytime. 8Gb technology in DS8800 and SVC Use 8Gb technology everywhere if possible SVC I/O ports & node CPUs can get very busy. The move to faster SANS (8Gb) may mean bottlenecks will shift around Dom Let’s just say ALL NODEs… not all 4 nodes 50% -- why? port constrained workloads will shift to the SVC ???????? What dos this mean?

HA and DR Considerations
SVC Metro and Global Mirror configurations Additional overhead & performance impact on a per IO group basis Zoning impacts Target VDisks from same MDG due to heavy writes and cache partitioning VDisk mirroring Data Migration for MDGs of different extent sizes High availability configuration because write goes to 2 MDGs Can switch read primary at anytime Dom

Arrays and MDGs Common to put all DS8000 LUNs/MDisks in one big MDG
Best overall performance Lose one Array (very rare) Restore all applications from backups Recover at replication site Consider a most-loved, business critical application - dedicated MDG Increase availability Guaranteed performance - dedicated spindles (hear this all the time from DB folks) What do you do? Determine the performance requirements  Disk Magic  X Ranks Determine data storage requirements  Y Ranks Go with whichever is greater: X or Y Dom – unit of failure, cache partitioning,

Questions and Answers innovation that matters

Reference Section

Reference Information
Updated SSD capacity calculator IBM link: Partner link: Web lecture of sales teleconference IBM link: Partner link:

Reference Information – continued
SVC Best Practice Redbook EasyTier for DS8000 32

Disk Magic Objective - How many HAs do I need for a full DS880 base frame? Customers Know this tool exists For IBMers Select 'Tools' Select 'Disk Magic' Follow instructions to download and install For Business Partners Search on 'Disk Magic' Follow instructions to download and install point Start Disk Magic Select ‘Storage Virtualization Wizard Project’ Hit ‘OK’ Great to answer ‘what if?’ questions Customers - it’s for sale, contact your IBM sales team for additional information on all applicable fees, contracts, titles, taxes and additional red tape It is time for all tech sales to have this tool in their back pocket Complex projects or customer presentations - specialized Techline team - 2 week turnaround

Disk Magic Enter SVC cluster info Enter Disk Subsystem info
Hardware Type: SVC 2145-CF* (V5.1) Number of Node Pairs: 2 Hit ‘Next’ Enter Disk Subsystem info Type of DSSs: IBM DS8800 (4-way) Number of DSSs: 1 Total Capacity [GB]: Capacity Magic - binary capacity of full rack Enter Server info Number of Servers [Groups] to create: 1 Use wizard to select workload profiles: select this

Disk Magic Enter Workload info Hit Finish
Check Workload profile to use Select ‘OLTP (typical) from dropdown Or best choice Hit ‘Next’ Hit Finish

Disk Magic Customize DS8800 config Double click on DSS1
Select ‘Hardware Details’ System Memory (GB): 64 Fibre Host Adapters: 4 Use table for starters

Disk Magic Customize DS8800 config (cont) Select ‘Interfaces’ tab
Select ‘From Disk Subsystem’ tab Select Edit Make everything 8 Gb (max.) from dropdowns Host port ‘Count’: 16 Use table for starters Hit ‘OK’ Match up the ports from Servers for equal bandwidth Select ‘From Servers’ tab

Disk Magic Customize SVC config Double click on DSS1
Select ‘Interfaces’ tab Select ‘Fibre Links from Servers’ tab Match ‘Type’ and ‘Count’ to DS8800 Select ‘Server Workload’ tab

Disk Magic Solve the model Select ‘Server Workload’ tab
Hit ‘Solve’ for the default input workload onto the SVC The model will most likely successfully solve Iteratively increase the ‘I/O Rate’ number and hit ‘Solve’ until the model fails. Now back the ‘I/O Rate’ number down a little until the model solves again This is max. I/O workload this SVC/DS8800 configuration can sustain

Disk Magic - DS8800 Utilization
Look at DS8800 Utilizations Double click ‘DSS1’ again Select ‘Open Workload’ tab Select ‘Utilizations’ Observe ‘Highest Fibre HA Utilization’ Observe ‘Hightest Fibre Port Utilization’

Trademarks and Disclaimers
IBM Corporation All rights reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. Trademarks of International Business Machines Corporation in the United States, other countries, or both can be found on the World Wide Web at Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. The customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown may be engineering prototypes. Changes may be incorporated in production models.

Good DS SVC Best Practices January 11, 2011

Similar presentations

Presentation on theme: "Good DS SVC Best Practices January 11, 2011"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Good DS SVC Best Practices January 11, 2011

Similar presentations

Presentation on theme: "Good DS SVC Best Practices January 11, 2011"— Presentation transcript:

Similar presentations

About project

Feedback