Download presentation
Presentation is loading. Please wait.
Published byDillon Dedman Modified over 10 years ago
1
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS® Grid 101: How It Can Modernize Your Existing SAS Environment Cheryl Doninger Glenn Horton R&D Director, SAS Software Manager, SAS Cheryl Doninger Glenn Horton R&D Director, SAS Software Manager, SAS
2
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. The Need to Modernize...
3
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Enterprise IT Challenges Need reduced complexity increased manageability SMP obsolete or at capacity Predict long term capacity and budget planning Need to grow incrementally Many users on single resource Frequent server crashes Higher availability flexibility to maintain and upgrade Increased data growth
4
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Enterprise Application Needs Interactive SAS sessions Interactive EG users Longer ad-hoc command line batch jobs Short ad-hoc interactive jobs Many users on single resource Parallel production jobs Production jobs w/ complex schedules or dependencies Increased data growth
5
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS Grid Manager Bar size 2048 SAS Grid Manager Distributed Enterprise Scheduling Multi-User Workload Balancing Parallelized Workload Balancing Distribute parallelized SAS workloads to a shared pool of resources. Distribute workloads to a shared pool of resources. Distribute jobs within workflows to a range of hosts. Optimize the Efficiency and Utilization of Computing Resources
6
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Three Paths to Grid Enabling SAS SAS Products and Solutions Automatic Parallel Code Generation SAS Program with parallel sub-tasks Add Grid Computing Program Header SAS Programs Create Sub-Tasks Using 4GL Syntax Add Grid Computing Program Header Create Workflow and Schedule It
7
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Storage Performance is Critical in a Grid Fiber Channel Switch Shared Storage Device(s) Grid Node n Grid Node 1 Shared File System Volume 1 Mounted Volume 1 Mounted Volume 1
8
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS DI Studio – Enterprise Scheduling
9
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS DI Studio – Multi-User Workload Balancing
10
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS DI Studio – Parallelized Workload Balancing
11
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS Enterprise Miner – Parallelized Workload Balancing
12
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Phased Approach Phase 1 Phase 3 Multi-User Jobs on the Grid Schedule Production Jobs Migrate Parallel Workloads Phase 2 minimize risk minimize change
13
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Workload Balancing - Interactive Save following to external file (c:\gpre.sas for example) options noconnectpersist; options noconnectwait; options metaserver='dnnnn'; options metaport=8561; %let rc=%sysfunc(grdsvc_enable(grid,resource=SASMain)); signon grid; rsubmit; Create new DMS key definition (F12 for example) gsubmit “%include ‘c:\gpre.sas’;”; rsubmit;
14
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Workload Balancing - Batch Download Unix script file from Scalability site./runsas.sh Where runsas.sh will do the following: create a temporary file write necessary grid and SAS/Connect syntax to file copy contents to temporary file submit contents of file and to the grid perform error checking and messaging in event of errors
15
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Deploy a SAS Program for Scheduling Bar size 2048
16
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Create a New Flow Bar size 2048
17
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Schedule the Flow Bar size 2048
18
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Parallelized Workload Balancing %let rc=%sysfunc(grdsvc_enable(_all_,resource=SASMain)); signon task1; rsubmit task1 wait=no; /* code to be remote submitted */ endrsubmit; signon task2; rsubmit task2 wait=no; /* code to be remote submitted */ endrsubmit; waitfor _all_ task1 task2; /* continue local execution */
19
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS Grid Application - Results The application Proc IML calculate Pairwise test statistic compare 2 matrices, 4600 columns, 40 rows 21,000,000 total comparisons The results serial execution 16:16 parallel grid execution over 5 cpus 2:38 nearly linear scalability
20
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. 20 Better Job Control Job names can be set in SAS code %let jnvar=FinanceSummary; %let jovar=queue=priority; %put %sysfunc(grdsvc_enable(_ALL_, server=SASApp; jobname=jnvar; jobopts=jovar)); signon t; Setting Name of Grid Jobs
21
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. 21 Better Job Control Setting Name of Grid Jobs - Results
22
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. 22 Better Job Control Job options can be set in SAS code %let jnvar=FinanceSummary; %let jovar=queue=priority; %put %sysfunc(grdsvc_enable(_ALL_, server=SASApp; jobname=jnvar; jobopts=jovar)); signon t; Setting Grid Job Options
23
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. 23 Better Job Control Job options can be set in SASMC Setting Grid Job Options
24
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. 24 Better Job Control Setting Grid Job Options - Results
25
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Queue Definitions Bar size 2048 Begin Queue QUEUE_NAME = priority PRIORITY = 40 DESCRIPTION = high priority users End Queue Begin Queue QUEUE_NAME = normal PRIORITY = 30 PREEMPTION = PREEMPTABLE[short] DESCRIPTION = default queue End Queue Begin Queue QUEUE_NAME = night PRIORITY = 30 DISPATCH_WINDOW = (18:00-07:30) RUN_WINDOW = (18:00–08:00) HOSTS = all ~host1 DESCRIPTION = night time batch jobs End Queue Begin Queue QUEUE_NAME = short PRIORITY = 35 PREEMPTION = PREEMPTIVE[normal] DESCRIPTION = for jobs of short duration End Queue
26
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. SAS Application SAS Grid Manager HOST A HOST B HOST C Normal Queue SAS Grid Manager – Load Balancing High Priority Queue Night Queue Short Queue
27
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Summary Centralized, shared environment Dynamic resource allocation to meet peak demand Policies and prioritization for use of resources Run large more complex analysis De-couple applications from infrastructure Ease maintenance of computing infrastructure Improve price/performance with commodity hardware Scale out cost effectively as needs grow Bar size 2048
28
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Want to Know More? Introducing the SAS ® Code Analyzer, Mon, 3:00 214A, Paper 006-2008 Using SAS/OR® and SAS® Grid Manager to Solve Optimization Problems on the Grid, Mon, 5:00 209, Paper 203-2008 Architecting a Finely Tuned SAS® Grid Solution, Tues, 8:00 214B, Paper 390-2008 Balancing the Load: SAS® Server Technologies for Scalability, Tues, 8:30 214B, Paper 391-2008 Data Integration in a Grid-Enabled Environment, Wed, 8:00 217B, Paper 133-2008 SAS® Grid 101: How It Can Modernize Your Existing SAS Environment, Wed, 11:00 Ballroom C1, Paper 046-2008
29
Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.