Easier Platform Administration using SAS 9.4 Grid Option Sets SAS New South Wales User Group - Nov 2015 Andrew Howell ANJ Solutions Pty Ltd

Agenda Background What is a grid & why use it How it works Platform Computing (LSF) Running SAS clients on a grid Grid configuration Prior to SAS v9.4 SAS v9.4 - Grid Option Sets Examples Recommendations Wrap up References, questions, etc.

Why use Grid Computing? Maximise existing environment Workload management Parallel workload management Scalability High-availability environment Traditional client-server Gridded client-server

Examples Texas Tech University Investigate stock market triggers 10,000 samples, 10,000 simulations 100,000,000 sample data sets Single machine: 25 hours SAS/Grid (100 Windows nodes): 40 mins 2.7% of original runtime National Institute for Environmental Health Statistics Gene sample profiling 124,000 gene samples Single machine: 448 hours SAS/Grid (32 Linux nodes): 14.5 hours 3.2 % of original runtime

How does it work IBM Platform Computing Platform LSF Platform Process Manager Platform Grid Management Service Enterprise Grid Orchestrator SAS Grid Components Grid Manager Plug-in for SAS Mgt Console, Env Manager Platform RTM for SAS SAS Grid components IBM Platform Computing

Grid definitions Host: Each computer Cluster: All computers Job: Unit of work Queue: Waiting list Slot: Number of processes per host (By default, number of processor cores) Queue settings PRIORITY - jobs with higher priorities always take preference. QJOB_LIMIT - the maximum job slots queue can use. UJOB_LIMIT - the maximum job slots per user in a queue. CPULIMIT - a time limit applied to jobs. DISPATCH_WINDOW - the time window when jobs can be dispatched to a queue. FAIRSHARE - Adjusts priority based on workload to prevent resource hogging.

IBM Platform Computing lsb.queues file one of MANY LSF configuration files Platform RTM for SAS Begin Queue QUEUE_NAME = normal PRIORITY = 30 DESCRIPTION = default queue End Queue Begin Queue QUEUE_NAME = priority PRIORITY = 50 DESCRIPTION = high priority users End Queue Begin Queue QUEUE_NAME = night PRIORITY = 40 DISPATCH_WINDOW = (18:00-07:30) RUN_WINDOW = (18:00-08:00) DESCRIPTION = night time batch jobs End Queue

LSF Queue Configuration

Running Enterprise Guide on SAS Grid

Running Data Integration Studio on SAS Grid

Grid configuration prior to SAS v9.4 Apply default queue for a (gridded) Application Context Can override: Data Step Batch Server SASGSUB script

Grid configuration prior to SAS v9.4 Typical SAS Grid scenario: Multiple business units / departments Each BU with own application context with BU-specific configuration: - cores, batch windows, folder locations, etc. Each with own logical grid server, own queue But with same priority ! All have similar usage, but must be managed separately

Grid configuration in SAS v9.4 Same as previous versions.... but with more !! SAS Grid Option Sets New to SAS v9.4 Apply default SAS options & grid settings Can apply alternate SAS options & grid settings based on users, user groups and/or SAS applications

SAS Grid Option Sets Example: Give SAS Platform Administrators higher priority when running Enterprise Guide

SAS Grid Option Sets Other possibilities: User specific options: Reporting teams, data miners, etc. Client specific options: AMO, EG, EM, DI Studio, etc

Recommendations Support multiple business groups centrally Leave SASApp for SAS Solutions Credit Risk, etc Have a dedicated Application Context (SASUsr?) for “regular” SAS usage Enterprise Guide DI Studio Enterprise Miner, etc. When NOT to run on a grid Small programs Lots of workspace server stored processes?

Wrap up References SAS Support Documentation Grid Computing in SAS 9.4, Third Edition SAS/Connect Users Guide, Third Edition SAS Training Course SAS Grid Manager Administration (PCAD94) SAS White Paper “Grid Computing and SAS”, Merry Rabb & Cheryl Doninger SAS Global Forum Paper “Queues for Newbies - How to speak LSF in a SAS World”, Andrew Howell

Questions? Many thanks for your interest in SAS/Grid Andrew Howell ANJ Solutions Pty Ltd LinkedIn: