Doug Haigh SAS Institute The Top Four User Requested Grid Features Delivered With SAS Grid Manager 9.4 Doug Haigh SAS Institute Copyright © 2010, SAS Institute Inc. All rights reserved.

Summary of New Features Option Sets Workspace Servers launched using the grid Logging SASGSUB enhanced wait #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Grid Options Sets

Managing users and grid applications in 9.3 Specific users using specific applications needing specific grid options + One set of grid options per SAS application server context = Multiple SAS application server contexts #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

An example… Start with two departments Each department expands to use 2 grid applications Another department is added using the same grid applications A new application is added and used by each department #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

What is needed SASGSUB DIS Risk DIS Finance SASGSUB SAS Options: -memsize 256 Resources: <none> Grid Options: queue=normal DIS Grid Options DIS Finance SAS Options: -memsize 0 Resources: GSUB Grid Options: queue=night SASGSUB OPTIONS SASGSUB #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Same deployment in 9.4 Using Grid Options Sets #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Grid Options Sets Grouping of options commonly used by grid Mapped in a grid server to a grid application for a user Can be mapped multiple times Override default options Grouping of SAS Options Required Resources Grid Options Associated with A specific user or group A specific SAS client application A specific logical grid server Can be associated multiple times Can be used by multiple logical grid servers Can be used by multiple users Can be used by multiple SAS client applications #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Logical Grid Server Metadata Default options #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Grid Options Set Mapping Wizard Where grid options sets are managed Options Sets can be mapped to a user or a group. #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Grid Options Set Editor #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

How it is used New gridoptset option on grdsvc_enable grdsvc_enable(_All_, "server=SASApp; gridoptset='MyAppGridOpts'") Client applications that generate grid code Client applications that submit to grid Object Spawner Apps that generate code: EM & DIS If only one grid option set defined, it is used by default. Apps that submit to grid: SASGSUB Can use GRIDOPTSET to override if more than one. #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Grid-launched Workspace Servers

In 9.3, using the grid in EG Workspace Server EG EG/AMO Grid macros Workspace server showed up in SASMC, but not RTM Grid server showed up in RTM, but not SASMC #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Now in 9.4… Workspace Server EG GRID -Grid policy applied to starting of workspace server -Server side configuration - can be used for all workspace servers -Application may be able to use Grid Options Sets -Workspace server shows up in SASMC and RTM -Eliminates RMTWORK, GRIDWORK and SASWORK related issues #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

New workspace server load balancing option 1 2 You can either do grid launched workspace servers or grid directed load balancing, but not both for the same app server #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Logging Enhancements

In 9.3, debugging involved… Different debug options TCPDEBUG CNTDEBUG SAS_GRID_DEBUG STDOUTPUT VERBOSE SAS Logging Facility Different debug output TCPDFILE CNTDFILE SAS LOG SPAWNER.DAT SASGRID.STDOUT SASGRID.STDERR SAS Logging Facility #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Now in 9.4, debugging is standardized SAS Logging Facility SASGRID script log file Used in SAS/CONNECT SAS/CONNECT Spawner SAS/SECURE SAS Grid Manager TCP/IP SAS Grid Manager Client Utility (SASGSUB) SASGRID script log file Always created Automatically deleted if SAS runs OK #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

SASGSUB Wait Enhancements

In 9.3, waiting for grid job using SASGSUB sasgsub –gridsubmitpgm MyProgram.sas –gridwait <parse output to get job-id & job-dir> sasgsub –gridgetresults <job-id> <copy log & output from job-dir> <parse job.info file for job rc> <return job rc as script rc> Lots of parsing No timeout Lots of copying & file manipulation #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Now in 9.4, the script is… sasgsub –gridsubmitpgm MyProgram.sas –gridwaitresults –gridwaittimeout 3600 rc=$? (or rc=%ERRORLEVEL% for Windows) Used by enterprise schedulers #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Summary of New Features Option Sets Workspace Servers launched using the grid Logging SASGSUB enhanced wait #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

#SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

How does it work? Resulting Option Values SAS Options -DBCS -memsize 0 Default Values SAS Options -memsize 256M -DBCS Required Resources SASApp Grid Options queue=normal Options Set Values SAS Options -memsize 0 Required Resources Teradata Grid Options queue=priority Merged Option Values SAS Options -memsize 256M –DBCS –memsize 0 Required Resources SASApp Teradata Grid Options queue=normal queue=priority Resulting Option Values SAS Options -DBCS -memsize 0 Required Resources SASApp Teradata Grid Options queue=priority #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Where Options Sets live #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Application Names in Mapping Wizard Name of folder in /System/Applications Configuration set property indicating application was grid capable #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Grid Workspace Servers & Options Sets For Object Spawner to get Options Set, it needs client application name New client connection (6) accepted from server port 8594 for user ... for APPNAME=SAS Data Integration Studio. Application name must match folder name #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

What about Old Applications? Create a folder under /System/Applications Set isGridCapable keyword If name is different, set name on isGridCapable keyword isGridCapable:SAS Enterprise Miner Mid-Tier isGridCapable:*SAS Enterprise Miner isGridCapable:SAS Enterprise Miner* isGridCapable:*SAS Enterprise Miner* <app_Name> in /System/Applications/<app_Name> is what shows up in mapping wizard #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Keywords #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.

Helping the match-making process Make sure application sends APPNAME to spawner Make sure APPNAME matches Folder name isGridCapable pattern Make sure a mapping exists for app server and user #SASGF13 Copyright © 2013, SAS Institute Inc. All rights reserved.