“Come out of the desert of ignorance to the OASUS of knowledge” Grid Computing with SAS ® Foundation Statistics Canada SAS Technology Centre Systems Development Division Greg McLean

Friday, September 19, 2008Grid Computing With SAS2Introduction  What is Grid Computing  Why use Grid Computing  When to use Grid Computing  Grid Computing using SAS (SAS\Connect ® )  Real uses of Grid Computing  “Generalized Grid Computing System”  Capital Stock Calculations  SAS Grid Manager  Questions / Comments

Friday, September 19, 2008Grid Computing With SAS3 What is Grid Computing  Introduced in the academic research community  Pioneered by Dr. Ian Foster & Dr. Carl Kesselman (Globus Project for Grid Software)  Parallel / Distributed Processing  Leverage existing hardware  Primary Goals:  Reduce overall elapsed processing time

Friday, September 19, 2008Grid Computing With SAS4 What is Grid Computing  Utility Grid  Compute Grid  Multiple users that require processing  Multiple machines available to process  Dynamic allocation of process to available machine  Task that can be decomposed into sub-units  Sub-units dynamically allocated to available machines  Sub-units able to run in parallel

Friday, September 19, 2008Grid Computing With SAS5 What is Grid Computing Serial vs. Parallel Processing Sort North Sort South Merge Both iMac Sort North Sort South Merge Both Results

Friday, September 19, 2008Grid Computing With SAS6 What is Grid Computing Sort North Sort South Merge Both Serial vs. Parallel Processing Sort South Merge Both iMac iMac Sort North Sort South Sort North Sort Results Sort Results iMac Merge Both Results

Friday, September 19, 2008Grid Computing With SAS7 Why use Grid Computing  Budget constraints  Higher volume of Data  Tighter processing schedules  Idle processing power of existing hardware

Friday, September 19, 2008Grid Computing With SAS8 When to use Grid Computing  Applications requiring hours / days to process  Applications that are more processing intensive  Applications that can be decomposed into sub-tasks

Friday, September 19, 2008Grid Computing With SAS9 Grid Computing with SAS  MP\Connect (Multi-Processing)  SAS\Connect  Introduced in SAS Version 8  Allows Asynchronous Processing  Multiple Machines / Multi-Processor Machine

Friday, September 19, 2008Grid Computing With SAS10 SAS\Connect Overview iMac Machine X Data

Friday, September 19, 2008Grid Computing With SAS11 SAS\Connect Overview iMac iMaciMac iMac

Friday, September 19, 2008Grid Computing With SAS12 Real uses of Grid Computing  Optimization in a grid of PC Laptops Case 1  60 laptops ( Mhz)  600 Sales Territories 87% Improvement 92% Improvement Total Elapsed Time

Friday, September 19, 2008Grid Computing With SAS13 Real uses of Grid Computing  NIEHS – National Institute of Environmental Health Sciences Case 2 - NIEHS  Data Analysis of a toxicogenomics microarray study  Processing 1.4 million combinations of genes  Two environments used for grid processing

Friday, September 19, 2008Grid Computing With SAS14 Real uses of Grid Computing Case 2 – NIEHS - Linux Environment 97% Improvement Total Elapsed Time  32 node Linux cluster running Mosix  1 Ghz Intel P3 processors (1 Gig / processor, 100 Mb backplane)

Friday, September 19, 2008Grid Computing With SAS15 Real uses of Grid Computing Case 2 – NIEHS - Heterogeneous Grid 99% Improvement Total Elapsed Time  100 nodes running mixture of W2K, WXP, variety of Unix OS’s  Combination of SAS v8 and SAS v9 on nodes

Friday, September 19, 2008Grid Computing With SAS16 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS17 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS18 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS19 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS20 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS21 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS22 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS23 Generalized Grid Computing System

Friday, September 19, 2008Grid Computing With SAS24 Grid Computing at STC Capital Stock Calculation 83% Improvement Total Elapsed Time  8 desktop PCs (1.5 Ghz, 1 Gig RAM)  100 Mb/Sec backplane  Using “Generalized Grid Computiong System” (prototype)

Friday, September 19, 2008Grid Computing With SAS25 SAS Grid Manager  “Plug-in” integrated into SAS Management Console  Visual interface for monitoring and management of: JobsJobs UsersUsers QueuesQueues

Friday, September 19, 2008Grid Computing With SAS26 SAS Grid Manager Features  Dynamic Load Balancing  Resource Assignment  Management & Monitoring  Job Priority  Termination Management

Friday, September 19, 2008Grid Computing With SAS27 SAS Grid Manager Usage  SAS Data Integration Studio 3.3  SAS Enterprise Miner 5.2  Stored Process used by other BI Clients  Foundation SAS (with SAS Meta-Data Server)

Friday, September 19, 2008Grid Computing With SAS28 Questions / Comments StatisticsStatistiqueCanada Greg McLean Project Leader - SAS Technology Centre System Development Division R.H. Coats Building, 14th Floor, Section Q Ottawa, Ontario, CanadaK1A 0T6 (613) Fax (613) Canada