1. www.OASUS.ca “Come out of the desert of ignorance to the OASUS of knowledge” Grid Computing with SAS ® Foundation Statistics Canada SAS Technology Centre Systems Development Division Greg McLean

2. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS2Introduction  What is Grid Computing  Why use Grid Computing  When to use Grid Computing  Grid Computing using SAS (SAS\Connect ® )  Real uses of Grid Computing  “Generalized Grid Computing System”  Capital Stock Calculations  SAS Grid Manager  Questions / Comments

3. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS3 What is Grid Computing  Introduced in the academic research community  Pioneered by Dr. Ian Foster & Dr. Carl Kesselman (Globus Project for Grid Software)  Parallel / Distributed Processing  Leverage existing hardware  Primary Goals:  Reduce overall elapsed processing time

4. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS4 What is Grid Computing  Utility Grid  Compute Grid  Multiple users that require processing  Multiple machines available to process  Dynamic allocation of process to available machine  Task that can be decomposed into sub-units  Sub-units dynamically allocated to available machines  Sub-units able to run in parallel

5. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS5 What is Grid Computing Serial vs. Parallel Processing Sort North Sort South Merge Both iMac Sort North Sort South Merge Both Results

6. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS6 What is Grid Computing Sort North Sort South Merge Both Serial vs. Parallel Processing Sort South Merge Both iMac iMac Sort North Sort South Sort North Sort Results Sort Results iMac Merge Both Results

7. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS7 Why use Grid Computing  Budget constraints  Higher volume of Data  Tighter processing schedules  Idle processing power of existing hardware

8. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS8 When to use Grid Computing  Applications requiring hours / days to process  Applications that are more processing intensive  Applications that can be decomposed into sub-tasks

9. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS9 Grid Computing with SAS  MP\Connect (Multi-Processing)  SAS\Connect  Introduced in SAS Version 8  Allows Asynchronous Processing  Multiple Machines / Multi-Processor Machine

10. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS10 SAS\Connect Overview iMac Machine X Data

11. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS11 SAS\Connect Overview iMac iMaciMac iMac

12. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS12 Real uses of Grid Computing  Optimization in a grid of PC Laptops Case 1  60 laptops (266 - 400 Mhz)  600 Sales Territories 87% Improvement 92% Improvement Total Elapsed Time

13. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS13 Real uses of Grid Computing  NIEHS – National Institute of Environmental Health Sciences Case 2 - NIEHS  Data Analysis of a toxicogenomics microarray study  Processing 1.4 million combinations of genes  Two environments used for grid processing

14. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS14 Real uses of Grid Computing Case 2 – NIEHS - Linux Environment 97% Improvement Total Elapsed Time  32 node Linux cluster running Mosix  1 Ghz Intel P3 processors (1 Gig / processor, 100 Mb backplane)

15. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS15 Real uses of Grid Computing Case 2 – NIEHS - Heterogeneous Grid 99% Improvement Total Elapsed Time  100 nodes running mixture of W2K, WXP, variety of Unix OS’s  Combination of SAS v8 and SAS v9 on nodes

16. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS16 Generalized Grid Computing System

17. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS17 Generalized Grid Computing System

18. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS18 Generalized Grid Computing System

19. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS19 Generalized Grid Computing System

20. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS20 Generalized Grid Computing System

21. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS21 Generalized Grid Computing System

22. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS22 Generalized Grid Computing System

23. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS23 Generalized Grid Computing System

24. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS24 Grid Computing at STC Capital Stock Calculation 83% Improvement Total Elapsed Time  8 desktop PCs (1.5 Ghz, 1 Gig RAM)  100 Mb/Sec backplane  Using “Generalized Grid Computiong System” (prototype)

25. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS25 SAS Grid Manager  “Plug-in” integrated into SAS Management Console  Visual interface for monitoring and management of: JobsJobs UsersUsers QueuesQueues

26. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS26 SAS Grid Manager Features  Dynamic Load Balancing  Resource Assignment  Management & Monitoring  Job Priority  Termination Management

27. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS27 SAS Grid Manager Usage  SAS Data Integration Studio 3.3  SAS Enterprise Miner 5.2  Stored Process used by other BI Clients  Foundation SAS (with SAS Meta-Data Server)

28. www.OASUS.ca Friday, September 19, 2008Grid Computing With SAS28 Questions / Comments StatisticsStatistiqueCanada Greg McLean Project Leader - SAS Technology Centre System Development Division R.H. Coats Building, 14th Floor, Section Q Ottawa, Ontario, CanadaK1A 0T6 (613) 951-2396 Fax (613) 951-0607 greg.mclean@statcan.gc.ca Canada