1. Multiprocessing with SAS ® Software Now Bill Fehlner, Kathleen Wong, Kifah Mansour SAS Toronto

2. Multiprocessing n Purpose – complete a job sooner n Benchmarks – it really works! n Hidden sources of computing power n MP CONNECT functionality n Examples – how hard is it? n How to learn more

3. Complete a job sooner n It takes the same amount of resource, but the task is done sooner.

4. The Purpose of Multiprocessing n Complete a job in less total elapsed time. n Increase usage of available CPUs.

5. The Practice of Multiprocessing In years past, life was simple. Each computer had one central processing unit (CPU) and was not connected to another computer. SAS was created as a single-threaded application, which means the program executes in a top-down approach on the one processor.

6. The Practice of Multiprocessing Today a computer can have multiple processors, or be part of a network. Version 8 and previous versions of SAS software are still single-threaded applications, but now have the ability through MP CONNECT to take advantage of additional processors for different steps.

7. The Practice of Multiprocessing n Divide application into sub-units of work. FREQ ProcedureTABULATE ProcedureUNIVARIATE Procedure FREQ Procedure TABULATE Procedure UNIVARIATE Procedure n Execute independent tasks in parallel.

8. What if not every task is independent? Extract Oracle Data Read/Summarize SAS Data Set Read/Summarize Raw Data File 0elapsed time Merge Data

9. Benchmarks n SUN Solaris n IBM RS/6000 n HP 9000

10. Benchmark – SUN Solaris n Solaris 7 with twelve 400 MHZ Ultrasparc processors n 14 SAS steps operating on one table, including SORT, SUMMARY, FREQ n 480 MB per data set

11. Benchmark – SUN Solaris

12. Benchmark – HP 9000 n Six 240 MHZ PA8200 processors n Transform 6 data sets, interleave the data, generate reports and write a text file. n 650 MB per data set.

13. Benchmark – HP 9000

14. Benchmark – RS/6000 n Four 332 MHZ processors n Score 3 data sets, generate statistics, and output three text files containing scores. n 900 MB per data set.

15. Benchmark – RS/6000

16. Hidden Sources of Computing Power n Under utilized servers n Workstations after hours

17. Hidden Sources of Computing Power n Under utilized servers

18. Hidden Sources of Computing Power n Workstations after hours

19. MP CONNECT functionality n Start and finish asynchronous sessions n Communicate between sessions n Process return codes n Implement program dependencies n Manage the SAS sessions

20. SAS/CONNECT Monitor Window The list of tasks is dynamically updated as new tasks start, and the Status field changes from “Running” to “Complete”, as appropriate.

21. NOTIFY= YES Option displays a message window that indicates the completion of a task.

22. Examples n Scaling up – concurrent sessions on one machine n Scaling out – concurrent sessions on distributed processors

23. Scaling up – one machine Statements required: What’s new in MP CONNECT n Options... ; Autoconnect = Sascmd = n Rsubmit taskname... ; Wait=no Persist = n Waitfor.... ; n Signoff taskname;

24. Scaling up - one machine Task 1: convert and sort a text file. Task 2: convert and sort a text file. 0elapsed time Interleave Data

25. Scaling up – one machine options SASCMD= '/bin/SAS/V8/sas.exe -nosyntaxcheck ' autosignon=YES ; rsubmit task1 wait=no sysrputsync=yes; data sample1; length type1 $12 region $13;

26. Scaling out – distributed processing Statements required: n Filename rlink... ; 3 diff remote systems n %let taskname =... ; n Signon taskname; n Rsubmit taskname... ; Wait=no n Waitfor.... ; n Signoff taskname;

27. Client (Local) Server (Remote) Task 1 Task 2 Asynchronous Execution on Multiple Machines Server (Remote)

28. Scaling out – distributed processing filename rlink '!sasroot\connect\saslink\tcpunix.scr'; %let task1 = bcom1; signon task1; /* unix host */ rsubmit task1 wait=no ; data sample1; length type1 $12 region $13;

29. How to learn more n SUGI articles www.sas.com/usergroups/sugi/proceedings/ n Cheryl Garner, SUGI 25, paper 16 n John E. Bentley, SUGI 26, paper 269 n John E. Bentley, SUGI 27, paper 107

30. How to learn more n SAS courses www.sas.com/service/edu/cantrain/ n Multiprocessing with SAS ® Software n SAS ® Macro Language n Optimizing SAS ® Programs

31. Questions?