1. Introduction to the LOGPARSE Macro GASUG Atlanta, GA July 18 th, 2007

2. Page 2 © Copyright ERMAS Corporation 2007 LOGPARSE  Integration Consulting  Marketing Automation and Optimization  Marketing Campaign Management, Fulfillment, and Response Analysis  Risk & Analytics  Analytical Modeling  Data Warehousing  Business Intelligence Solutions  CRM  Database Solutions  Web Development Services Our Services

3. Page 3 © Copyright ERMAS Corporation 2007 LOGPARSE Our Locations: 5 Main Hubs + International 5. UK & Europe We have completed projects in Korea, Singapore, Malaysia, United Kingdom, Netherlands, Sweden, Germany, and Peru. 3. New York 2. Charlotte 4. Dallas 1. Atlanta International Corporate Location: 4080 McGinnis Ferry Rd Bldg 100, Suite 101 Alpharetta, GA 30005

4. Page 4 © Copyright ERMAS Corporation 2007 LOGPARSE Our Atlanta Lab Always Seeking Talent Send inquiries to: info@ermas.com

5. Page 5 © Copyright ERMAS Corporation 2007 LOGPARSE Today’s Topic  Introduce the LOGPARSE macro to capture FULLSTIMER statistics  Demonstrate a technique for profiling very large SAS applications  Provide code for creating a Performance Dashboard …Capture Tuning Statistics …Not Tune SAS Applications

6. Page 6 © Copyright ERMAS Corporation 2007 LOGPARSE LOGPARSE MACRO  Provided by SAS, experimential with SAS 9.1  Captures FULLSTIMER statistics from the SAS Log and stores them in a SAS Dataset For every SAS Step  Available on the SAS Scalability & Performance Community Site www.support.sas.com/rnd/scalability/tools/fullstim/fullstim.html www.support.sas.com/rnd/scalability/tools/fullstim/logparse.zip

7. Page 7 © Copyright ERMAS Corporation 2007 LOGPARSE What’s Inside logparse.zip  Four files in logparse.zip  %LOGPARSE() Extracts the statistics from the SAS LOG(s)  %PASSINFO() Optional, provides session information to the %LOGPARSE() macro.  %MVSNAME() Called by %PASSINFO() for MVS only  Readme.txt A good place to start

8. Page 8 © Copyright ERMAS Corporation 2007 LOGPARSE Getting Started with %LOGPARSE  OPTIONS FULLSTIMER; as the first line for the program you wish to collect statistics  %PASSINFO; optionally on the second line  Run the program to generate the log(s)  Create another program to use %LOGPARSE and collect the statistics %include logparse; %logparse( yourProgram.log, yourPerfData, OTH ); proc print data=yourPerfData; run;

9. Page 9 © Copyright ERMAS Corporation 2007 LOGPARSE Big Picture LOG FILE(S) YOUR LOGPARSE PROGRAM Statistics Dataset %LOGPARSE( saslog, outds, system, pdsloc, append=no )

10. Page 10 © Copyright ERMAS Corporation 2007 LOGPARSE %LOGPARSE Syntax %LOGPARSE( saslog, outds, system, pdsloc, append=no )  saslog is the name of the SAS LOG to parse, requires the.log extension  outds is the output SAS dataset for the parsed LOG results  system is the 3-CHAR code for the system on which %LOGPARSE is run. If not specified, the default value is the system %LOGPARSE is executing on. MVS - z/OS, OS/390, or MVS ALP - OpenVMS Alpha VMS - VMS OTH - All other OSs, like Windows  pdsloc identifies the PDS member where the SAS LOG is stored on MVS system. See the readme.txt file.  append=no specifies whether to append the results to an existing file or overwrite NO – creates a new SAS file based on the outds parameter YES – appends the output to file specified by outds or creates a new file if the file does not exist

11. Page 11 © Copyright ERMAS Corporation 2007 LOGPARSE The STATISTICS Dataset  The statistics captured by %LOGPARSE are dependent on the NOTES: produced by FULLSTIMER and vary by system.  Important non-statistics columns to understand logfile – name of the log file stepname – DATA, SORT, FORMAT, PRINTO, SAS, initialization. NOTE: DATA statement used (Total process time): NOTE: The SAS System used: stepcnt – sequence number applied to the order of the step within the log.  Windows Statistics Columns realtime usertime systemtime obsin obsout varsout memused

12. Page 12 © Copyright ERMAS Corporation 2007 LOGPARSE Big Picture LOG FILE(S) YOUR LOGPARSE PROGRAM Statistics Dataset %LOGPARSE( saslog, outds, system, pdsloc, append=no )

13. Page 13 © Copyright ERMAS Corporation 2007 LOGPARSE ExampleExample options fullstimer; %let defaultPath=C:\Documents and Settings\Administrator\My Documents\My SAS Code; %include "&defaultPath\timerUtils\passinfo.sas"; %passinfo; %include "&defaultPath\census\ipumsFormats.inc"; %include "&defaultPath\census\convertToSAS.inc"; %include "&defaultPath\census\splitData.inc"; %include "&defaultPath\census\subsetMetro.inc"; %include "&defaultPath\census\hhRollup.inc"; libname ipumsdat "&defaultPath\census"; /*----------------------------------------------------------------------------+ Main Program +----------------------------------------------------------------------------*/ %ipumsFormats(ipumsdat.fmmiller_ermas_com_001); %convertToSAS(asciidat, ipumsdat.mmiller_ermas_com_001); %splitData(ipumsdat, ipumsdat.mmiller_ermas_com_001); %subsetMetro("Atlanta", 2003 2004 2005 2006); %hhRollup(2003 2004 2005 2006)

14. Page 14 © Copyright ERMAS Corporation 2007 LOGPARSE The Bad News  A dataset of statistics is produced – map that back to your program/log…. OR

15. Page 15 © Copyright ERMAS Corporation 2007 LOGPARSE The Good News

16. Page 16 © Copyright ERMAS Corporation 2007 LOGPARSE Use PRINTTO to Write Separate Logs filename fmtLog "&defaultPath\census\ipumsFormtas.timingDemo.log"; filename cvrtLog "&defaultPath\census\convertToSAS.timingDemo.log"; filename splitLog "&defaultPath\census\splitData.timingDemo.log"; filename substLog "&defaultPath\census\subsetMetro.timingDemo.log"; filename hhrupLog "&defaultPath\census\hhRollup.timingDemo.log"; /*----------------------------------------------------------------------------+ Main Program +----------------------------------------------------------------------------*/ proc printto log=fmtLog new; run; %ipumsFormats(ipumsdat.fmmiller_ermas_com_001); proc format cntlin=ipumsdat.fmmiller_ermas_com_001; run; proc printto log=cvrtLog new; run; %convertToSAS(asciidat, ipumsdat.mmiller_ermas_com_001); proc printto log=splitLog new; run; %splitData(ipumsdat, ipumsdat.mmiller_ermas_com_001); proc printto log=substLog new; run; %subsetMetro("Atlanta", 2003 2004 2005 2006); proc printto log=hhrupLog new; run; %hhRollup(2003 2004 2005 2006); run; proc printto; run; MODULE.PROGRAM.LOG Use PROC PRINTTO to Write Separate Log Files

17. Page 17 © Copyright ERMAS Corporation 2007 LOGPARSE  Use our toolkit package to create a visual performance dashboard (WINDOWS ONLY)  Name each log in the format module.program.log for each subset of the program to be analyzed  Use the perfDash.sas program to analyze the logs ERMAS Performance Toolkit /*----------------------------------------------------------------------------+ User Parameters rootDir - location of the SAS utilities, see installation instructions below logDir - directory where the SAS LOGs are located program - name of the SAS program for which to generate the dashboard +----------------------------------------------------------------------------*/ %let rootDir = %str(C:\GASUG2007); %let logDir = %str(C:\GASUG2007\census); %let program = timingDemo; /* case sensitive */

18. Page 18 © Copyright ERMAS Corporation 2007 LOGPARSE  How to get the Ermas Performance Toolkit GASUG2007.ZIP file contains four directories:  timerUtils – location of the perfDash.sas program  census – log files to analyze  myLibrary – utility library  utils – more utilities.  SAS Scalability & Performance Community  Great SUGI Paper on the LOGPARSE Macro ResourcesResources www.support.sas.com/rnd/scalability/tools/fullstim/fullstim.html http://www.ermas.com/downloads.htm http://www2.sas.com/proceedings/sugi30/219-30.pdf