Statistics Function Implementation Traditional Programming Approach Winter Quarter 2005 Rolando V. Raqueño

minimum.pro function minimum, x n = n_elements(x) answer = x(0) for i=1L, n-1 do begin if( answer gt x(i) ) then $ answer = x(i) endfor return, answer end Winter Quarter 2005 Rolando V. Raqueño

sum.pro function sum, x n = n_elements( x ) answer = 0 for i = 0, n-1 do begin answer = answer + x(i) endfor return, answer end Winter Quarter 2005 Rolando V. Raqueño

Overflow Problems What if the array is an integer array? Sums will have a maximum limit based on the number of bits specified by the integer type. Winter Quarter 2005 Rolando V. Raqueño

Corrected sum.pro function sum, x n = n_elements( x ) answer = 0.0D for i = 0L, n-1 do begin answer = answer + x(i) endfor return, answer end Winter Quarter 2005 Rolando V. Raqueño

mean.pro function mean, x n = n_elements(x) answer = sum(x) / n return, answer end Winter Quarter 2005 Rolando V. Raqueño

Integer Problems IDL> b=9/5 IDL> print,b 1 Winter Quarter 2005 Rolando V. Raqueño

Integer Problems IDL> b=9/double(5) IDL> print,b 1.8000000 Winter Quarter 2005 Rolando V. Raqueño

Corrected mean.pro function mean, x n = n_elements(x) answer = sum(x) / double(n) return, answer end Winter Quarter 2005 Rolando V. Raqueño

sum_squares.pro function sum_squares, x squares = x^2.0 answer= sum( squares ) return, answer end Winter Quarter 2005 Rolando V. Raqueño

variance.pro function variance, x n = double(n_elements(x)) answer=(sum_squares(x)-(sum(x)^2.0/n))/ (n-1) return, answer end Winter Quarter 2005 Rolando V. Raqueño

sd.pro function sd, x answer = sqrt(variance(x)) return, answer end Winter Quarter 2005 Rolando V. Raqueño

Debugging in IDL Winter Quarter 2005 Rolando V. Raqueño

Basic IDL Debugging Commands breakpoint .step (.s) .stepover (.so) .continue .return .out Winter Quarter 2005 Rolando V. Raqueño

mean.pro 10 ;- 11 ; 12 function mean,x 13 n = n_elements(x) 10 ;- 11 ; 12 function mean,x 13 n = n_elements(x) 14 answer = sum(x)/double(n) 15 return, answer 16 end Winter Quarter 2005 Rolando V. Raqueño

Set/Clear Breakpoint Winter Quarter 2005 Rolando V. Raqueño

Enable/Disable Breakpoint Winter Quarter 2005 Rolando V. Raqueño

Edit breakpoints Winter Quarter 2005 Rolando V. Raqueño

Breakpoints IDL> breakpoint, ‘mean.pro’, 12 IDL> help,/breakpoint Breakpoints: Index Module Line File ----- ------ ---- ---- 0 MEAN 12 mean.pro IDL> breakpoint, /clear, 0 OR IDL> breakpoint, /clear, ‘mean.pro’, 12 Winter Quarter 2005 Rolando V. Raqueño

.step Winter Quarter 2005 Rolando V. Raqueño

.step Used after a breakpoint is reached Single step through commands If statement is a function or procedure Will enter into it After a breakpoint is reached, you can single step through each statement using the .step command If the next statement is a function or procedure, the .step command will enter that module and allow you to step through each statement Winter Quarter 2005 Rolando V. Raqueño

.stepover Winter Quarter 2005 Rolando V. Raqueño

.stepover Similar to .step Executes functions and procedures to completion without stopping inside the function Winter Quarter 2005 Rolando V. Raqueño

.out Winter Quarter 2005 Rolando V. Raqueño

.out Continues execution out of the current routine. Useful in leaving a routine that you stepped into, but have determined that all is working properly and want to return to the calling routine. Winter Quarter 2005 Rolando V. Raqueño

.return sqrt(variance(x)) Continues execution until a return statement is encountered. Useful in checking the return value of an inner (nested) function in a function composition e.g. sqrt(variance(x)) Winter Quarter 2005 Rolando V. Raqueño

mean.pro 10 ;- 11 ; 12 function mean,x 13 n = n_elements(x) 10 ;- 11 ; 12 function mean,x 13 n = n_elements(x) 14 answer = sum(x)/double(n) 15 return, answer 16 end Winter Quarter 2005 Rolando V. Raqueño

Navigating the stack Winter Quarter 2005 Rolando V. Raqueño

Internally Documenting Your IDL Routines Winter Quarter 2005 Rolando V. Raqueño

IDL_PATH environment variable setenv IDL_DIR /cis/common/rsi/idl_5 setenv IDL_PATH \+$IDL_DIR/lib:\+$IDL_DIR/examples: \+/dirs/common/rsi/idl_5:\+/dirs/common/lib/idl :~/lib/idl Winter Quarter 2005 Rolando V. Raqueño

Use the following template for IDL documentation /cis/common/rsi/idl_5.5/examples/template.pro ;+ ; NAME: ; PURPOSE: ; INPUTS: ; OPTIONAL INPUTS: ; KEYWORD PARAMETERS: ; OUTPUTS: ; OPTIONAL OUTPUTS: ; PROCEDURE: ; EXAMPLE: ; MODIFICATION HISTORY: ; Written by: Your name here, Date. ; July, 1994 Any additional mods get described here. ;- Winter Quarter 2005 Rolando V. Raqueño

doc_library example ;+ ; The following routine computes the mean ; of an arbitrary sized array ; ; $Header$ ; $Log$ ;- function mean,x ... Winter Quarter 2005 Rolando V. Raqueño

Executing doc_library IDL> doc_library Name of procedure or * for all: mean Enter 1 for printer, 0 for terminal: 0 ----- Documentation for /cis/staff/rvrpci/lib/idl/statistics/mean.pro ----- The following routine computes the mean of an arbitrary sized array Winter Quarter 2005 Rolando V. Raqueño

Executing doc_library $Header: /nfs/cis/staff/rvrpci/lib/idl/statistics/RCS/mean.pro,v 1.1 1998/01/13 15:44:47 rvrpci Exp $ $Log: mean.pro,v $ Revision 1.1 1998/01/13 15:44:47 rvrpci Initial revision Winter Quarter 2005 Rolando V. Raqueño

More updated help feature mk_html_help routine Comment your routines in a given directory using the template describe previously Give the mk_html_help command with agruments for directory of routines to convert to name of html file IDL> mk_html_help,’.’,’stats.html’ Winter Quarter 2005 Rolando V. Raqueño

Winter Quarter 2005 Rolando V. Raqueño

Figuring out where to optimize your code Code Profiling in IDL Figuring out where to optimize your code Winter Quarter 2005 Rolando V. Raqueño

PROFILER Command in IDL Allows you to gather statistics where your program is spending the most time Guides you to the routine that needs the most optimization Winter Quarter 2005 Rolando V. Raqueño

Let’s take the sum example ; Scalar Approach function sum, x n = n_elements(x) answer = x[0] for i=1L, n-1 do begin answer = answer + x[i] endfor return, answer end ; Vector Approach function sum, x n = n_elements(x) a = reform(x,n) answer = a ## transpose(replicate(1.0, n)) return, answer end Winter Quarter 2005 Rolando V. Raqueño

Sample test program for profile analysis pro test_profiler a = findgen(100) for i = 1, 100 do begin print, i s = sum( a ) s2 = sum_squares( a ) m = mean( a ) v = variance( a ) sd = standard_deviation( a ) endfor end Winter Quarter 2005 Rolando V. Raqueño

sum.pro function sum, x n = n_elements( x ) answer = 0.0 for i = 0L, n-1 do begin answer = answer + x(i) endfor return, answer end Winter Quarter 2005 Rolando V. Raqueño

sum.pro function sum, x n = n_elements( x ) a = reform( x, n) answer=a##transpose(replicate(1.0,n)) return, answer end Winter Quarter 2005 Rolando V. Raqueño

Using Matrix Multiplication reform ## = transpose(replicate(1.0,n)) Winter Quarter 2005 Rolando V. Raqueño

sum_squares.pro function sum_squares, x squares = x^2.0 answer= sum( squares ) return, answer end Winter Quarter 2005 Rolando V. Raqueño

sum_squares.pro function sum_squares, x n = n_elements( x ) a = reform( x, n) answer = a ## transpose(a) return, answer end Winter Quarter 2005 Rolando V. Raqueño

Using Matrix Multiplication reform ## = Winter Quarter 2005 Rolando V. Raqueño

To Profile Your Code Compile (and run) all the routines you want to analyze At the IDL prompt (or through the GUI), invoke the profiler IDL> profiler Run your code Output the report using IDL> profiler, /report Winter Quarter 2005 Rolando V. Raqueño

Interpreting the Profile report MODULE Name of routine TYPE U = user-defined function S = system-defined function COUNT Number of times a function has been called since the invocation of the “profiler” command Winter Quarter 2005 Rolando V. Raqueño

ONLY Time Statistic in Profile Report Time spent in the function ONLY Calls made to other functions from within this function are not part of this value AVERAGE Average of the time spent ONLY in this function for all the calls made so far Winter Quarter 2005 Rolando V. Raqueño

TIME Statistic in Profile Report Cumulative time spent in a given routine including subsequent calls to other functions AVERAGE Average of TIME for all the calls made so far Winter Quarter 2005 Rolando V. Raqueño

Result of initial profile IDL> profiler,/report Module Type Count Only(s) Avg.(s) Time(s) Avg.(s) MEAN (U) 100 0.003368 0.000034 0.046592 0.000466 STANDARD_DEVIATION (U) 100 0.001531 0.000015 0.100819 0.001008 SUM (U) 700 0.313726 0.000448 0.313726 0.000448 SUM_SQUARES (U) 300 0.010265 0.000034 0.141342 0.000471 TEST_PROFILER (U) 1 0.247449 0.247449 0.581684 0.581684 VARIANCE (U) 200 0.005345 0.000027 0.208235 0.001041 Winter Quarter 2005 Rolando V. Raqueño

Result of vector sum routine IDL> profiler,/report Module Type Count Only(s) Avg.(s) Time(s) Avg.(s) MEAN (U) 100 0.002323 0.000023 0.008024 0.000080 STANDARD_DEVIATION (U) 100 0.002149 0.000021 0.036169 0.000362 SUM (U) 700 0.056344 0.000080 0.056344 0.000080 SUM_SQUARES (U) 300 0.011034 0.000037 0.043131 0.000144 TEST_PROFILER (U) 1 0.036511 0.036511 0.115969 0.115969 VARIANCE (U) 200 0.007608 0.000038 0.052291 0.000261 SCALAR RESULTS SUM (U) 700 0.313726 0.000448 0.313726 0.000448 Winter Quarter 2005 Rolando V. Raqueño

Let’s take the sum_squares example ; Scalar Approach function sum_squares, x answer = sum(x^2) return, answer end ; Vector Approach function sum_squares, x n = n_elements(x) a = reform(x,n) answer = a ## transpose(a) return, answer end Winter Quarter 2005 Rolando V. Raqueño

Profile report of vectorized sum_squares function IDL> profiler,/report Module Type Count Only(s) Avg.(s) Time(s) Avg.(s) MEAN (U) 100 0.002285 0.000023 0.007718 0.000077 STANDARD_DEVIATION (U) 100 0.002084 0.000021 0.016103 0.000161 SUM (U) 400 0.024090 0.000060 0.024090 0.000060 SUM_SQUARES (U) 300 0.013434 0.000045 0.013434 0.000045 TEST_PROFILER (U) 1 0.099387 0.099387 0.149792 0.149792 VARIANCE (U) 200 0.008513 0.000043 0.028404 0.000142 SCALAR RESULTS SUM_SQUARES (U) 300 0.011034 0.000037 0.043131 0.000144 Winter Quarter 2005 Rolando V. Raqueño

Use this as an exercise to become familiar with profiling techniques What is the performance difference between the scalar and vector implementation of RECT? Use this as an exercise to become familiar with profiling techniques Winter Quarter 2005 Rolando V. Raqueño

Profile the Rect Function VECTOR FORM function rect, x answer=x ax=abs(x) gt_half=where(ax gt 0.5,n_gt) eq_half=where(ax eq 0.5,n_eq) lt_half=where(ax lt 0.5,n_lt) if (n_gt ne 0) then answer(gt_half)=0.0 if (n_eq ne 0) then answer(eq_half)=0.5 if (n_lt ne 0) then answer(lt_half)=1.0 return, answer end SCALAR FORM function rect, x, x0, b a = abs( (x-x0)/b ) if(a gt 0.5)then return, 0.0 if(a lt 0.5)then return, 1.0 if(a eq 0.5)then return, 0.5 end Winter Quarter 2005 Rolando V. Raqueño