1. Statistics Function Implementation
Traditional Programming Approach
Winter Quarter 2005
Rolando V. Raqueño

2. minimum.pro function minimum, x n = n_elements(x) answer = x(0)
for i=1L, n-1 do begin
if( answer gt x(i) ) then $
answer = x(i)
endfor
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

3. sum.pro function sum, x n = n_elements( x ) answer = 0
for i = 0, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

4. Overflow Problems What if the array is an integer array?
Sums will have a maximum limit based on the number of bits specified by the integer type.
Winter Quarter 2005
Rolando V. Raqueño

5. Corrected sum.pro function sum, x n = n_elements( x ) answer = 0.0D
for i = 0L, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

6. mean.pro function mean, x n = n_elements(x) answer = sum(x) / n
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

7. Integer Problems IDL> b=9/5 IDL> print,b 1 Winter Quarter 2005
Rolando V. Raqueño

8. Integer Problems IDL> b=9/double(5) IDL> print,b 1.8000000
Winter Quarter 2005
Rolando V. Raqueño

9. Corrected mean.pro function mean, x n = n_elements(x)
answer = sum(x) / double(n)
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

10. sum_squares.pro function sum_squares, x squares = x^2.0
answer= sum( squares )
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

11. variance.pro function variance, x n = double(n_elements(x))
answer=(sum_squares(x)-(sum(x)^2.0/n))/ (n-1)
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

12. sd.pro function sd, x answer = sqrt(variance(x)) return, answer end
Winter Quarter 2005
Rolando V. Raqueño

13. Debugging in IDL
Winter Quarter 2005
Rolando V. Raqueño

14. Basic IDL Debugging Commands
breakpoint
.step (.s)
.stepover (.so)
.continue
.return
.out
Winter Quarter 2005
Rolando V. Raqueño

15. mean.pro 10 ;- 11 ; 12 function mean,x 13 n = n_elements(x)
10 ;-
11 ;
12 function mean,x
13 n = n_elements(x)
14 answer = sum(x)/double(n)
15 return, answer
16 end
Winter Quarter 2005
Rolando V. Raqueño

16. Set/Clear Breakpoint
Winter Quarter 2005
Rolando V. Raqueño

17. Enable/Disable Breakpoint
Winter Quarter 2005
Rolando V. Raqueño

18. Edit breakpoints
Winter Quarter 2005
Rolando V. Raqueño

19. Breakpoints IDL> breakpoint, ‘mean.pro’, 12
IDL> help,/breakpoint
Breakpoints:
Index Module Line File
0 MEAN mean.pro
IDL> breakpoint, /clear, 0 OR
IDL> breakpoint, /clear, ‘mean.pro’, 12
Winter Quarter 2005
Rolando V. Raqueño

20. .step
Winter Quarter 2005
Rolando V. Raqueño

21. .step Used after a breakpoint is reached Single step through commands
If statement is a function or procedure
Will enter into it
After a breakpoint is reached, you can single step through each statement using the .step command
If the next statement is a function or procedure, the .step command will enter that module and allow you to step through each statement
Winter Quarter 2005
Rolando V. Raqueño

22. .stepover
Winter Quarter 2005
Rolando V. Raqueño

23. .stepover Similar to .step
Executes functions and procedures to completion without stopping inside the function
Winter Quarter 2005
Rolando V. Raqueño

24. .out
Winter Quarter 2005
Rolando V. Raqueño

25. .out Continues execution out of the current routine.
Useful in leaving a routine that you stepped into, but have determined that all is working properly and want to return to the calling routine.
Winter Quarter 2005
Rolando V. Raqueño

26. .return sqrt(variance(x))
Continues execution until a return statement is encountered.
Useful in checking the return value of an inner (nested) function in a function composition
e.g.
sqrt(variance(x))
Winter Quarter 2005
Rolando V. Raqueño

27. mean.pro 10 ;- 11 ; 12 function mean,x 13 n = n_elements(x)
10 ;-
11 ;
12 function mean,x
13 n = n_elements(x)
14 answer = sum(x)/double(n)
15 return, answer
16 end
Winter Quarter 2005
Rolando V. Raqueño

28. Navigating the stack
Winter Quarter 2005
Rolando V. Raqueño

29. Internally Documenting Your IDL Routines
Winter Quarter 2005
Rolando V. Raqueño

30. IDL_PATH environment variable
setenv IDL_DIR /cis/common/rsi/idl_5
setenv IDL_PATH \+$IDL_DIR/lib:\+$IDL_DIR/examples:
\+/dirs/common/rsi/idl_5:\+/dirs/common/lib/idl
:~/lib/idl
Winter Quarter 2005
Rolando V. Raqueño

31. Use the following template for IDL documentation
/cis/common/rsi/idl_5.5/examples/template.pro ;+
; NAME:
; PURPOSE:
; INPUTS:
; OPTIONAL INPUTS:
; KEYWORD PARAMETERS:
; OUTPUTS:
; OPTIONAL OUTPUTS:
; PROCEDURE:
; EXAMPLE:
; MODIFICATION HISTORY:
; Written by: Your name here, Date.
; July, Any additional mods get described here. ;-
Winter Quarter 2005
Rolando V. Raqueño

32. doc_library example ;+ ; The following routine computes the mean
; of an arbitrary sized array ;
; $Header$
; $Log$ ;-
function mean,x
...
Winter Quarter 2005
Rolando V. Raqueño

33. Executing doc_library
IDL> doc_library
Name of procedure or * for all: mean
Enter 1 for printer, 0 for terminal: 0
----- Documentation for /cis/staff/rvrpci/lib/idl/statistics/mean.pro -----
The following routine computes the mean
of an arbitrary sized array
Winter Quarter 2005
Rolando V. Raqueño

34. Executing doc_library
$Header: /nfs/cis/staff/rvrpci/lib/idl/statistics/RCS/mean.pro,v /01/13
15:44:47 rvrpci Exp $
$Log: mean.pro,v $
Revision /01/13 15:44:47 rvrpci
Initial revision
Winter Quarter 2005
Rolando V. Raqueño

35. More updated help feature
mk_html_help routine
Comment your routines in a given directory using the template describe previously
Give the mk_html_help command with agruments for directory of routines to convert to name of html file
IDL> mk_html_help,’.’,’stats.html’
Winter Quarter 2005
Rolando V. Raqueño

36. Winter Quarter 2005
Rolando V. Raqueño

37. Figuring out where to optimize your code
Code Profiling in IDL
Figuring out where to optimize your code
Winter Quarter 2005
Rolando V. Raqueño

38. PROFILER Command in IDL
Allows you to gather statistics where your program is spending the most time
Guides you to the routine that needs the most optimization
Winter Quarter 2005
Rolando V. Raqueño

39. Let’s take the sum example
; Scalar Approach
function sum, x
n = n_elements(x)
answer = x[0]
for i=1L, n-1 do begin
answer = answer + x[i]
endfor
return, answer
end
; Vector Approach
function sum, x
n = n_elements(x)
a = reform(x,n)
answer = a ## transpose(replicate(1.0, n))
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

40. Sample test program for profile analysis
pro test_profiler
a = findgen(100)
for i = 1, 100 do begin
print, i
s = sum( a )
s2 = sum_squares( a )
m = mean( a )
v = variance( a )
sd = standard_deviation( a )
endfor
end
Winter Quarter 2005
Rolando V. Raqueño

41. sum.pro function sum, x n = n_elements( x ) answer = 0.0
for i = 0L, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

42. sum.pro function sum, x n = n_elements( x ) a = reform( x, n)
answer=a##transpose(replicate(1.0,n))
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

43. Using Matrix Multiplication
reform ## =
transpose(replicate(1.0,n))
Winter Quarter 2005
Rolando V. Raqueño

44. sum_squares.pro function sum_squares, x squares = x^2.0
answer= sum( squares )
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

45. sum_squares.pro function sum_squares, x n = n_elements( x )
a = reform( x, n)
answer = a ## transpose(a)
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

46. Using Matrix Multiplication
reform ## =
Winter Quarter 2005
Rolando V. Raqueño

47. To Profile Your Code
Compile (and run) all the routines you want to analyze
At the IDL prompt (or through the GUI), invoke the profiler
IDL> profiler
Run your code
Output the report using
IDL> profiler, /report
Winter Quarter 2005
Rolando V. Raqueño

48. Interpreting the Profile report
MODULE
Name of routine
TYPE
U = user-defined function
S = system-defined function
COUNT
Number of times a function has been called since the invocation of the “profiler” command
Winter Quarter 2005
Rolando V. Raqueño

49. ONLY Time Statistic in Profile Report
Time spent in the function ONLY
Calls made to other functions from within this function are not part of this value
AVERAGE
Average of the time spent ONLY in this function for all the calls made so far
Winter Quarter 2005
Rolando V. Raqueño

50. TIME Statistic in Profile Report
Cumulative time spent in a given routine including subsequent calls to other functions
AVERAGE
Average of TIME for all the calls made so far
Winter Quarter 2005
Rolando V. Raqueño

51. Result of initial profile
IDL> profiler,/report
Module Type Count Only(s) Avg.(s) Time(s) Avg.(s)
MEAN (U)
STANDARD_DEVIATION
(U)
SUM (U)
SUM_SQUARES (U)
TEST_PROFILER (U)
VARIANCE (U)
Winter Quarter 2005
Rolando V. Raqueño

52. Result of vector sum routine
IDL> profiler,/report
Module Type Count Only(s) Avg.(s) Time(s) Avg.(s)
MEAN (U)
STANDARD_DEVIATION
(U)
SUM (U)
SUM_SQUARES (U)
TEST_PROFILER (U)
VARIANCE (U)
SCALAR RESULTS
SUM (U)
Winter Quarter 2005
Rolando V. Raqueño

53. Let’s take the sum_squares example
; Scalar Approach
function sum_squares, x
answer = sum(x^2)
return, answer
end
; Vector Approach
function sum_squares, x
n = n_elements(x)
a = reform(x,n)
answer = a ## transpose(a)
return, answer
end
Winter Quarter 2005
Rolando V. Raqueño

54. Profile report of vectorized sum_squares function
IDL> profiler,/report
Module Type Count Only(s) Avg.(s) Time(s) Avg.(s)
MEAN (U)
STANDARD_DEVIATION
(U)
SUM (U)
SUM_SQUARES (U)
TEST_PROFILER (U)
VARIANCE (U)
SCALAR RESULTS
SUM_SQUARES (U)
Winter Quarter 2005
Rolando V. Raqueño

55. Use this as an exercise to become familiar with profiling techniques
What is the performance difference between the scalar and vector implementation of RECT?
Use this as an exercise to become familiar with profiling techniques
Winter Quarter 2005
Rolando V. Raqueño

56. Profile the Rect Function
VECTOR FORM
function rect, x
answer=x
ax=abs(x)
gt_half=where(ax gt 0.5,n_gt)
eq_half=where(ax eq 0.5,n_eq)
lt_half=where(ax lt 0.5,n_lt)
if (n_gt ne 0) then answer(gt_half)=0.0
if (n_eq ne 0) then answer(eq_half)=0.5
if (n_lt ne 0) then answer(lt_half)=1.0
return, answer
end
SCALAR FORM
function rect, x, x0, b
a = abs( (x-x0)/b )
if(a gt 0.5)then return, 0.0
if(a lt 0.5)then return, 1.0
if(a eq 0.5)then return, 0.5
end
Winter Quarter 2005
Rolando V. Raqueño