1. CSc 352 Performance Tuning
Saumya Debray
Dept. of Computer Science
The University of Arizona, Tucson

2. Background
Performance tuning  modifying software to make it more efficient
often the performance metric is execution speed
other metrics also possible, e.g., memory footprint, response time, energy efficiency
How to get performance improvements
“system tweaking” (e.g., compiler optimizations) can get some improvement; typically this is relatively small
most large improvements are algorithmic in nature
needs active and focused human intervention
requires data to identify where to focus efforts

3. When to optimize? Get the program working correctly
calculating incorrect results quickly isn’t useful
“premature optimization is the root of all evil” – Knuth (?)
Determine whether performance is adequate
Optimization unnecessary for many programs
Figure out what code changes are necessary to improve performance
be cognizant of the possibility that performance tuning may be necessary later on
►design and write the program with this in mind

4. Compiler optimizations
Invoked using compiler options, e.g., “gcc –O2”
usually several different levels supported (gcc: -O0 … -O3)
may also allow fine-grained control over code optimization
gcc supports ~200 optimization-related command-line options
They address machine-level inefficiencies, not algorithm-level inefficiencies
e.g., gcc optimizations improve hardware register usage…
… but not sequential search over a long linked list
Significant performance improvements usually need human intervention

5. Example about 10% improvement overall
not atypical; possible to do better
compiler optimization effect small if either:
code already highly optimized; or
algorithm is lousy

6. Where to optimize?
Consider a program with this execution time distribution:
doubling speed of func3  overall improvement = 5%
doubling speed of func1  overall improvement = 30%
focusing on func1 gives better results for time invested

7. Profiling tools These are tools that:
monitor the program’s execution at runtime
give data on how often functions are called, where the program spends its time
provide guidance on where to focus one’s efforts
Many different tools available, we’ll focus on:
gprof: connected to gcc
gcov: already used in connection with code coverage
kcachegrind: connected to valgrind

8. Using gprof Compile using “gcc –pg”
this adds some book-keeping code, so this will be a little slower
Run this executable, say a.out, on “representative” inputs
creates a data file “gmon.out”
Run “gprof a.out”
extracts information from gmon.out
“flat profile” : time and #calls info per function
“call graph” : time and #calls per function broken down on each place where the function is called

9. Using gprof: example
ave. time per call spent in the function and its descendants
% time spent in each function
time accounted for by each function alone
no. of times called
ave. time per call spent in the function

10. Using the profile information
Expect %time and self-seconds to correlate
If self μs/call high [or: self-seconds is high and calls is low]:
each call is expensive; overhead is due to the code for the function
if calls is high and self μs/call is low:
each call is inexpensive; overhead mainly due to no. of function calls
if self μs/call is low and total μs/call is high:
each call is expensive, but overhead due to some descendant routine

11. Examining the possibilities 1
Code for the function is expensive [self μs/call high]
need to get a better idea of where time is being spent in the function body
may help to pull parts of the function body into separate functions
allows more detailed profile info
can be “inlined back” after performance optimization
Optimization approach:
reduce the cost of the common-case execution path through the function

12. Examining the possibilities 2
No. of calls to a function is the problem [calls is high but self μs/call is low]:
need to reduce the number/cost of calls
possible approaches:
[best] avoid the call entirely whenever possible, e.g.:
use hashing to reduce the set of values to be considered; or
see if the call can be avoided in the common case (e.g., maybe by maintaining asome extra information)
reduce the cost of making the call
inline the body of the called function into the caller

13. Examining the possibilities
Often, performance improvement will involve a tradeoff. E.g.:
transform linear to binary search:
reduces no. of values considered in the search
requires sorting
transform a simple linked list into a hash table
reduces the no. of values considered when searching
requires more memory (hash table), some computation (hash values)
Need to be aware of this tradeoff

14. Approaching performance optimization
Different problems may require very different solutions
Essential idea:
avoid unnecessary work whenever possible
prefer cheap operations to expensive ones
Apply these ideas at all levels:
library routines used
language-level operations (e.g., function calls vs. macros)
higher-level algorithms

15. Optimization 1: Filtering
Useful when:
we are searching a large collection of items, most of which don’t match the search criteria
determining whether a particular item matches is expensive
there is a (relatively) cheap check that is satisfied iff an item does not match
What we do:
use the cheap check to quickly disqualify items that won’t match
effectiveness depends on how many items get disqualifed

16. Filtering Hashing Binary search
particularly useful for strings (but not restricted to them)
can give order-of-improvement performance improvements
sensitive to quality of hash function
Binary search
knowing that the data items are sorted allows us to quickly exclude many of them that won’t match

17. Filters can apply to complex structures
In a research project, we were searching through a large no. of code fragments looking for repetition:
code in compiler’s internal form (directed graph), not source code
we used a 64-bit “fingerprint” for each code region
48 bits
type and size of the first 8 code blocks in the region
(6 bits per block: 2 bits for type, 6 bits for no. of instrs)
16 bits
size of region

18. Optimization 2: Buffering
Useful when:
an expensive operation is being applied to a large no. of items
the operation can also be applied collectively to a group of items
What we do:
collect the items into groups
apply the operation to the groups instead of individual items
Most often used for I/O operations

19. Optimization technique 3: precomputation
Useful when:
a result can be computed once and reused many times
we can predict which results will be computed
we can look up a result cheaply
What we do:
identify operations that get executed over and over
compute the result ahead of time and save it
use the saved result later in the program

20. Optimization 3: cacheing
Useful when:
we repeatedly perform an expensive operation
there is a cheap way to check whether a computation has been done before
What we do:
keep a cache of computations and results; reuse a result if it is already in the cache
Difference from precomputation:
caches usually have a limited size
the cache may need to be emptied if it fills up

21. Optimization 4: Using cheaper operations
Macros vs. functions
sometimes it may be cheaper to write a code fragment as a macro than as a function
the macro does not incur the cost of function call/return
macro arguments may be evaluated multiple times
#define foo(x, y, z) …. x …. y … x … y … x… y … z … x … y …
foo(e1, e2, e3)  …. e1 …. e2 … e1 … e2 … e1 … e2 … e3 … e1 … e2 …
Function inlining
conceptually similar to (but slightly different from) macros
replace a call to a function by a copy of the function body
eliminates function call/return overhead

22. Optimization 4: Using cheaper operations

23. Hashing and Filtering
Many computations involve looking through data to find those that have some property
for each data item X {
if (X has property) {
process X }
This can be expensive if: no. of items is large; and /or checking for the property is expensive.
Hashing and filtering can be used to reduce the cost of checking.
Total cost =
no. of data items x
cost of checking each item

24. Filtering: Basic Idea Goal: (Cheaply) reduce no. of items to process
Given:
a set of items S
some property P
Find:
a function h such that
h() is easy to compute;
h(x) says something useful about whether x has property P h

25. Filtering: Examples isPrime(n): equality of two strings s1 and s2
full test: check for divisors between 1 and n
filter: n == 2 or n is odd
filters out even numbers > 2
equality of two strings s1 and s2
full test: strcmp(s1, s2)
filter: s1[0] == s2[0]
isDivisibleBy3(n)
s1 and s2 are anagrams
The filter depends on the property we’re testing!
Must be a necessary condition:
(forall x)[filter (x)  full_test(x)]

26. Hashing Conceptually related to filtering
Basic idea: Given a set of items S and a property P:
use a hash function h() to divide up the set S into a number of “buckets”
usually, h() maps S to integers (natural numbers)
h(x) == h(y) means x and y are in the same bucket
if x and y fall in the same bucket, they may share the property P (need to check)
if x and y are in different buckets, they definitely don’t share the property P (no need to check)

27. Hashing: An Implementation
compute a hash function h() where
h(x)  {0, …, n-1}
use h() to index into the appropriate bucket
search/insert in this bucket
hash bucket 1 2 …
n-1
hash table
(n buckets)

28. Performance Tuning: Summary
Big improvements come from algorithmic changes
but don’t ignore code-level issues (e.g., cheaper operations)
Use profiling to understand performance behavior
where to focus efforts
reasons for performance overheads
Figure out how to transform the program based on nature of overheads
Good design, modularization essential