2. Ch4b- 2 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University Performance metrics I’m concerned with how long it takes to run my program Chances are, that number isn’t published with the specs for the computer Standardized metrics Benchmarks (SPEC, etc.) Clock Frequency MIPS

3. Ch4b- 3 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University Benchmarks Run a suite of benchmark programs, average the performance Benchmarks - programs thought to be representative of commonly-used programs Advantages Actually corresponds to execution time! Represents a wider range of programs Disadvantages Are they running your program? Who picks the benchmarks? Be wary if the manufacturer does!

4. Ch4b- 4 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University SPEC Benchmarks SPEC (System Performance Evaluation Cooperative) maintains a set of benchmark suites SPEC Web Page (www.spec.org) New tests use SPEC CPU2006 CINT2006 - Performance on integer programs CFP2006 - Performance on floating-point programs Larger numbers indicate better performance Tests prior to 2006 used CPU 2000 or CPU95

5. Ch4b- 5 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University CINT2000 Results for Various Processors Clock Speed (GHz) CINT2000 Note: Results depend on Cache size, memory system, and motherboard

6. Ch4b- 6 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University CFP2000 Results for Various Processors Note: Results depend on Cache size, memory system, and motherboard Clock Speed (GHz) CFP2000

7. Ch4b- 7 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University Alternative: Web Benchmarks SPEC benchmarks are good, but difficult to sort through It can take hours of work just to find out the best performing variety of a system Results are available only if manufacturer submits them Web Sources – PassMark, Tom’s Hardware Guide Track performance of Windows-based machines for the past few years PassMark (cpubenchmark.net) uses user-submitted results – Benchmark your system free and compare. Tom’s (tomshardware.com) does their own testing

8. Ch4b- 8 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University Improving System Peformance Assume we’re running a program that spends 40% of its time accessing memory Now, we upgrade the processor from 1 GHz to 4 GHz How much faster does the program run? (4X faster?) We’ve reduced the time for 60% of the program by 4 But we haven’t touched the memory access time New total = Old * (40% + (60% / 4)) = Old * (40% + 15%) = Old * 55% Not even twice as fast!

9. Ch4b- 9 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University Amdahl’s Law Practical effect: “Make the common case fast” Corollary: “Forget about the rare case” New Execution time = Execution time affected by impr. + Unaffected Execution Time Amount of Improvement Example: 70% of my execution time is done on integer ADDs, and 6% on floating point ADDs. Total execution time is 100 seconds. What’s the effect of making integer ADDs twice as fast? New time = (100 *.70) / 2 + (100 *.30) = 35+30=65 seconds What’s the effect of making F.P. Adds twice as fast? New time = (100 *.06) / 2 + (100 *.94) = 3+94 = 97 seconds

10. Ch4b- 10 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University Speedup When analyzing a change to a system, we often look at speedup. Speedup = Old Execution Time / New Execution Time Speedup > 1  Improvement! Speedup = 1  No change Speedup < 1  UhOh! Continuing example from last slide: Int. Adds 2X faster: Speedup = 100s/65s = 1.54 FP Adds 2X faster: Speedup = 100s/97s = 1.03

11. Ch4b- 11 EE/CS/CPE 3760 - Computer Organization  Seattle Pacific University Performance Summary Execution time is the most important performance metric Basic formula for performance: Execution time = instructions * cycle time * CPI Amdahl’s law describes how making limited improvements affects the bottom line Only make improvements in areas that are commonly used Standard benchmarks help us to compare performance of various computers Beware of overly-simplified comparisons