1. November 29, 2015204521 Digital System Architecture Computer Performance and Cost Pradondet Nilagupta Fall 2005 (original notes from Randy Katz, & Prof. Jan M. Rabaey, UC Berkeley Prof. Shaaban, RIT)

2. November 29, 2015204521 Digital System Architecture2 Review: What is Computer Architecture? Technology Applications Computer Architect Interfaces Machine Organization Measurement & Evaluation ISAAPI Link I/O Chan Regs IR

3. November 29, 2015204521 Digital System Architecture3 Review: Computer System Components SDRAM PC100/PC133 100-133MHZ 64-128 bits wide 2-way inteleaved ~ 900 MBYTES/SEC )64bit) Double Date Rate (DDR) SDRAM PC3200 200 MHZ DDR 64-128 bits wide 4-way interleaved ~3.2 GBYTES/SEC (one 64bit channel) ~6.4 GBYTES/SEC (two 64bit channels) RAMbus DRAM (RDRAM) 400MHZ DDR 16 bits wide (32 banks) ~ 1.6 GBYTES/SEC CPU Caches Front Side Bus (FSB) I/O Devices: Memory Controllers adapters Disks Displays Keyboards Networks NICs I/O Buses Memory Controller Example: PCI, 33-66MHz 32-64 bits wide 133-528 MBYTES/SEC PCI-X 133MHz 64 bit 1024 MBYTES/SEC CPU Core 1 GHz - 3.8 GHz 4-way Superscaler RISC or RISC-core (x86): Deep Instruction Pipelines Dynamic scheduling Multiple FP, integer FUs Dynamic branch prediction Hardware speculation L1 L2 L3 Memory Bus All Non-blocking caches L1 16-128K 1-2 way set associative (on chip), separate or unified L2 256K- 2M 4-32 way set associative (on chip) unified L3 2-16M 8-32 way set associative (off or on chip) unified Examples: Alpha, AMD K7: EV6, 200-400 MHz Intel PII, PIII: GTL+ 133 MHz Intel P4 800 MHz North Bridge South Bridge Chipset Off or On-chip Current Standard I/O Subsystem

4. November 29, 2015204521 Digital System Architecture4 The Architecture Process New concepts created Estimate Cost & Performance Sort Good ideas Mediocre ideas Bad ideas

5. November 29, 2015204521 Digital System Architecture5 Performance Measurement and Evaluation Many dimensions to computer performance –CPU execution time by instruction or sequence –floating point –integer –branch performance –Cache bandwidth –Main memory bandwidth –I/O performance bandwidth seeks pixels or polygons per second Relative importance depends on applications P C M

6. November 29, 2015204521 Digital System Architecture6 Evaluation Tools Benchmarks, traces, & mixes –macrobenchmarks & suites application execution time –microbenchmarks measure one aspect of performance –traces replay recorded accesses –cache, branch, register Simulation at many levels –ISA, cycle accurate, RTL, gate, circuit trade fidelity for simulation rate Area and delay estimation Analysis –e.g., queuing theory MOVE39% BR20% LOAD20% STORE10% ALU11% LD 5EA3 ST 31FF …. LD 1EA2 ….

7. November 29, 2015204521 Digital System Architecture7 Benchmarks Microbenchmarks –measure one performance dimension cache bandwidth main memory bandwidth procedure call overhead FP performance –weighted combination of microbenchmark performance is a good predictor of application performance –gives insight into the cause of performance bottlenecks Macrobenchmarks –application execution time measures overall performance, but on just one application Perf. Dimensions Applications Micro Macro

8. November 29, 2015204521 Digital System Architecture8 Some Warnings about Benchmarks Benchmarks measure the whole system –application –compiler –operating system –architecture –implementation Popular benchmarks typically reflect yesterday’s programs –computers need to be designed for tomorrow’s programs Benchmark timings often very sensitive to –alignment in cache –location of data on disk –values of data Benchmarks can lead to inbreeding or positive feedback –if you make an operation fast (slow) it will be used more (less) often so you make it faster (slower) –and it gets used even more (less) »and so on…

9. November 29, 2015204521 Digital System Architecture9 Choosing Programs To Evaluate Performance Levels of programs or benchmarks that could be used to evaluate performance: –Actual Target Workload: Full applications that run on the target machine. –Real Full Program-based Benchmarks: Select a specific mix or suite of programs that are typical of targeted applications or workload (e.g SPEC95, SPEC CPU2000). –Small “Kernel” Benchmarks: Key computationally-intensive pieces extracted from real programs. –Examples: Matrix factorization, FFT, tree search, etc. Best used to test specific aspects of the machine. –Microbenchmarks: Small, specially written programs to isolate a specific aspect of performance characteristics: Processing: integer, floating point, local memory, input/output, etc.

10. November 29, 2015204521 Digital System Architecture10 Types of Benchmarks Actual Target Workload Full Application Benchmarks Small “Kernel” Benchmarks Microbenchmarks Pros Cons Representative Very specific. Non-portable. Complex: Difficult to run, or measure. Portable. Widely used. Measurements useful in reality. Easy to run, early in the design cycle. Identify peak performance and potential bottlenecks. Less representative than actual workload. Easy to “fool” by designing hardware to run them well. Peak performance results may be a long way from real application performance

11. November 29, 2015204521 Digital System Architecture11 SPEC: System Performance Evaluation Cooperative The most popular and industry-standard set of CPU benchmarks. SPECmarks, 1989: –10 programs yielding a single number (“SPECmarks”). SPEC92, 1992: –SPECInt92 (6 integer programs) and SPECfp92 (14 floating point programs). SPEC95, 1995: –SPECint95 (8 integer programs): go, m88ksim, gcc, compress, li, ijpeg, perl, vortex –SPECfp95 (10 floating-point intensive programs): tomcatv, swim, su2cor, hydro2d, mgrid, applu, turb3d, apsi, fppp, wave5 –Performance relative to a Sun SuperSpark I (50 MHz) which is given a score of SPECint95 = SPECfp95 = 1 SPEC CPU2000, 1999: –CINT2000 (11 integer programs). CFP2000 (14 floating-point intensive programs) –Performance relative to a Sun Ultra5_10 (300 MHz) which is given a score of SPECint2000 = SPECfp2000 = 100

12. November 29, 2015204521 Digital System Architecture12 SPEC First Round One program: 99% of time in single line of code New front-end compiler could improve dramatically

13. November 29, 2015204521 Digital System Architecture13 Impact of Means on SPECmark89 for IBM 550 (without and with special compiler option) Ratio to VAX: Time: Weighted Time: ProgramBeforeAfterBeforeAfterBeforeAfter gcc302949518.919.22 espresso353465677.647.86 spice47475105105.695.69 doduc464941385.815.45 nasa7781442581403.431.86 li34341831837.867.86 eqntott404028286.686.68 matrix300787305863.430.37 fpppp908734352.973.07 tomcatv3313820192.011.94 Mean547212410854.4249.99 Geometric Arithmetic Weighted Arith. Ratio1.33Ratio1.16Ratio1.09

14. November 29, 2015204521 Digital System Architecture14 SPEC CPU2000 Programs Benchmark Language Descriptions 164.gzip C Compression 175.vpr C FPGA Circuit Placement and Routing 176.gcc C C Programming Language Compiler 181.mcf C Combinatorial Optimization 186.crafty C Game Playing: Chess 197.parser C Word Processing 252.eon C++ Computer Visualization 253.perlbmk C PERL Programming Language 254.gap C Group Theory, Interpreter 255.vortex C Object-oriented Database 256.bzip2 C Compression 300.twolf C Place and Route Simulator 168.wupwise Fortran 77 Physics / Quantum Chromodynamics 171.swim Fortran 77 Shallow Water Modeling 172.mgrid Fortran 77 Multi-grid Solver: 3D Potential Field 173.applu Fortran 77 Parabolic / Elliptic Partial Differential Equations 177.mesa C 3-D Graphics Library 178.galgel Fortran 90 Computational Fluid Dynamics 179.art C Image Recognition / Neural Networks 183.equake C Seismic Wave Propagation Simulation 187.facerec Fortran 90 Image Processing: Face Recognition 188.ammp C Computational Chemistry 189.lucas Fortran 90 Number Theory / Primality Testing 191.fma3d Fortran 90 Finite-element Crash Simulation 200.sixtrack Fortran 77 High Energy Nuclear Physics Accelerator Design 301.apsi Fortran 77 Meteorology: Pollutant Distribution CINT2000 (Integer) CFP2000 (Floating Point) Source: http://www.spec.org/osg/cpu2000/ Programs application domain: Engineering and scientific computation

15. November 29, 2015204521 Digital System Architecture15 Top 20 SPEC CPU2000 Results (As of March 2002) # MHz Processor int peak int baseMHz Processor fp peak fp base 1 1300 POWER4 814 790 1300 POWER4 1169 1098 2 2200 Pentium 4 811 790 1000 Alpha 21264C 960 776 32200 Pentium 4 Xeon 810 7881050 UltraSPARC-III Cu 827 701 4 1667 Athlon XP 724 697 2200 Pentium 4 Xeon 802 779 5 1000 Alpha 21264C 679 621 2200 Pentium 4 801 779 6 1400 Pentium III 664 648 833 Alpha 21264B 784 643 7 1050 UltraSPARC-III Cu 610 537 800 Itanium 701 701 8 1533 Athlon MP 609 587 833 Alpha 21264A 644 571 9 750 PA-RISC 8700 604 568 1667 Athlon XP 642 596 10 833 Alpha 21264B 571 497 750 PA-RISC 8700 581 526 11 1400 Athlon 554 495 1533 Athlon MP 547 504 12 833 Alpha 21264A 533 511 600 MIPS R14000 529 499 13 600 MIPS R14000 500 483 675 SPARC64 GP 509 371 14 675 SPARC64 GP 478 449 900 UltraSPARC-III 482 427 15 900 UltraSPARC-III 467 438 1400 Athlon 458 426 16 552 PA-RISC 8600 441 417 1400 Pentium III 456 437 17 750POWER RS64-IV 439 409 500 PA-RISC 8600 440 397 18 700 Pentium III Xeon 438 431 450 POWER3-II 433 426 19 800 Itanium 365 358 500 Alpha 21264 422 383 20 400 MIPS R12000 353 328 400 MIPS R12000 407 382 Source: http://www.aceshardware.com/SPECmine/top.jsp Top 20 SPECfp2000 Top 20 SPECint2000

16. November 29, 2015204521 Digital System Architecture16 Common Benchmarking Mistakes (1/2) Only average behavior represented in test workload Skewness of device demands ignored Loading level controlled inappropriately Caching effects ignored Buffer sizes not appropriate Inaccuracies due to sampling ignored

17. November 29, 2015204521 Digital System Architecture17 Common Benchmarking Mistakes (2/2) Ignoring monitoring overhead Not validating measurements Not ensuring same initial conditions Not measuring transient (cold start) performance Using device utilizations for performance comparisons Collecting too much data but doing too little analysis

18. November 29, 2015204521 Digital System Architecture Architectural Performance Laws and Rules of Thumb

19. November 29, 2015204521 Digital System Architecture19 Measurement and Evaluation Architecture is an iterative process: –Searching the space of possible designs –Make selections –Evaluate the selections made Good measurement tools are required to accurately evaluate the selection.

20. November 29, 2015204521 Digital System Architecture20 Measurement Tools Benchmarks, Traces, Mixes Cost, delay, area, power estimation Simulation (many levels) –ISA, RTL, Gate, Circuit Queuing Theory Rules of Thumb Fundamental Laws

21. November 29, 2015204521 Digital System Architecture21 Measuring and Reporting Performance What do we mean by one Computer is faster than another? –program runs less time Response time or execution time –time that users see the output Elapsed time –A latency to complete a task including disk accesses, memory accesses, I/O activities, operating system overhead Throughput –total amount of work done in a given time

22. November 29, 2015204521 Digital System Architecture22 Performance “Increasing and decreasing” ????? We use the term “improve performance” or “ improve execution time” When we mean increase performance and decrease execution time. improve performance = increase performance improve execution time = decrease execution time

23. November 29, 2015204521 Digital System Architecture Metrics of Performance Compiler Programming Language Application Datapath Control TransistorsWiresPins ISA Function Units (millions) of Instructions per second: MIPS (millions) of (FP) operations per second: MFLOP/s Cycles per second (clock rate) Megabytes per second Answers per month Operations per second

24. November 29, 2015204521 Digital System Architecture Does Anybody Really Know What Time it is? User CPU Time (Time spent in program) System CPU Time (Time spent in OS) Elapsed Time (Response Time = 159 Sec.) (90.7+12.9)/159 * 100 = 65%, % of lapsed time that is CPU time. 45% of the time spent in I/O or running other programs UNIX Time Command: 90.7u 12.9s 2:39 65%

25. November 29, 2015204521 Digital System Architecture25 Example UNIX time command 90.7u 12.95 2:39 65% user CPU time is 90.7 sec system CPU time is 12.9 sec elapsed time is 2 min 39 sec. (159 sec) % of elapsed time that is CPU time is 90.7 + 12.9 = 65% 159

26. November 29, 2015204521 Digital System Architecture26 Time CPU time –time the CPU is computing –not including the time waiting for I/O or running other program User CPU time –CPU time spent in the program System CPU time –CPU time spent in the operating system performing task requested by the program decrease execution time CPU time = User CPU time + System CPU time

27. November 29, 2015204521 Digital System Architecture27 Performance System Performance –elapsed time on unloaded system CPU performance –user CPU time on an unloaded system

28. November 29, 2015204521 Digital System Architecture28 Two notions of “performance” ° Time to do the task (Execution Time) – execution time, response time, latency ° Tasks per day, hour, week, sec, ns... – throughput, bandwidth Response time and throughput often are in opposition DC to Paris 6.5 hours 3 hours Plane Boeing 747 BAD/Sud Concorde Speed 610 mph 1350 mph Passengers 470 132 Throughput 286,700 178,200 Which has higher performance?

29. November 29, 2015204521 Digital System Architecture29 Example Time of Concorde vs. Boeing 747? Concord is 1350 mph / 610 mph = 2.2 times faster = 6.5 hours / 3 hours Throughput of Concorde vs. Boeing 747 ? Concord is 178,200 pmph / 286,700 pmph = 0.62 “times faster” Boeing is 286,700 pmph / 178,200 pmph = 1.6 “times faster” Boeing is 1.6 times (“60%”)faster in terms of throughput Concord is 2.2 times (“120%”) faster in terms of flying time We will focus primarily on execution time for a single job

30. November 29, 2015204521 Digital System Architecture30 Computer Performance Measures: Program Execution Time (1/2) For a specific program compiled to run on a specific machine (CPU) “A”, the following parameters are provided: –The total instruction count of the program. –The average number of cycles per instruction (average CPI). –Clock cycle of machine “A”

31. November 29, 2015204521 Digital System Architecture31 Computer Performance Measures: Program Execution Time (2/2) How can one measure the performance of this machine running this program? –Intuitively the machine is said to be faster or has better performance running this program if the total execution time is shorter. –Thus the inverse of the total measured program execution time is a possible performance measure or metric: Performance A = 1 / Execution Time A How to compare performance of different machines? What factors affect performance? How to improve performance?!!!!

32. November 29, 2015204521 Digital System Architecture32 Comparing Computer Performance Using Execution Time To compare the performance of two machines (or CPUs) “A”, “B” running a given specific program: Performance A = 1 / Execution Time A Performance B = 1 / Execution Time B Machine A is n times faster than machine B means (or slower? if n < 1) : Speedup = n = = Performance A Performance B Execution Time B Execution Time A

33. November 29, 2015204521 Digital System Architecture33 Example For a given program: Execution time on machine A: Execution A = 1 second Execution time on machine B: Execution B = 10 seconds The performance of machine A is 10 times the performance of machine B when running this program, or: Machine A is said to be 10 times faster than machine B when running this program. The two CPUs may target different ISAs provided the program is written in a high level language (HLL)

34. November 29, 2015204521 Digital System Architecture34 CPU Execution Time: The CPU Equation A program is comprised of a number of instructions executed, I –Measured in:instructions/program The average instruction executed takes a number of cycles per instruction (CPI) to be completed. –Measured in: cycles/instruction, CPI CPU has a fixed clock cycle time C = 1/clock rate –Measured in: seconds/cycle CPU execution time is the product of the above three parameters as follows: CPU time = Seconds = Instructions x Cycles x Seconds Program Program Instruction Cycle CPU time = Seconds = Instructions x Cycles x Seconds Program Program Instruction Cycle T = I x CPI x C execution Time per program in seconds Number of instructions executed Average CPI for programCPU Clock Cycle

35. November 29, 2015204521 Digital System Architecture35 CPU Execution Time: Example A Program is running on a specific machine with the following parameters: –Total executed instruction count: 10,000,000 instructions Average CPI for the program: 2.5 cycles/instruction. –CPU clock rate: 200 MHz. (clock cycle = 5x10 -9 seconds) What is the execution time for this program: CPU time = Instruction count x CPI x Clock cycle = 10,000,000 x 2.5 x 1 / clock rate = 10,000,000 x 2.5 x 5x10 -9 =.125 seconds CPU time= Seconds= Instructions x Cycles x Seconds Program Program Instruction Cycle CPU time= Seconds= Instructions x Cycles x Seconds Program Program Instruction Cycle

36. November 29, 2015204521 Digital System Architecture36 Aspects of CPU Execution Time CPU Time = Instruction count x CPI x Clock cycle Instruction Count I ClockCycle C CPI Depends on: CPU Organization Technology (VLSI) Depends on: Program Used Compiler ISA CPU Organization Depends on: Program Used Compiler ISA (executed) (Average CPI) T = I x CPI x C

37. November 29, 2015204521 Digital System Architecture37 Factors Affecting CPU Performance CPU time= Seconds= Instructions x Cycles x Seconds Program Program Instruction Cycle CPU time= Seconds= Instructions x Cycles x Seconds Program Program Instruction Cycle CPIClock Cycle C Instruction Count I Program Compiler Organization (CPU Design) Technology (VLSI) Instruction Set Architecture (ISA) X X X X X X X X X

38. November 29, 2015204521 Digital System Architecture38 Performance Comparison: Example From the previous example: A Program is running on a specific machine with the following parameters: –Total executed instruction count, I: 10,000,000 instructions –Average CPI for the program: 2.5 cycles/instruction. –CPU clock rate: 200 MHz. Using the same program with these changes: –A new compiler used: New instruction count 9,500,000 New CPI: 3.0 –Faster CPU implementation: New clock rate = 300 MHZ What is the speedup with the changes? Speedup = (10,000,000 x 2.5 x 5x10 -9 ) / (9,500,000 x 3 x 3.33x10 -9 ) =.125 /.095 = 1.32 or 32 % faster after changes. Speedup= Old Execution Time = I old x CPI old x Clock cycle old New Execution Time I new x CPI new x Clock Cycle new Speedup= Old Execution Time = I old x CPI old x Clock cycle old New Execution Time I new x CPI new x Clock Cycle new

39. November 29, 2015204521 Digital System Architecture39 Instruction Types & CPI Given a program with n types or classes of instructions executed on a given CPU with the following characteristics: C i = Count of instructions of type i CPI i = Cycles per instruction for type i Then: CPI = CPU Clock Cycles / Instruction Count I Where: Instruction Count I =  C i i = 1, 2, …. n

40. November 29, 2015204521 Digital System Architecture40 Instruction Types & CPI: An Example An instruction set has three instruction classes: Two code sequences have the following instruction counts: CPU cycles for sequence 1 = 2 x 1 + 1 x 2 + 2 x 3 = 10 cycles CPI for sequence 1 = clock cycles / instruction count = 10 /5 = 2 CPU cycles for sequence 2 = 4 x 1 + 1 x 2 + 1 x 3 = 9 cycles CPI for sequence 2 = 9 / 6 = 1.5 Instruction class CPI A 1 B 2 C 3 Instruction counts for instruction class Code Sequence A B C 1 2 1 2 2 4 1 1 For a specific CPU design

41. November 29, 2015204521 Digital System Architecture41 Instruction Frequency & CPI Given a program with n types or classes of instructions with the following characteristics: C i = Count of instructions of type i CPI i = Average cycles per instruction of type i F i = Frequency or fraction of instruction type i executed = C i / total executed instruction count = C i / I Then: Fraction of total execution time for instructions of type i = CPI i x F i CPI

42. November 29, 2015204521 Digital System Architecture42 Instruction Type Frequency & CPI: A RISC Example Typical Mix Base Machine (Reg / Reg) Op Freq, F i CPI i CPI i x F i % Time ALU50% 1.5 23% =.5/2.2 Load20% 5 1.0 45% = 1/2.2 Store10% 3.3 14% =.3/2.2 Branch20% 2.4 18% =.4/2.2 CPI i x F i CPI Sum = 2.2 Program Profile or Executed Instructions Mix

43. November 29, 2015204521 Digital System Architecture43 Performance Terminology “X is n% faster than Y” means: ExTime(Y) Performance(X) n --------- =-------------- = 1 + ----- ExTime(X)Performance(Y) 100 n = 100(Performance(X) - Performance(Y)) Performance(Y) n = 100(ExTime(Y) - ExTime(X)) ExTime(X)

44. November 29, 2015204521 Digital System Architecture Example Example: Y takes 15 seconds to complete a task, X takes 10 seconds. What % faster is X? n = 100(ExTime(Y) - ExTime(X)) ExTime(X) n = 100(15 - 10) 10 n = 50%

45. November 29, 2015204521 Digital System Architecture45 Speedup Speedup due to enhancement E: ExTime w/o E Performance w/ E Speedup(E) = ------------- = ------------------- ExTime w/ E Performance w/o E Suppose that enhancement E accelerates a fraction enhanced of the task by a factor Speedup enhanced, and the remainder of the task is unaffected, then what is ExTime(E) = ? Speedup(E) = ?

46. November 29, 2015204521 Digital System Architecture46 Amdahl’s Law States that the performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time faster mode can be used Speedup = Performance for entire task using the enhancement Performance for the entire task without using the enhancement or Speedup = Execution time without the enhancement Execution time for entire task using the enhancement

47. November 29, 2015204521 Digital System Architecture Amdahl’s Law ExTime new = ExTime old x (1 - Fraction enhanced ) + Fraction enhanced Speedup overall = ExTime old ExTime new Speedup enhanced = 1 (1 - Fraction enhanced ) + Fraction enhanced Speedup enhanced

48. November 29, 2015204521 Digital System Architecture Example of Amdahl’s Law Floating point instructions improved to run 2X; but only 10% of actual instructions are FP Speedup overall = ExTime new =

49. November 29, 2015204521 Digital System Architecture Example of Amdahl’s Law Floating point instructions improved to run 2X; but only 10% of actual instructions are FP Speedup overall = 1 0.95 =1.053 ExTime new = ExTime old x (0.9 +.1/2) = 0.95 x ExTime old

50. November 29, 2015204521 Digital System Architecture50 Performance Enhancement Calculations: Amdahl's Law The performance enhancement possible due to a given design improvement is limited by the amount that the improved feature is used Amdahl’s Law: Performance improvement or speedup due to enhancement E: Execution Time without E Performance with E Speedup(E) = -------------------------------------- = --------------------------------- Execution Time with E Performance without E –Suppose that enhancement E accelerates a fraction F of the execution time by a factor S and the remainder of the time is unaffected then: Execution Time with E = ((1-F) + F/S) X Execution Time without E Hence speedup is given by: Execution Time without E 1 Speedup(E) = --------------------------------------------------------- = -------------------- ((1 - F) + F/S) X Execution Time without E (1 - F) + F/S

51. November 29, 2015204521 Digital System Architecture51 Pictorial Depiction of Amdahl’s Law Before: Execution Time without enhancement E: (Before enhancement is applied) After: Execution Time with enhancement E: Enhancement E accelerates fraction F of original execution time by a factor of S Unaffected fraction: (1- F) Affected fraction: F Unaffected fraction: (1- F) F/S Unchanged Execution Time without enhancement E 1 Speedup(E) = ------------------------------------------------------ = ------------------ Execution Time with enhancement E (1 - F) + F/S shown normalized to 1 = (1-F) + F =1

52. November 29, 2015204521 Digital System Architecture52 Performance Enhancement Example For the RISC machine with the following instruction mix given earlier: Op FreqCyclesCPI(i)% Time ALU 50% 1.5 23% Load 20% 5 1.0 45% Store 10% 3.3 14% Branch 20% 2.4 18% If a CPU design enhancement improves the CPI of load instructions from 5 to 2, what is the resulting performance improvement from this enhancement: Fraction enhanced = F = 45% or.45 Unaffected fraction = 100% - 45% = 55% or.55 Factor of enhancement = 5/2 = 2.5 Using Amdahl’s Law: 1 1 Speedup(E) = ------------------ = --------------------- = 1.37 (1 - F) + F/S.55 +.45/2.5

53. November 29, 2015204521 Digital System Architecture53 An Alternative Solution Using CPU Equation Op FreqCyclesCPI(i)% Time ALU 50% 1.5 23% Load 20% 5 1.0 45% Store 10% 3.3 14% Branch 20% 2.4 18% If a CPU design enhancement improves the CPI of load instructions from 5 to 2, what is the resulting performance improvement from this enhancement: Old CPI = 2.2 New CPI =.5 x 1 +.2 x 2 +.1 x 3 +.2 x 2 = 1.6 Original Execution Time Instruction count x old CPI x clock cycle Speedup(E) = ----------------------------------- = ---------------------------------------------------------------- New Execution Time Instruction count x new CPI x clock cycle old CPI 2.2 = ------------ = --------- = 1.37 new CPI 1.6 Which is the same speedup obtained from Amdahl’s Law in the first solution.

54. November 29, 2015204521 Digital System Architecture54 Performance Enhancement Example (1/2) A program runs in 100 seconds on a machine with multiply operations responsible for 80 seconds of this time. By how much must the speed of multiplication be improved to make the program four times faster? 100 Desired speedup = 4 = ----------------------------------------------------- Execution Time with enhancement  Execution time with enhancement = 25 seconds 25 seconds = (100 - 80 seconds) + 80 seconds / n 25 seconds = 20 seconds + 80 seconds / n  5 = 80 seconds / n  n = 80/5 = 16 Hence multiplication should be 16 times faster to get a speedup of 4.

55. November 29, 2015204521 Digital System Architecture55 Performance Enhancement Example (2/2) For the previous example with a program running in 100 seconds on a machine with multiply operations responsible for 80 seconds of this time. By how much must the speed of multiplication be improved to make the program five times faster? 100 Desired speedup = 5 = ----------------------------------------------------- Execution Time with enhancement   Execution time with enhancement = 20 seconds 20 seconds = (100 - 80 seconds) + 80 seconds / n 20 seconds = 20 seconds + 80 seconds / n  0 = 80 seconds / n No amount of multiplication speed improvement can achieve this.

56. November 29, 2015204521 Digital System Architecture56 Extending Amdahl's Law To Multiple Enhancements Suppose that enhancement E i accelerates a fraction F i of the execution time by a factor S i and the remainder of the time is unaffected then: Note: All fractions F i refer to original execution time before the enhancements are applied..

57. November 29, 2015204521 Digital System Architecture57 Amdahl's Law With Multiple Enhancements: Example Three CPU performance enhancements are proposed with the following speedups and percentage of the code execution time affected: Speedup 1 = S 1 = 10Percentage 1 = F 1 = 20% Speedup 2 = S 2 = 15Percentage 1 = F 2 = 15% Speedup 3 = S 3 = 30Percentage 1 = F 3 = 10% While all three enhancements are in place in the new design, each enhancement affects a different portion of the code and only one enhancement can be used at a time. What is the resulting overall speedup? Speedup = 1 / [(1 -.2 -.15 -.1) +.2/10 +.15/15 +.1/30)] = 1 / [.55 +.0333 ] = 1 /.5833 = 1.71

58. November 29, 2015204521 Digital System Architecture58 Pictorial Depiction of Example Before: Execution Time with no enhancements: 1 After: Execution Time with enhancements:.55 +.02 +.01 +.00333 =.5833 Speedup = 1 /.5833 = 1.71 Note: All fractions (F i, i = 1, 2, 3) refer to original execution time. Unaffected, fraction:.55 Unchanged Unaffected, fraction:.55 F 1 =.2 F 2 =.15 F 3 =.1 S 1 = 10S 2 = 15S 3 = 30 / 10 / 30 / 15

59. November 29, 2015204521 Digital System Architecture59 Corollary: Make The Common Case Fast All instructions require an instruction fetch, only a fraction require a data fetch/store. –Optimize instruction access over data access Programs exhibit locality –90% of time in 10% of code –Spatial Locality –Temporal Locality Access to small memories is faster –Provide a storage hierarchy such that the most frequent accesses are to the smallest (closest) memories. Reg's Cache Memory Disk / Tape

60. November 29, 2015204521 Digital System Architecture60 Marketing Metrics MIPS = Instruction Count / Time * 10^6 = Clock Rate / CPI * 10^6 Machines with different instruction sets ? Programs with different instruction mixes ? – Dynamic frequency of instructions Uncorrelated with performance MFLOP/s = FP Operations / Time * 10^6 Machine dependent Often not where time is spent Normalized: add,sub,compare,mult 1 divide, sqrt 4 exp, sin,... 8 Normalized: add,sub,compare,mult 1 divide, sqrt 4 exp, sin,... 8

61. November 29, 2015204521 Digital System Architecture61 Means Arithmetic mean Geometric mean Harmonic mean Consistent independent of reference Can be weighted. Represents total execution time

62. November 29, 2015204521 Digital System Architecture62 How to Summarize Performance (1/2) Arithmetic mean (weighted arithmetic mean) tracks execution time: Harmonic mean (weighted harmonic mean) of rates (e.g., MFLOPS) tracks execution time:

63. November 29, 2015204521 Digital System Architecture63 How to Summarize Performance (2/2) Normalized execution time is handy for scaling performance (e.g., X times faster than SPARCstation 10) –Arithmetic mean impacted by choice of reference machine Use the geometric mean for comparison: –Independent of chosen machine –but not good metric for total execution time

64. November 29, 2015204521 Digital System Architecture64 Summarizing Performance Arithmetic mean –Average execution time –Gives more weight to longer-running programs Weighted arithmetic mean –More important programs can be emphasized –But what do we use as weights? –Different weight will make different machines look better (see Figure 1.16 in your text book – Hennessey/Patterson 3 rd Edition) (see board for extension of previous example)

65. November 29, 2015204521 Digital System Architecture65 Normalizing & the Geometric Mean Normalized execution time –Take a reference machine R –Time to run A on X / Time to run A on R Geometric mean Neat property of the geometric mean: Consistent whatever the reference machine Do not use the arithmetic mean for normalized execution times

66. November 29, 2015204521 Digital System Architecture Which Machine is “Better”? Computer A Computer B Computer C Program P1(sec)1 10 20 Program P2 1000 100 20 Total Time 1001 110 40

67. November 29, 2015204521 Digital System Architecture Weighted Arithmetic Mean Assume three weighting schemes P1/P2 Comp AComp BComp C.5/.5 500.5 55.0 20.909/.09191.82 18.8 20.999/.0012 10.09 20

68. November 29, 2015204521 Digital System Architecture Performance Evaluation Given sales is a function of performance relative to the competition, big investment in improving product as reported by performance summary Good products created then have: –Good benchmarks –Good ways to summarize performance If benchmarks/summary inadequate, then choose between improving product for real programs vs. improving product to get more sales; Sales almost always wins! Execution time is the measure of performance

69. November 29, 2015204521 Digital System Architecture69 Computer Performance Measures : MIPS Rating (1/3) For a specific program running on a specific CPU the MIPS rating is a measure of how many millions of instructions are executed per second: MIPS Rating = Instruction count / (Execution Time x 10 6 ) = Instruction count / (CPU clocks x Cycle time x 10 6 ) = (Instruction count x Clock rate) / (Instruction count x CPI x 10 6 ) = Clock rate / (CPI x 10 6 ) Major problem with MIPS rating: As shown above the MIPS rating does not account for the count of instructions executed (I). –A higher MIPS rating in many cases may not mean higher performance or better execution time. i.e. due to compiler design variations.

70. November 29, 2015204521 Digital System Architecture70 In addition the MIPS rating: –Does not account for the instruction set architecture (ISA) used. Thus it cannot be used to compare computers/CPUs with different instruction sets. –Easy to abuse: Program used to get the MIPS rating is often omitted. Often the Peak MIPS rating is provided for a given CPU which is obtained using a program comprised entirely of instructions with the lowest CPI for the given CPU design which does not represent real programs. Computer Performance Measures : MIPS Rating (2/3)

71. November 29, 2015204521 Digital System Architecture71 Computer Performance Measures : MIPS Rating (3/3) Under what conditions can the MIPS rating be used to compare performance of different CPUs? The MIPS rating is only valid to compare the performance of different CPUs provided that the following conditions are satisfied: 1The same program is used (actually this applies to all performance metrics) 2The same ISA is used 3The same compiler is used  (Thus the resulting programs used to run on the CPUs and obtain the MIPS rating are identical at the machine code level including the same instruction count)

72. November 29, 2015204521 Digital System Architecture72 Compiler Variations, MIPS, Performance: An Example (1/2) For the machine with instruction classes: For a given program two compilers produced the following instruction counts: The machine is assumed to run at a clock rate of 100 MHz Instruction class CPI A 1 B 2 C 3 Instruction counts (in millions) for each instruction class Code from: A B C Compiler 1 5 1 1 Compiler 2 10 1 1

73. November 29, 2015204521 Digital System Architecture73 Compiler Variations, MIPS, Performance: An Example (2/2) MIPS = Clock rate / (CPI x 10 6 ) = 100 MHz / (CPI x 10 6 ) CPI = CPU execution cycles / Instructions count CPU time = Instruction count x CPI / Clock rate For compiler 1: –CPI 1 = (5 x 1 + 1 x 2 + 1 x 3) / (5 + 1 + 1) = 10 / 7 = 1.43 –MIP Rating 1 = 100 / (1.428 x 10 6 ) = 70.0 –CPU time 1 = ((5 + 1 + 1) x 10 6 x 1.43) / (100 x 10 6 ) = 0.10 seconds For compiler 2: –CPI 2 = (10 x 1 + 1 x 2 + 1 x 3) / (10 + 1 + 1) = 15 / 12 = 1.25 –MIPS Rating 2 = 100 / (1.25 x 10 6 ) = 80.0 –CPU time 2 = ((10 + 1 + 1) x 10 6 x 1.25) / (100 x 10 6 ) = 0.15 seconds

74. November 29, 2015204521 Digital System Architecture74 Computer Performance Measures :MFLOPS (1/2) A floating-point operation is an addition, subtraction, multiplication, or division operation applied to numbers represented by a single or a double precision floating-point representation. MFLOPS, for a specific program running on a specific computer, is a measure of millions of floating point-operation (megaflops) per second: MFLOPS = Number of floating-point operations / (Execution time x 10 6 ) MFLOPS rating is a better comparison measure between different machines (applies even if ISAs are different) than the MIPS rating. –Applicable even if ISAs are different

75. November 29, 2015204521 Digital System Architecture75 Computer Performance Measures :MFLOPS (2/2) Program-dependent: Different programs have different percentages of floating- point operations present. i.e compilers have no floating- point operations and yield a MFLOPS rating of zero. Dependent on the type of floating-point operations present in the program. –Peak MFLOPS rating for a CPU: Obtained using a program comprised entirely of the simplest floating point instructions (with the lowest CPI) for the given CPU design which does not represent real floating point programs.

76. November 29, 2015204521 Digital System Architecture76 Other ways to measure performance (1) Use MIPS (millions of instructions/second) MIPS is a rate of operations/unit time. Performance can be specified as the inverse of execution time so faster machines have a higher MIPS rating So, bigger MIPS = faster machine. Right? MIPS = Exec. Time x 10 6 Instruction Count CPI x 10 6 Clock Rate =

77. November 29, 2015204521 Digital System Architecture77 Wrong!!! 3 significant problems with using MIPS: –Problem 1: MIPS is instruction set dependent. (And different computer brands usually have different instruction sets) –Problem 2: MIPS varies between programs on the same computer –Problem 3: MIPS can vary inversely to performance! Let’s look at an example of why MIPS doesn’t work…

78. November 29, 2015204521 Digital System Architecture78 A MIPS Example (1) Consider the following computer: Code from:ABC Compiler 1511 Compiler 21011 Instruction counts (in millions) for each instruction class The machine runs at 100MHz. Instruction A requires 1 clock cycle, Instruction B requires 2 clock cycles, Instruction C requires 3 clock cycles.  CPI i x C i i =1 n CPI = Instruction Count CPU Clock Cycles Instruction Count = ! Note important formula!

79. November 29, 2015204521 Digital System Architecture79 A MIPS Example (2) CPI 1 = (5 + 1 + 1) x 10 6 [(5x1) + (1x2) + (1x3)] x 10 6 10/7 = 1.43= MIPS 1 = 1.43 100 MHz 69.9= CPI 2 = (10 + 1 + 1) x 10 6 [(10x1) + (1x2) + (1x3)] x 10 6 15/12 = 1.25= MIPS 2 = 1.25 100 MHz 80.0= So, compiler 2 has a higher MIPS rating and should be faster? count cycles

80. November 29, 2015204521 Digital System Architecture80 A MIPS Example (3) Now let’s compare CPU time: CPU Time = Clock Rate Instruction Count x CPI = 0.10 secondsCPU Time 1 = 100 x 10 6 7 x 10 6 x 1.43 = 0.15 seconds CPU Time 2 = 100 x 10 6 12 x 10 6 x 1.25 Therefore program 1 is faster despite a lower MIPS! ! Note important formula!