1. Some Notes on Performance Noah Mendelsohn Tufts University Email: noah@cs.tufts.edunoah@cs.tufts.edu Web: http://www.cs.tufts.edu/~noah COMP 40: Machine structure and assembly language (Spring 2014)

2. © 2010 Noah Mendelsohn 2 Performance Concepts and Terminology

3. © 2010 Noah Mendelsohn Performance, Scalability, Availability, Reliability  Performance –Get a lot done quickly –Preferably a low cost  Scalability –Low barriers to growth –A scalable system isn’t necessarily fast… –…but it can grow without slowing down  Availability –Always there when you need it  Reliability –Never does the wrong thing, never loses or corrupts data 3

4. © 2010 Noah Mendelsohn 4 Where does the time go?

5. © 2010 Noah Mendelsohn Hard disks are slow Platter Sector

6. © 2010 Noah Mendelsohn Know how fast things are  Register – register instruction: ~ 1 cycle (maybe less) –In 2014: our machines do perhaps 2Billion cycles/sec (0.5nsec / cycle)  L0 Cache hit: a few cycles  Memory access (cache miss): 100-200 cycles  Network round trip: ~1ms (local), 10-100ms (long distance)  Disk access: –Depends on where data is on disk –Streaming: several hundred Mbytes/sec –Random access: several msec/seek –Watch for contention from other programs or accessing multiple files. 6

7. © 2010 Noah Mendelsohn Throughput vs. Response Time vs. Latency  Throughput: the aggregate rate at which a system does work  Response time: the time to get a response to a request  Latency: time spent waiting (e.g. for disk or network)  We can improve throughput by: –Minimizing work done per request –Doing enough work at once to keep all hardware resources busy… –…and when some work is delayed (latency) find other work to do  We can improve response time by: –Minimizing total work and delay (latency) on critical path to a response –Applying parallel resources to an individual response…including streaming –Precomputing response values 7

8. © 2010 Noah Mendelsohn 8 Making Systems Faster Understanding Bottlenecks

9. © 2010 Noah Mendelsohn Amdahl’s claim: parallel processing won’t scale 9 1967: Major controversy …will parallel computers work? “Demonstration is made of the continued validity of the single processor approach and of the weaknesses of the multiple processor approach in terms of application to real problems and their attendant irregularities. Gene Amdahl*” * Gene Amdahl, Validity of the single processor approach to achieving large scale computing capabilities, AFIPS spring joint computer conference, 1967 http://www- inst.eecs.berkeley.edu/~n252/paper/Amdahl.pdfhttp://www- inst.eecs.berkeley.edu/~n252/paper/Amdahl.pdf

10. © 2010 Noah Mendelsohn Amdahl: why no parallel scaling? 10 “The first characteristic of interest is the fraction of the computational load which is associated with data management housekeeping. This fraction […might eventually be reduced to 20%...]. The nature of this overhead appears to be sequential so that it is unlikely to be amenable to parallel processing techniques. Overhead alone would then place an upper limit on throughput of five to seven times the sequential processing rate. Gene Amdahl (Ibid) In short: even if the part you’re optimizing went to zero time, the speedup would be only 5x. Speedup = 1/(r s +(r p /n)) where r s and r p are sequential/parallel fractions of computation As r p /n  0, Speedup -> 1/r s

11. © 2010 Noah Mendelsohn Amdahl: why no parallel scaling? 11 “The first characteristic of interest is the fraction of the computational load which is associated with data management housekeeping. This fraction […might eventually be reduced to 20%...]. The nature of this overhead appears to be sequential so that it is unlikely to be amenable to parallel processing techniques. Overhead alone would then place an upper limit on throughput of five to seven times the sequential processing rate. Gene Amdahl (Ibid) In short: even if the part you’re optimizing went to zero time, the speedup would be only 5x. Speedup = 1/(r s +(r p /n)) where r s and r p are sequential/parallel fractions of computation As r p /n  0, Speedup -> 1/r s Although Amdahl was motivated by debates about parallelism, Amdahl’s law is important even for ordinary programs. Informally: Even if you could optimize one part of a program so that it took no time at all, you’d still be left with the rest that you didn’t optimize.