Flynn’s Taxonomy SISD: Although instruction execution may be pipelined, computers in this category can decode only a single instruction in unit time SIMD: An array of processors all executing the same instruction but on different data

Flynn’s Taxonomy MISD: Some consider this category to be empty. However, some consider systolic arrays to fall in this category: –Data is pumped through processors, each processor applying a different operation to the same data stream. MIMD: More than one CPU, each running its own program on its own data

Speedup & Efficiency Speedup = (Execution Time on N processors) / (Execution Time on 1 Single Processor) NB: Must be the most efficient serial algorithm Efficiency = Speedup / N Example: if sequential time = 8 sec, parallel time = 2 sec (on 5 processors), then Speedup = 4, Efficiency = 0.8

Speedup Parallelizability: Same as speedup but comparing against the parallel algorithm running on a 1-processor machine Can speedup be greater than 1? –Called superlinear speedup –Some say not possible because: a single processor can always emulate thru time-slicing –Moreover, since parallelization always contains some overhead, some argue it must be sublinear.

Superlinear Speedup? Some say it is possible because: –sometimes one of the parallel processors may luck out. –Architecturally, emulation is never perfect because of factors such as cache hit rate

Control and Data Parallelism Control Parallelism: Different operations are applied to different data elements simultaneously Data Parallelism: Same operations are applied to different different data Example: Sieve of Erastosthenes

Amdahl’s Law If f is an inherently sequential fraction of execution time, then speedup  1 / [f + (1-f)/p] Corollary: Maximum speedup = 1/f