“elbowing out”

Processors used Speedup Efficiency timeexecution Parallel Processors timeexecution Sequential Efficiency   

All terms > 0   (n,p) > 0 Denominator > numerator   (n,p) < 1

Let f =  (n)/(  (n) +  (n)); i.e., f is the fraction of the code which is inherently sequential

15 CS 491 – Parallel and Distributed Computing

n = 100 n = 1,000 n = 10,000 Speedup Processors

Let T p =  (n)+  (n)/p = 1 unit Let s be the fraction of time that a parallel program spends executing the serial portion of the code. s =  (n)/(  (n)+  (n)/p) Then,   = T 1 /T p = T 1 <= s + p*(1-s) (the scaled speedup) Thus, sequential time would be p times the parallelized portion of the code plus the time for the sequential portion.

  <= s + p*(1-s) (the scaled speedup) Restated, Thus, sequential time would be p times the parallel execution time minus (p-1) times the sequential portion of execution time.

Execution on 1 CPU takes 10 times as long… …except 9 do not have to execute serial code

Inherently serial component of parallel computation + processor communication and synchronization overhead Single processor execution time

p  What is the primary reason for speedup of only 4.7 on 8 CPUs? e0.1 Since e is constant, large serial fraction is the primary reason.

p  What is the primary reason for speedup of only 4.7 on 8 CPUs? e Since e is steadily increasing, overhead is the primary reason.

Determine overhead Substitute overhead into speedup equation Substitute T(n,1) =  (n) +  (n). Assume efficiency is constant. Hence, T 0 /T 1 should be a constant fraction. Isoefficiency Relation

where Isoefficiency Relation Problem size Sequential complexity Serial and communication overhead

Number of processors Memory needed per processor Cplogp Cp Clogp C Memory Size per processor Can maintain efficiency Cannot maintain efficiency