DCS/1 CENG Distributed Computing Systems Measures of Performance

DCS/2 Grosch’s Law-1960’s “ To sell a computer twice as much, it must be four times as fast” It was Ok at the time, but soon it became meaningless After 1970, it was possible to make faster computers and sell even cheaper…. Ultimately the switching speeds reach a limit, which is the speed of the light on an integrated circuit…

DCS/3 Von Neumann’s Bottleneck Serial single processor computer architectures based on John Von Neumann’s architecture of has: One processor, single control unit, single memory This is no more valid: Low cost parallel computers can easily deliver the performance of the fastest single processor computer…

DCS/4 Amdahl’s Law; 1967 Amdahl’s law is still valid! Let speedup (S) be ratio of serial time (one processor) to parallel time (N processors) S=T 1 /T N < 1/f Where f is the serial fraction of the problem, 1-f is the parallel fraction of the problem, T 1 is one processor sequential time /T N is N processor parallel time, then The proof of Amdahl’s law: T N = T 1 *f+T 1 (1-f)/N S=1/(f+(1-f)/N), thus S < 1/f

DCS/5 Amdahl’s Law; 1967 At f=0.10, Amdahl’ Law predicts, at best a tenfold speedup, which is very pessimistic This was soon broken, encouraged by Gordon Bell Prize*! * Gordon Bell is computer scientist contributing to parallel computing while at DEC

DCS/6 Gustafson-Barsis Law; 1988 The team of researchers of Sandia Labs (John Gustafson and Ed Barsis), using 1024 processor nCube/10, overthrew Amdahl’s Law, by achieving 1000 fold speedup with f=0.004 to According to Amdahl’s Law, the speedup would have been from 125 to 250. The key point was found to be that 1-f was not independent of N. The relationship between N and 1-f may not be linear… Parallel algorithms may perform better than their sequential counter parts.

DCS/7 Gustafson-Barsis Law; 1988 They interpreted the speedup formula, by scaling up the problem to fit the parallel machine: T 1 =f+(1-f)N After redefining T N as T N =f+(1-f)=1, then the speedup can be computed as S=T1/TN= (f+(1-f)N)/1= f+N-Nf= S=N-(N-1)f

DCS/8 Extreme case analysis Assuming Amdahl’s Law, an upper and lower bound can be given for the speedup!: N/log 2 N <= S <= N where logN is based on divide and conquer

DCS/9 Inclusion of the communication time Some researchers (Gelenbe) suggests speedup to be approximated by S=1/C ( N) where C(N) is some function of N For example, C(N) can be estimated as C (N) =A+Blog 2 N where A and B are constants determined by the communication mechanisms

DCS/10 Benchmark Performance Benchmark is a program whose purpose is to measure a performance characteristic of a computer system, such as floating point speed, I/O speed, or for a restricted class of problems The benchmarks are arranged to be either Kernels of real applications, such as Linpacks, Livermore Loops, or Synthetic, approximating the behavior of the real problem, such as Whetstone and Wichmann…These benchmarks were synthetic, consisting of artificial kernels intended to represent the computationally intensive part of certain scientific codes. They have been in use since 1972 …

DCS/11 Scalability example  System and the application should experience performance proportional to the scale of the change in the system and the application.  For example, if more computers are added to the system, the performance experienced should increase proportional to the addition.  A change in a system or an application should not require other changes in the system or applications.  The scalability is generally achieved through replication of hardware, software, and data.

DCS/12 Scalability example  Speedup S of an application on a system of P processors is equal to Ts/Tp= P (ideally)  Efficiency E =S/P=1 (ideally)  Scalability of a system is a measure of its capability to increase speed-up in proportion to the number of processors (and/or other resources) effectively.  Scalability of adding n numbers on a hypercube of p processors.  Let p = 8 (p = 2 d, d is the degree of a processor), let n = func(p log p)

DCS/13 Scalability example  Assume that it takes one time unit both to add two numbers and to communicate a number between two adjacent processors. Thus, parallel time Tp Tp = [n/p] + 2log p  Where n/p is serial time per node and 2log p is parallel addition of two numbers and communication of one number (there are logp or d such operations).

DCS/14 Scalability example  Note that sequential time, Ts = n, Thus, S and E will be as follows: S = Ts/Tp = [np] / [n+2plog p] E = S/p = [n]/[n+ 2plog p] E = 0.8 for n = 64 p = 4, note that n = 8plogp n = 192 p = 8 n = 512 p = 16  Hence, system is scalable! increase in the system size is reflected in the application size, yet the efficiency is fixed.

DCS/15 Fault Tolerance  " The fault tolerant systems will survive faults, thus securing the correct operation of the applications and the servers, thus improving the availability of the system.  Fault tolerance can be achieved by: Hardware and software redundancy Software recovery.  " DS provide high degree of availability in theface of hardware faults.

DCS/16 Transparency  To present the user the system as a whole rather than a collection of components.  This is an important issue in characterization of the distributed systems.  " Transparency allows a distributed system to be distinguished from network systems.  More transparency more distributed system!  " There are eight forms of transparencies accepted by an international standardization work in this field: access, location, concurrency, replication, failure, migration, performance and scaling

DCS/17