Lecture 3 of Computer Science II Analysis Tools

Agenda What Is Running Time Anyway? Pseudo-Code Analysis Tools Agenda What Is Running Time Anyway? Pseudo-Code A Quick Mathematical Review Analysis of algorithms Average-Case and Worst-Case Analysis Asymptotic Notation  Page 2

What Is Running Time Anyway? Definition If we want to know which of two methods is faster, the most obvious approach is to time them. Running time : How long it takes a show to go from start to finish.  Page 3

What Is Running Time Anyway? Experimental Studies Run a piece of code and record the time spent in execution. Java.System.currentTimeMillis() In general the time increases with the input size, and is affected by the hardware, and software environment. One of the first steps that we take to understand the performance of algorithms is to do empirical analysis. Given two algorithms to solve the same problem, there is no mystery in the method: We run them both to see which one takes longer! This concept might seem too obvious to mention, but it is an all-too-common omission in the comparative study of algorithms. The fact that one algorithm is 10 times faster than another is unlikely to escape the notice of someone who waits 3 seconds for one to finish and 30 seconds for the other to finish, but it is easy to overlook as a small constant overhead factor in a mathematical analysis. When we monitor the performance of careful implementations on typical input, we get performance results that not only give us a direct indicator of efficiency but also provide us with the information that we need to compare algorithms and to validate any mathematical analyses that may apply.  Page 4

What Is Running Time Anyway? Limitations of running time studies Experiments can be done on a limited set of test inputs, and may not be indicative of other inputs. It is difficult to compare two algorithms unless experiments have the same software, and hardware environments. It is necessary to implement and study an algorithm in order to study its running time.  Page 5

What Is Running Time Anyway? Requirements for a general methodology Takes into account all possible inputs. Allows us to evaluate the efficiency of an algorithm in a way that is independent from hardware and software environment. Can be performed by studying a high-level description of the algorithm without actually implementing it or running experiments on it.  Page 6

Pseudo-Code Definition Pseudo-code (derived from pseudo and code) is a compact and informal high-level description of a computer programming algorithm that uses the structural conventions of programming languages, but omits detailed subroutines, variable declarations or language-specific syntax. The programming language is augmented with natural language descriptions of the details, where convenient.  Page 7

Characteristics High- level description of an algorithm Pseudo-Code Characteristics High- level description of an algorithm More structured than English prose Less detailed than a program Preferred notation for describing algorithms Hides program design issues  Page 8

Pseudo-Code Syntax As the name suggests, pseudo-code generally does not actually obey the syntax rules of any particular language. There is no systematic standard form, although any particular writer will generally borrow the appearance of a particular language.  Page 9

Syntax Expression: Use ← instead of =, Use = instead of == Pseudo-Code Syntax Expression: Use ← instead of =, Use = instead of == Method declaration: algorithm name(A,n) Decision structures: If condition then true-actions else false-actions While-loops: while condition do actions Repeat-loops: repeat actions until condition For-loops: for increment-action do actions Array-indexing: A[i] Method-calling: object.mehtod(args) or simply method(args). Method-returs: return  Page 10

Example Algorithm arrayMax(A, n) Input array A of n integers Pseudo-Code Example Algorithm arrayMax(A, n) Input array A of n integers Output maximum element of A currentMax ← A[0] for i ← 1 to n − 1 do if A[i] > currentMax then currentMax ← A[i] return currentMax  Page 11

Pseudo-Code Vs. Flowchart ? A flowchart (also spelled flow-chart and flow chart) is a schematic representation of an algorithm or a process.  Page 12

A Quick Mathematical Review Logarithms and Exponents One of the interesting and sometimes even surprising aspects of the analysis of data structures and algorithms is the ubiquitous presence of the logarithm function, f(n) = logbn, for some constant b > 1. This function is defined as follows: x = logb n if and only if bx = n. By definition, logb 1 = 0. The value b is known as the base of the logarithm. Since computers store integers in binary, the most common base for the logarithm function in computer science is 2. In fact, this base is so common that we will typically leave it off when it is 2. That is, for us, logn = log2n.  Page 13

A Quick Mathematical Review Preposition Let a, b, and c positive real numbers, we have: a  Page 14

A Quick Mathematical Review Examples 1+logn+loglogn, by rule1 (twice) Logn-log2 = logn-1 , by rule2 Log(n)1/2 = (logn)/2 , by rule3 Log(logn)/2= loglogn – log2= loglogn - 1 , by rules 1.2.3 Log4n = logn / log4 = log n /2 by rule 4 n by rule 3 n by rule 5,3  Page 15

A Quick Mathematical Review Summations  Page 16

A Quick Mathematical Review Proposition For any integer n>=0 and any real number 0 <a 1 , consider the summation n  Page 17

A Quick Mathematical Review Example  Page 18

A Quick Mathematical Review Proposition For any integer n>=1 We have: n  Page 19

Analysis of algorithms Primitive Operations Definition Assigning a value to a variable. Calling a method. Performing an arithmetic operation (e.g., adding two Numbers). Comparing two numbers. Indexing into an array. Following an object reference. Returning from a method.  Page 20

Analysis of algorithms Primitive Operations Analysis approach Code up the algorithm in some high-level computer language. Compile the program into some low-level executable language. Determine for each instruction i of the low-level language, the time ti needed to execute the instruction. Determine for each instruction i of the low-level language, the times ni that instruction i gets executed when the algorithm is run. Sum up the products ni.ti over all the instructions, which yields the running time of the algorithm. Advantages Vs. Disadvantages ?  Page 21

Analysis of algorithms Primitive Operations Example Algorithm arrayMax(A, n) #operations currentMax ← A[0] for i ← 1 to n − 1 do if A[i] < currentMax then currentMax ← A[i] return currentMax  Page 22

Analysis of algorithms Primitive Operations Example Algorithm arrayMax(A, n) #operations currentMax ← A[0] 2 for i ← 1 to n − 1 do 1+ n if A[i] < currentMax then 2(n-1) currentMax ← A[i] at most 2(n-1) { increment counter i} 2(n − 1) return currentMax 1 AT LEAST: 2+1+n+4(n-1)+1=5n-1 AT MOST: 2+1+n+6(n-1)+1=7n-3  Page 23

Average-Case and Worst-Code Analysis Primitive Operations Example Best Case. Worst Case. Average Case. - Expected number of execution times  probability theory  Page 24

Average-Case and Worst-Code Analysis Primitive Operations Example Worst case time Average Running Time ms Best case time An average-case analysis usually requires that we calculate expected running times based on a given input distribution, which usually involves sophisticated probability theory. Worst-case analysis is much easier than average-case analysis, as it requires only the ability to identify the worst-case input, which is often simple. Also, this approach typically leads to better algorithms. Making the standard of success for an algorithm to perform well in the worst case necessarily requires that it will do well on every input. That is, designing for the worst case leads to stronger algorithmic "muscles”. Input Instance  Page 25

Asymptotic Notation Definition Asymptotic Notation is often used to describe how the size of the input data affects an algorithm's usage of computational resources (usually running time or memory).  Page 26