TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Analysis of Algorithms Introductory Example Principles of Analysis Big-Oh notation Time complexity of iterative algorithms Time complexity of recursive algorithms References: [G&T] 4.2, or [L&D] Chapter 1, , or [CL] Chapter 1-3, or …

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Example problem: Checking results of an exam: Input: –A set of students (name, personal no) who passed the exam –Name of a student Output: ”yes” or ”no” To implement this on a computer we: Characterize the data and the needed operations on data  ADT (e.g ADT Set) Choose a representation of the data in the computer?  datastructure (e.g. table/array) Implement the needed operations: algorithm (e.g. Table search) Analyse the efficiency of the implementation

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT How to measure time/space Describe algorithms in pseudocode (or in a high-level programming language) Analyse number of basic operations as function of the size of input data Use simple data memory model Analyse number of needed memory cells

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Example of pseudocode: function TableSearch ( table T[1..n], key k ):boolean for i from 1 to n do if T [i] = k then return true if T [i] > k then return false return false...”scope” defined by indentation!

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Principles of Algorithm Analysis An algorithm should work for (input) data of any size. (Example TableSearch: input size is the size of the table.) Show the resource (time/memory) used as an increasing function of input size. Focus on the worst case performance. Ignore constant factors analysis should be machine-independent; more powerful computers introduce speed-up by constant factors. Study scalability / asymptotic behaviour for large problem sizes: ignore lower-order terms, focus on dominating terms.

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT ”Order” notation Compares growth rate of functions f  O( g ) ” f grows at most as fast as g” apart of constant factors Refer to simple well known functions f(n) grows aproximately as n 2

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Types of growth comparison: f, g growing functions from natural numbers to positive real numbers f is (in) O ( g) iff there exist c > 0, n 0  1 such that f(n )  c g(n) for all n  n 0 Intuition: Apart from constant factors, f grows at most as quickly as g f is (in)  ( g ) iff there exist c > 0, n 0  1 such that f ( n)  c g(n) for all n  n 0 Intuition: Apart from constant factors, f grows at least as quickly as g  is the converse of O, i.e. f is in  (g) iff g is in O(f)  f is (in)  (g) iff f(n)  O(g(n)) and g(n)  O(f(n)) Intuition: Apart from constant factors, f grows exactly as quickly as g

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Types of growth comparison:  (g),  (g), O ( g)..??

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT …comparison with simple math function: 1.84* µ sec = 2.14 * 10 8 days = 5845 centuries nlog 2 nnn log 2 nn2n2 2n2n * * 10 19

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT To check growth rate?

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Estimating execution time for iterative programs

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Example of ”algebraic” analysis

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Example: Dependent Nested Loops

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Analysis of Recursive Program (1)

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Analysis of Recursive Programs...

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Towers of Hanoi.... As stated earlier: Formulate an equation T(1)=... T(2)=... Unroll a few times, get hypothesis for T(n)=... Prove the hypothesis!

TTIT33 Algorithms and Optimization – Lecture 1 Jan Maluszynski - HT Average case analysis Reconsider TableSearch(): sequential search through a table Input argument: one of the table elements, assume it is chosen with equal probability for all elements. Expected search time: Time to find the element when it was in the n:th place