CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Lecture 8+9 Time complexity 1 Jan Maluszynski, IDA,

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Outline Time Complexity 1 (Sipser 7.1 – 7.2) 1.Motivation 2.Big Oh notation 3.Complexity classes TIME(t(n)) and P 4.Relations between various models of algorithms 5.Analysis of example algorithms

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Motivation We want to compare efficiency of algorithms Algorithm : Turing Machine Efficiency: number of steps to accept an input. Other models should also be considered: RAM with pseudocode Real programming languages

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Principles of Algorithm Analysis An algorithm works for input of any size. Show the resource (time/memory) used as an increasing function of input size. Focus on the worst case performance. Ignore constant factors analysis should be machine-independent; more powerful computers introduce speed-up by constant factors. Study scalability / asymptotic behaviour for large problem sizes: ignore lower-order terms, focus on dominating terms.

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Use asymptotic analysis! Consider two growing functions f, g from natural numbers to positive real numbers: f dominates g iff f(n) / g(n) increases without bounds for n   that is, for a given constant factor c > 0, there is some threshold value such that

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Types of growth comparison: f, g growing functions from natural numbers to positive real numbers f is (in) O ( g) iff there exist c > 0, n 0  1 such that f(n )  c g(n) for all n  n 0 Intuition: Apart from constant factors, f grows at most as quickly as g f is (in)  ( g ) iff there exist c > 0, n 0  1 such that f ( n)  c g(n) for all n  n 0 Intuition: Apart from constant factors, f grows at least as quickly as g  is the converse of O, i.e. f is in  (g) iff g is in O(f)  f is (in)  (g) iff f(n)  O(g(n)) and g(n)  O(f(n)) Intuition: Apart from constant factors, f grows exactly as quickly as g

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Types of growth comparison:  (g),  (g), O ( g)..??

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT …comparison with simple math function: 1.84* µ sec = 2.14 * 10 8 days = 5845 centuries nlog 2 nnn log 2 nn2n2 2n2n * * 10 19

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Asymptotic analysis: Dominance Relation Revisited Growing functions on natural numbers: f and g there exists f is (in) o ( g) iff there exist c > 0, n 0  1 such that f(n )  c g(n) for all n  n 0 for any c f(n ) < g(n) i.e., f is dominated by g Intuition: g grows strictly more quickly than f. If f  o(g) then f O(g) but not vice versa. Example: n  o(2 n )

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Time complexity classes t: N  R + TIME(t(n)) : collection of all languages decidable by an O(t(n)) single tape deterministic TM P: collection of all languages decidable in polynomial time by single tape deterministic TM

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Example A = {0 k 1 k |k  0} On input string w: M1: 1.Scan the tape reject if 0 found to the right of 1 2.Repeat if both 0 and 1 are on tape: Scan crossing off single 0 and single 1 3. If there are still some 0’s or some 1’s reject otherwise accept M1 is O(n 2 ); A is in TIME (n 2 );

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Example A = {0 k 1 k |k  0} On input string w: M2: 1.Scan the tape reject if 0 found to the right of 1 2.Repeat if both 0 and 1 are on tape: Scan if total number of 1’s and 0’s is odd reject Scan crossing off every second 0 and every second 1 3. If there are still some 0’s or some 1’s reject otherwise accept M1 is O(n log n); A is in TIME (n log n);

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Relations between models t: N  R + t(n)  n Every t(n) time multitape Turing machine has an equivalent O(t 2 (n)) single-tape Turing machine. Every t(n) time nondeterministic Turing machine decider has an equivalent 2 +O(t(n)) deterministic single-tape Turing machine. All deterministic computational models are polynomially equivalent.

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Examples of problems in P PATH: is there a path from s to t in a graph G? RELPRIME: are natural numbers x and y relatively prime ? CFG parsing: is a string x derivable in a CFG G Searching: is a string x in a set S ? ….

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Estimating execution time for iterative programs

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Analysis of algorithms in pseudocode What is the worst-case problem instance? What is the worst case time? What is the ”complexity” for this function?

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Example: Dependent Nested Loops

CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Analysis of Recursive Program (1)