1. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.1 Lecture 8+9 Time complexity 1 Jan Maluszynski, IDA, 2007 http://www.ida.liu.se/~janma

2. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.2 Outline Time Complexity 1 (Sipser 7.1 – 7.2) 1.Motivation 2.Big Oh notation 3.Complexity classes TIME(t(n)) and P 4.Relations between various models of algorithms 5.Analysis of example algorithms

3. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.3 Motivation We want to compare efficiency of algorithms Algorithm : Turing Machine Efficiency: number of steps to accept an input. Other models should also be considered: RAM with pseudocode Real programming languages

4. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.4 Principles of Algorithm Analysis An algorithm works for input of any size. Show the resource (time/memory) used as an increasing function of input size. Focus on the worst case performance. Ignore constant factors analysis should be machine-independent; more powerful computers introduce speed-up by constant factors. Study scalability / asymptotic behaviour for large problem sizes: ignore lower-order terms, focus on dominating terms.

5. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.5 Use asymptotic analysis! Consider two growing functions f, g from natural numbers to positive real numbers: f dominates g iff f(n) / g(n) increases without bounds for n   that is, for a given constant factor c > 0, there is some threshold value such that

6. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.6 Types of growth comparison: f, g growing functions from natural numbers to positive real numbers f is (in) O ( g) iff there exist c > 0, n 0  1 such that f(n )  c g(n) for all n  n 0 Intuition: Apart from constant factors, f grows at most as quickly as g f is (in)  ( g ) iff there exist c > 0, n 0  1 such that f ( n)  c g(n) for all n  n 0 Intuition: Apart from constant factors, f grows at least as quickly as g  is the converse of O, i.e. f is in  (g) iff g is in O(f)  f is (in)  (g) iff f(n)  O(g(n)) and g(n)  O(f(n)) Intuition: Apart from constant factors, f grows exactly as quickly as g

7. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.7 Types of growth comparison:  (g),  (g), O ( g)..??

8. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.8 …comparison with simple math function: 1.84* 10 19 µ sec = 2.14 * 10 8 days = 5845 centuries nlog 2 nnn log 2 nn2n2 2n2n 212244 164 64256 6.5 * 10 4 646 3844096 1.84 * 10 19

9. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.9 Asymptotic analysis: Dominance Relation Revisited Growing functions on natural numbers: f and g there exists f is (in) o ( g) iff there exist c > 0, n 0  1 such that f(n )  c g(n) for all n  n 0 for any c f(n ) < g(n) i.e., f is dominated by g Intuition: g grows strictly more quickly than f. If f  o(g) then f O(g) but not vice versa. Example: n  o(2 n )

10. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.10 Time complexity classes t: N  R + TIME(t(n)) : collection of all languages decidable by an O(t(n)) single tape deterministic TM P: collection of all languages decidable in polynomial time by single tape deterministic TM

11. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.11 Example A = {0 k 1 k |k  0} On input string w: M1: 1.Scan the tape reject if 0 found to the right of 1 2.Repeat if both 0 and 1 are on tape: Scan crossing off single 0 and single 1 3. If there are still some 0’s or some 1’s reject otherwise accept M1 is O(n 2 ); A is in TIME (n 2 );

12. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.12 Example A = {0 k 1 k |k  0} On input string w: M2: 1.Scan the tape reject if 0 found to the right of 1 2.Repeat if both 0 and 1 are on tape: Scan if total number of 1’s and 0’s is odd reject Scan crossing off every second 0 and every second 1 3. If there are still some 0’s or some 1’s reject otherwise accept M1 is O(n log n); A is in TIME (n log n);

13. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.13 Relations between models t: N  R + t(n)  n Every t(n) time multitape Turing machine has an equivalent O(t 2 (n)) single-tape Turing machine. Every t(n) time nondeterministic Turing machine decider has an equivalent 2 +O(t(n)) deterministic single-tape Turing machine. All deterministic computational models are polynomially equivalent.

14. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.14 Examples of problems in P PATH: is there a path from s to t in a graph G? RELPRIME: are natural numbers x and y relatively prime ? CFG parsing: is a string x derivable in a CFG G Searching: is a string x in a set S ? ….

15. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.15 Estimating execution time for iterative programs

16. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.16 Analysis of algorithms in pseudocode What is the worst-case problem instance? What is the worst case time? What is the ”complexity” for this function?

17. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.17 Example: Dependent Nested Loops

18. CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT 20077.18 Analysis of Recursive Program (1)