1. March 24, 2009 Complexity in everyday life 1 (Computational) Complexity: In every day life? Madhu Sudan MIT

2. March 24, 2009Complexity in everyday life2 Theory of Computing? Part I: Origins: Computers and Theory Part I: Origins: Computers and Theory Part II: Modern Complexity Part II: Modern Complexity Part III: Implications to everyday life. Part III: Implications to everyday life. Part IV: Future of computing Part IV: Future of computing

3. March 24, 2009Complexity in everyday life3 Origins of Computation

4. March 24, 2009Complexity in everyday life4 History of Computing Born: 1941 Born: 1941 1946: 1946: 1950s-2000: 1950s-2000: Died: 2005 Died: 2005 Konrad Zuse Z3 Eckert-Mauchly ENIAC ? ? Internet, Mainframes, PCs, WWW

5. March 24, 2009Complexity in everyday life5 Tracing Computing Backwards Turing (1936): Universal Computer (Model) Turing (1936): Universal Computer (Model) Gödel (1931): Logical predecessor. Gödel (1931): Logical predecessor. Hilbert (1900): Motivating questions/program. Hilbert (1900): Motivating questions/program. Gauss (1801): Efficient factoring of integers? Gauss (1801): Efficient factoring of integers? Euclid (-300): Computation of common divisors! Euclid (-300): Computation of common divisors! Prehistoric!! (adding, subtracting, multiplying, thinking (at least logically) are all computing!) Prehistoric!! (adding, subtracting, multiplying, thinking (at least logically) are all computing!) “Any effectively generated theory capable of expressing elementary arithmetic cannot be both consistent and complete.”complete “Entscheidung der L � sbarkeit einer diophantischen Gleichung. Eine diophantische Gleichung mit irgendwelchen Unbekannten und mit ganzen rationalen Zahlkoefficienten sei vorgelegt: man soll ein Verfahren angeben, nach welchen sich mittels einer endlichen Anzahl von Operationen entscheiden l � sst, ob die Gleichung in ganzen rationalen Zahlen l � sbar ist.”

6. March 24, 2009Complexity in everyday life6 Tracing Computing Forwards “Rumors of its demise are greatly exaggerated …” … More later.

7. March 24, 2009Complexity in everyday life7 Computation and Complexity

8. March 24, 2009Complexity in everyday life8 Addition: Suppose you want to add two ten-digit numbers. Does this take about 10 steps? Or about 10 x 10 steps? Addition: Suppose you want to add two ten-digit numbers. Does this take about 10 steps? Or about 10 x 10 steps? ~10 steps! Linear time! ~10 steps! Linear time! Example: Integer Addition 2 3 1 5 6 7 5 6 8 9 + 5 8 9 1 4 3 2 2 6 5 1 2 9 9 0 4 8 1 8 1 1 1 1 1

9. March 24, 2009Complexity in everyday life9 Computation! What we saw was a computational procedure (algorithm) to add integers. What we saw was a computational procedure (algorithm) to add integers. In general Algorithm = In general Algorithm = Sequence of steps Sequence of steps Each step very simple (finite + local) Each step very simple (finite + local) Every step of sequence determined by previous steps. Every step of sequence determined by previous steps. Formalization: Formalization: Turing Machine/Computer Program/Computer! Turing Machine/Computer Program/Computer! Moral: Computation is ancient! Eternal!! Moral: Computation is ancient! Eternal!!

10. March 24, 2009Complexity in everyday life10 First Law of Computation [Turing] Universality: There is a single computer which can execute every algorithm. Universality: There is a single computer which can execute every algorithm. Obvious today Obvious today … we all own such a “single computer”. Highly counterintuitive at the time of Turing. Highly counterintuitive at the time of Turing. Idea made practical by von Neumann. Idea made practical by von Neumann.

11. March 24, 2009Complexity in everyday life11 Multiplication: Suppose you want to multiply two n-digit numbers. Does this take about n steps? Or n x n steps? Multiplication: Suppose you want to multiply two n-digit numbers. Does this take about n steps? Or n x n steps? Above process: n 2 steps. Best? Above process: n 2 steps. Best? Example 2: Multiplication 2 3 1 5 6 7 2 0 8 4 1 0 3 1 3 6 4 2 5 3 8 2 3 8 2 3 1 5 6 7 x 5 8 9 1 4 9 2 6 2 6 8 1 8 5 2 5 3 6 1 1 5 7 8 3 5

12. March 24, 2009Complexity in everyday life12 Complexity Adding/Multiplying n-digit numbers Adding/Multiplying n-digit numbers Addition: ~n steps; Multiplication: ~n 2 steps. Addition: ~n steps; Multiplication: ~n 2 steps. Is addition really easier than multiplication? Is addition really easier than multiplication? Can we prove multiplying requires n 2 steps ? ( Needed to assert addition is easier !) Can we prove multiplying requires n 2 steps ? ( Needed to assert addition is easier !) Unfortunately, NO! Unfortunately, NO! Why? Why? Answer 1: Proving “every algorithm must be slow” is hard! Answer 1: Proving “every algorithm must be slow” is hard! Answer 2: Statement is incorrect! Answer 2: Statement is incorrect! Better algorithms (running in nearly linear time) exist! Better algorithms (running in nearly linear time) exist!

13. March 24, 2009Complexity in everyday life13 Computation and Complexity Broad goal of Computational Research: Broad goal of Computational Research: For each computational task For each computational task Find best algorithms [Algorithm Design] Find best algorithms [Algorithm Design] Prove they are best possible [Complexity] Prove they are best possible [Complexity] Challenges to the field: Challenges to the field: Algorithms: Can be ingenious Algorithms: Can be ingenious (in fact they model ingenuity!) (in fact they model ingenuity!) Complexity: Elusive, Misleading Complexity: Elusive, Misleading

14. March 24, 2009Complexity in everyday life14 Addition: Linear! Addition: Linear! Multiplication: Quadratic! Fastest? Not-linear Multiplication: Quadratic! Fastest? Not-linear Factoring? Write 13642538238 as product of two integers (each less than 1000000) Factoring? Write 13642538238 as product of two integers (each less than 1000000) Inverse of multiplication. Inverse of multiplication. Not known to be linear/quadratic/cubic. Not known to be linear/quadratic/cubic. Believed to require exponential time. Believed to require exponential time. Example: Integer Arithmetic

15. March 24, 2009Complexity in everyday life15 Computation and Complexity Broad classification of Computational Problems Broad classification of Computational Problems Easy Easy Those that can be solved in polynomial amount of time (n, n 2, n 10, …) Those that can be solved in polynomial amount of time (n, n 2, n 10, …) Hard Hard Those that require exponential time (2 n,10 n,…) Those that require exponential time (2 n,10 n,…) Doubling of resources increases size of largest feasible problem by multiplicative factor. Doubling of resources increases size of largest feasible problem by additive factor.

16. March 24, 2009Complexity in everyday life16 Computation and Complexity Broad classification of problems Broad classification of problems Easy: Doubling of resources increases size of largest feasible problem by multiplicative factor. Easy: Doubling of resources increases size of largest feasible problem by multiplicative factor. Hard: Doubling of resources increases size of largest feasible problem by additive factor. Hard: Doubling of resources increases size of largest feasible problem by additive factor. Computer Science Computer Science = (Mathematical) Study of Easiness. = (Mathematical) Study of Easiness. = (Mathematical) Study of Complexity. = (Mathematical) Study of Complexity.

17. March 24, 2009Complexity in everyday life17 Reversibility of Computation? Recall:Multiplication vs. Factoring Recall:Multiplication vs. Factoring Factoring reverses Multiplication Factoring reverses Multiplication Multiplication Easy Multiplication Easy Factoring seems Hard Factoring seems Hard P = Class of Easy Computational Problems. P = Class of Easy Computational Problems. Problem given by function f: input → output. Problem given by function f: input → output. NP = Reverses of P problems. NP = Reverses of P problems. Given function f in P, and output, give (any) input such that f(input) = output. Given function f in P, and output, give (any) input such that f(input) = output. Open: Is P=NP? Open: Is P=NP?

18. March 24, 2009Complexity in everyday life18 Second Law of Computation? [ Unproven ] Irreversibility Conjecture: Computation can not be easily reversed. (P ≠ NP). Irreversibility Conjecture: Computation can not be easily reversed. (P ≠ NP). The famed “P = NP?” question The famed “P = NP?” question Financially Interesting: Financially Interesting: Clay Institute offers US$ 1.000.000. Clay Institute offers US$ 1.000.000. Mathematically interesting: Mathematically interesting: Models essence of theorems and proofs. Models essence of theorems and proofs. Computationally interesting: Computationally interesting: Captures essential bottlenecks in computing. Captures essential bottlenecks in computing. Interesting to all: Interesting to all: Difference between goals and path to goals. Difference between goals and path to goals.

19. March 24, 2009Complexity in everyday life19 NP-completeness and consequences

20. March 24, 2009Complexity in everyday life20 Hardest problem in NP Even though we don’t know if NP = P, we know which problems in NP may be the hardest. E.g., Even though we don’t know if NP = P, we know which problems in NP may be the hardest. E.g., Travelling Salesman Problem Travelling Salesman Problem Integer Programming Integer Programming Finding proofs of theorems Finding proofs of theorems Folding protien sequences optimally Folding protien sequences optimally Computing optimal market strategies Computing optimal market strategies These problems are NP-complete. These problems are NP-complete. If any one can be easily solved, then all can be easily solved. If any one can be easily solved, then all can be easily solved.

21. March 24, 2009Complexity in everyday life21 An NP-complete Problem: Divisor? Given n-digit numbers A, B, C, does A have a divisor between B and C? Given n-digit numbers A, B, C, does A have a divisor between B and C? (Does there exist D such that B < D < C and D divides A?) (Does there exist D such that B < D < C and D divides A?) Theorem: Divisor is NP-complete. Theorem: Divisor is NP-complete. Equivalent view of NP: YES/NO problems where, if answer is YES, then it is easy to present short “proof”. Equivalent view of NP: YES/NO problems where, if answer is YES, then it is easy to present short “proof”. In Divisor, proof is the number D. In Divisor, proof is the number D. Example: Q: Does 3190966795047991905432 Q: Does 3190966795047991905432 have a divisor between have a divisor between 25800000000 and 25900000000. 25800000000 and 25900000000. A: YES A: YES Proof: 25846840632. Proof: 25846840632.

22. March 24, 2009Complexity in everyday life22 2.3 6.4 122.4 9.8 Example 2: Travelling Salesman Problem 5.1 17.3 3.2 #Hours so far Many cities; Want to visit all and return home; Can he do it with < 125 hours of driving? Easy to verify if answer is YES. Can you prove if answer is NO?

23. March 24, 2009Complexity in everyday life23 Theorems and Proofs 1900-2000: Mathematical formalization of Logic 1900-2000: Mathematical formalization of Logic [Hilbert, Gödel, Church, Turing …] [Hilbert, Gödel, Church, Turing …] Logic = Axioms + Deduction Rules Logic = Axioms + Deduction Rules Theorem, Proofs: Sentences over some alphabet. Theorem, Proofs: Sentences over some alphabet. Theorem: Valid if it follows from axioms and deduction rules. Theorem: Valid if it follows from axioms and deduction rules. Proof: Specifies axioms used and order of application of deduction rules. Proof: Specifies axioms used and order of application of deduction rules. Computational abstraction: Computational abstraction: (Theorem,Proof) easy to verify. (Theorem,Proof) easy to verify. Finding a proof for proposed theorem is hard. Finding a proof for proposed theorem is hard. Theorem: Finding short proofs is NP-complete. Theorem: Finding short proofs is NP-complete.

24. March 24, 2009Complexity in everyday life24 Theorems: Deep and Shallow A Deep Theorem: A Deep Theorem: Proof: (too long to fit in this section). Proof: (too long to fit in this section). A Shallow Theorem: A Shallow Theorem: The number 3190966795047991905432 has a divisor between 25800000000 and 25900000000. The number 3190966795047991905432 has a divisor between 25800000000 and 25900000000. Proof: 25846840632. Proof: 25846840632. x n + y n 6 = z n 8 x ; y ; z 2 Z + ; n ¸ 3

25. March 24, 2009Complexity in everyday life25 NP-Completeness & Logic Theory of NP-completeness: Theory of NP-completeness: Every (deep) theorem reduces to shallow one. Every (deep) theorem reduces to shallow one. Shallow theorem easy to compute from deep one. Shallow theorem easy to compute from deep one. Shallow proofs are not much longer. Shallow proofs are not much longer. Every NP-complete problem = “format” for proofs. Every NP-complete problem = “format” for proofs.

26. March 24, 2009Complexity in everyday life26 Is P=NP? Don’t know … Don’t know … If P=NP … If P=NP … Cryptography might well be impossible (current systems all broken simultaneously) Cryptography might well be impossible (current systems all broken simultaneously) All algorithmic problems become “easy” All algorithmic problems become “easy” … You get whatever you wish … if you just wish for it. … You get whatever you wish … if you just wish for it. Mathematicians replaced by computers. Mathematicians replaced by computers. If P≠NP … If P≠NP … … Consistent with current thinking, so no radical changes. … Consistent with current thinking, so no radical changes. Proof would be very interesting. Proof would be very interesting. Might provide sound cryptosystems. Might provide sound cryptosystems. Independent of Peano’s axioms, Choice …? Independent of Peano’s axioms, Choice …? “Of all the Clay Problems, this might be the one to find the shortest solution, by an amateur mathematician.” Devlin, The Millenium Problems (Possibly thinking of the case P=NP) “If someone shows P=NP, then they prove any theorem they wish. So they would walk away not just with $1M, but $6M by solving all the Clay Problems!” Lance Fortnow, Complexity Blog P = NP? is Mathematics-Complete

27. March 24, 2009Complexity in everyday life27 Probabilistic Verification of Proofs NP-completeness implies many surprising effects for logic. NP-completeness implies many surprising effects for logic. Examples: Examples: Proofs can be verified interactively much more quickly than in “published format”! Proofs can be verified interactively much more quickly than in “published format”! Proofs may reveal knowledge selectively! Proofs may reveal knowledge selectively! Proofs need not be fully read to verify them! Proofs need not be fully read to verify them! “Deep theorems” of computational complexity. “Deep theorems” of computational complexity.

28. March 24, 2009Complexity in everyday life28 Computation and You?

29. March 24, 2009Complexity in everyday life29 Computation beyond Computers Computation is not just about computers: Computation is not just about computers: It models all systematic processing … It models all systematic processing … Adding/Subtracting Adding/Subtracting Logical Deduction Logical Deduction Reasoning Reasoning Thought Thought Learning Learning Cooking (“Recipes = Algorithms”) Cooking (“Recipes = Algorithms”) Shampoo’ing your hair. Shampoo’ing your hair. Design, Engineering, Scientific … Design, Engineering, Scientific …

30. March 24, 2009Complexity in everyday life30 Biological organisms compute Folded structure of protiens determines their action. Folded structure of protiens determines their action. Common early belief: Protiens fold so as minimize their energy. Common early belief: Protiens fold so as minimize their energy. However … However … Minimum Energy configuration hard to compute (NP-complete). Minimum Energy configuration hard to compute (NP-complete). Implication: Implication: Perhaps achievable configurations are not global minima. Perhaps achievable configurations are not global minima.

31. March 24, 2009Complexity in everyday life31 NP-Completeness and Economics Economic belief: Economic belief: Individuals act rationally, optimizing their own profit, assuming rational behavior on other’s part. Individuals act rationally, optimizing their own profit, assuming rational behavior on other’s part. However … However … Optimal behavior is often hard to compute (NP-complete) Optimal behavior is often hard to compute (NP-complete) In such cases irrational (or bounded rationality) is best possible. In such cases irrational (or bounded rationality) is best possible. Alters behavior of market. Alters behavior of market.

32. March 24, 2009Complexity in everyday life32 NP-Completeness and the Brain Axiom: Brain is a computer Axiom: Brain is a computer ( Follows from Universality). Implications to Neuroscience: Implications to Neuroscience: What is the model of computing (neural network, other?) What is the model of computing (neural network, other?) More significantly … to Education: More significantly … to Education: Education = Programming of the brain Education = Programming of the brain (without losing creativity) (without losing creativity) What algorithms to “teach” What algorithms to “teach” Why multiplication? What is the point of “rote”? Why multiplication? What is the point of “rote”? Do resources matter? How much? Do resources matter? How much? How much complexity can a child’s brain handle? How much complexity can a child’s brain handle?

33. March 24, 2009Complexity in everyday life33 NP-Completeness and the Law Goals of Society: Goals of Society: Encourage creativity among individuals. Encourage creativity among individuals. But ensure “fair participation” by all. But ensure “fair participation” by all. Choice for “Legal system” Choice for “Legal system” How much of individual actions to prescribe? How much of individual actions to prescribe? If we don’t prescribe everything, don’t we eventually allow undesirable outcomes? If we don’t prescribe everything, don’t we eventually allow undesirable outcomes? Is “Law” contradictory to creativity? Is “Law” contradictory to creativity? NP ≠ P ⇒ Creativity feasible, provide law forbids undesirable end effects. NP ≠ P ⇒ Creativity feasible, provide law forbids undesirable end effects. ⇒

34. March 24, 2009Complexity in everyday life34 NP-Completeness and Life Life = Choices Life = Choices Which school should I go to? Which school should I go to? What subjects should I learn? What subjects should I learn? How should I spend my spare time? How should I spend my spare time? Which job should I take? Which job should I take? Should I insult my boss today? Or tomorrow? Should I insult my boss today? Or tomorrow? Sequence of simple steps that add up … Sequence of simple steps that add up … Eventually we find out if we did the right thing! Eventually we find out if we did the right thing! Life = (Non-deterministic) computation. Life = (Non-deterministic) computation. P = NP? Humans don’t need creativity/choice P = NP? Humans don’t need creativity/choice + Consequences,

35. March 24, 2009Complexity in everyday life35 Computation and You Eventually … humans are characterized by their intelligence. Eventually … humans are characterized by their intelligence. Intelligence is a “computational effect”. Intelligence is a “computational effect”. Inevitably “computation” is the “intellectual core of humanity”. Inevitably “computation” is the “intellectual core of humanity”. Shouldn’t be surprised if it affects all of us. Shouldn’t be surprised if it affects all of us.

36. March 24, 2009Complexity in everyday life36 Future of Computing

37. March 24, 2009Complexity in everyday life37 Tracing Computing Forwards “Rumors of its demise are greatly exaggerated …” Computing thus far … Computing thus far … First Law: Universality First Law: Universality Second Law(?): Irreversibility. Second Law(?): Irreversibility. Just the beginning … Just the beginning … … of Micro-Computer Science (one computer manipulating information). … of Micro-Computer Science (one computer manipulating information).

38. March 24, 2009Complexity in everyday life38 Future = Macro-Computer Science: The vast unknown What happens when many computers interact? What happens when many computers interact? What determines long term behavior? What determines long term behavior? What describes long term behavior? What describes long term behavior? What capabilities do we have (as intelligent beings, society) to control and alter this long term behavior? What capabilities do we have (as intelligent beings, society) to control and alter this long term behavior? How do computers evolve? How do computers evolve? Questions relevant already: Internet, WWW etc. Questions relevant already: Internet, WWW etc. What scientific quests are most similar? What scientific quests are most similar? (Statistical) Physics? Biology? Chemistry (big reactions)? (Statistical) Physics? Biology? Chemistry (big reactions)? Sociology? Logic? Sociology? Logic? Mathematics? Mathematics? Computation = Mathematics of the 21 st Century. Computation = Mathematics of the 21 st Century.

39. March 24, 2009Complexity in everyday life39 This talk is inspired by (and borrows freely from) … This talk is inspired by (and borrows freely from) … Christos Papadimitriou: The Algorithmic Lens Christos Papadimitriou: The Algorithmic Lens http://lazowska.cs.washington.edu/fcrc/Christos.FCRC.pdf http://lazowska.cs.washington.edu/fcrc/Christos.FCRC.pdf Avi Wigderson: A world view through the computational lens Avi Wigderson: A world view through the computational lens http://www.math.ias.edu/~avi/TALKS/ http://www.math.ias.edu/~avi/TALKS/ http://www.math.ias.edu/~avi/TALKS/ Many colleagues: esp. Oded Goldreich, Salil Vadhan Many colleagues: esp. Oded Goldreich, Salil Vadhan Acknowledgments (+ Pointers)

40. March 24, 2009Complexity in everyday life40 Thank You!