1. Roadmap of the lecture Computational complexity –polynomial vs. exponential algorithms –non-deterministic computing –N(ondetermiistic) P(olynomial) problems Example: the Traveling Salesman Problem (TSP) Some non-TM (?) models of computing –Social insects –(Genetic algorithms and “evolutionary computing”) –DNA –Quantum computing –Neural nets

2. Some types of problems Ill-defined problems Undecidable problems e.g. will an arbitrary Turing Machine program ever halt? Decidable problems (classified by complexity) –polynomial –exponential –non-deterministic polynomial

3. n=10n=20n=30n=40n=50n=60 n.00001 second.00002 second.00003 second.00004 second.00005 second.00006 second n2n2.0001 second.0004 second.0009 second.0016 second.0025 second.0036 second n3n3.001 second.008 second.027 second.064 second.125 second.216 second n5n5.1 second 3.2 seconds 24.3 seconds 1.7 minutes 5.2 minutes 13.0 minutes 2n2n.001 second 1.0 second 17.9 minutes 12.7 days 35.7 years 336 centuries 3n3n.059 second 58 minutes 6.5 years 3855 centuries 2x10 8 centuries 1.3x10 13 centuries Polynomial vs. exponential time complexity (one algorithm step = 1 microsecond) (Figure 1.2 from Garvey & Johnson 1979)

4. Time complexity function With Present computer With Computer 100 times faster With computer 1000 times faster nN1N1 100N 1 1000N 1 n2n2 N2N2 10 N 2 31.6 N 2 n3n3 N3N3 4.64 N 3 10 N 3 n5n5 N4N4 2.5 N 4 3.98 N 4 2n2n N5N5 N 5 +6.64N 5 +9.97 3n3n N6N6 N 6 +4.19N 6 +6.29 (Figure 1.3 from Garvey & Johnson 1979) The effect of technological improvements: size of the largest problem solvable in one hour...

5. Time vs. task size in human cognitive processing: The Sternberg Item Memory test

6. Non-deterministic algorithms Non-determinism in TMs and similar automata –one next state... or more? –guessing and backtracking –“determinization”: eliminating non-determinism Complexity of non-deterministic algorithms –interesting class of problems: guessing the answer seems hard (exponential or worse) checking a guess is easy (polynomial) –called “non-deterministic polynomial” problems abbreviated “NP” open question: P = NP? i.e. is guessing really, truly hard for these problems?

7. Some key equivalence classes of problem complexity “NP complete” A decision problem is NP-complete if it is in NP and if every other problem in NP is “quickly” reducible to it Examples: traveling salesman, boolean satisfiability, knapsack, subgraph isomorphism,... “NP hard” A problem is NP-hard if every other problem in NP is “quickly” reducible to it. I.e. “NP complete or even harder than that”

8. Practical approaches to NP problems Approximation efficient solution within some bound Probabilistic efficient solution to most expected problem instances Special case efficient solution for a certain class of cases Heuristic a method that sometimes sort of works

9. The traveling salesman

10. On beyond Turing... Plenty of pre-Turing specialized computer designs Samples from the post-Turing bestiary: –Genetic algorithms / evolutionary computing –Social insects, markets etc. –DNA computing / biomolecular computing –Neural nets / connectionism / PDP –Quantum computing Themes: –Parallelism / interconnection –Gradient vs. discrete values –Chance

11. Collective Intelligence in Social Insects Self-organization was originally introduced in the context of physics and chemistry to describe how microscopic processes give rise to macroscopic structures in out-of- equilibrium systems. Recent research that extends this concept to ethology, suggests that it provides a concise description of a wide rage of collective phenomena in animals, especially in social insects. This description does not rely on individual complexity to account for complex spatiotemporal features which emerge at the colony level, but rather assumes that interactions among simple individuals can produce highly structured collective behaviors. E. Bonabeau et al., Self-Organization in Social Insects, 1997

12. Buridan’s Ants make a decision Percentage of Iridomyrex Humulis workers passing each (equal) arm of bridge per 3-minute period

13. Finding the shortest path: (from Dorigo et al. 1997)

14. More complex emergent structure: termite mounds…

15. Bruinsma (1979): positive feedback mechanisms, involving responses to a short- lived pheromone in deposited soil pellets, a long-lived pheromone along travel paths, and a general tendency to orient pellet deposition to spatial heterogeneities; these lead to the construction of pillars and roofed lamellae around the queen. Deneubourg (1977): a simple model with parameters for the random walk of the termites and the diffusion and attractivity of the pellet pheronome, producing a regular array of pillars. Bonabeau et al. (1997): air convection, pheromone trails along walkways, and pheromones emitted by the queen; "under certain conditions, pillars are transformed into walls or galleries or chambers", with different outcomes depending not on changes in behavioral dispositions but on environmental changes caused by previous building. Thus "nest complexity can result from the unfolding of a morphogenetic process that progressively generates a diversity of history-dependent structures." Similar to models of embryological morphogenesis. Termite Theory:

16. Why neural nets? Computers: –fast circuits (10 9 state changes per second) –small number of interconnected processors (1-100) –digital flow of information –determinate, exactly repeatable behavior –designed and replicated exactly –small flaw or damage is catastrophic –mostly programmed –good at arithmetic, logic; bad at pattern recognition, abduction Brains: –slow circuits (10 3 state changes per second) –large number of interconnected processors (10 10 ) –quasi analog flow of information –stochastic behavior –grow organically with lots of case-to-case variation –small flaw or damage usually irrelevant –some “programming” (by evolution), some learning (from individual experience) –bad at arithmetic, logic; good at pattern recognition, abduction The appeal of “neural net” models: a family of mathematical abstraction that share many of the differential properties of brains vs. computers