1. 1 CS120: Lecture 9 MP Johnson Hunter mpjohnson@gmail.com

2. 2 Agenda: Algs Next topic: algs –Hw2 due Wed –MT Thursday –No unexcused absences Next after that: JavaScript –Next week’s lab: JS & advanced HTML For Tues: read handout on stable marriage Defs Complexity ftns Probs: –Add nums –Mult nums –Gcd –Sorting –Binary search –Stable marriage –Prefix-sum –Poly eval –Towers of Hanoi Order-of-magn calcs

3. 3 Hist, motiv Def: ordered set of precise instructions, executable steps that defines a finite process You write programs, which implement some alg Etym: Persian math. Al-Khowarizmi  Arabic al-khuwarizmi  Medieval Latin algorismus  Middle English algorisme  algorithm Also: title of book  algebra

4. 4 def 1.Alg: idea behind program –seq of steps for solving some prob finite well defined each step tractable 2.(step 3 can’t be “compute Pi exactly” or “find meaning of life”) 3.alg is a mapping: –A:{instances}  {outputs} –outputs could be bits/decisions: prime or not? or larger structure: position of number; sorted seq 4.intuition: what stays the same when prog is ported from C to J to Basic to Assembly

5. 5 To be an alg Each step must be unambiguous –Can be converted to machine instructions Each step must be doable –Can’t say “calc Pi exactly” or “print meaning of life” Must be finite –Description is finite –For each poss. input, must eventually stop

6. 6 Alg goals 1.ideally, alg should be: 1.Correct 2.efficient –or at least optimal 2.each can be difficult 3.sometimes: trade-off between difficulty of each –easy-to-understand alg may be slow – fast alg may be difficult to understand (or to find!) 4.not really surprising: “obvious” soln often very slow

7. 7 To be a good alg Also: want algorithm to be fast –In a precise sense… Should scale well on different inputs Shouldn’t take too much storage space Should be easy to understand/code

8. 8 Alg v. program A program implements an alg –Program v. alg v. process Book analogy: –Story:book :: alg:program –Story can be translated between langs, printed in hardcover of paperback –Alg can be translated between langs, run on Win or Mac –The alg is what stays the same when it’s ported

9. 9 Algs you already know 1.Dial your phone number 2.Combination lock 3.Cooking 4.Add numbers in memory: Fixed-length alg 5.Addition itself (of bitstrings) Length depends on bitstring size

10. 10 Algs you already know 6.Long additon, etc. (base-10 ops) 1.Let rightmost column be current 2.Add the digits in the current col 3.Write first dig of result as answer for that col 4.If result >9, add 1 to previous digits (“carry”) 5.Move “current column” to left 6.Go to step 2 7.Processor’s alg: –Fetch an instruction –Decode it –Execute it –Go to step 1

11. 11 Problem-solving Programming can be hard (fixing bugs) But deciding what to program (finding the alg) can often be harder Given problem description, must find some (good) alg No general-purpose method that always works Trial&error, experience Would like: an alg to produce alg, but there isn’t one

12. 12 TSP: Hilary motiv 1.algs are designed to solve problems, in all cases –should work correctly (quickly) on all possible inputs 2.eg: say Hilary has to campaign in N cities –maybe all state caps –maybe a subset 3.Hilary needs to save gas –no Air Force One –needs optimal route to: visit each stop, return to starting place Washington? Boston? 4.Hilary can look at the map –use geometric reasoning –intuition based on experience –very sophisticated –hard – see an ML class 5.Hilary’s intuition may work for small cases –for larger ones, very likely will be highly suboptimal 6.we want an alg: guaranteed to always return a best route –may not be unique

13. 13 TSP: Hilary motiv 1.Where does she start? NYC 2. Now what? Obvious choice: jump to closest city Say, Bos 3. Now what? Again, jump to closest (unvisited) city Thus ag: 4. Pick arbitrary p0; Set I = 1; While (I < n) Set pi = closest new city to p(i-1)

14. 14 TSP: Greedy alg 1.We’ve stumbled onto the “greedy algorithm” for this prob –many other probs have “Greedy algs”: alg does maximal optimization at each step –an. To short-term self interest v. long-term self-interest 2.this greedy alg has much to recommend it: –very easy to understand –very easy to code up –pretty efficient for each next move, just choose among those remaining 3.only problem: it’s wrong –i.e., the resulting path won’t always be the shortest one possible

15. 15 TSP: Greedy alg 1.consider this case:(sequence of dots) 2.greedy produces this 3.clearly optimal is that eg: –NYC  Bos 215 –NYC  DC 232 4. prob not just all in a line –Prob also occurs for T-shaped –Anytime something like this as a subgraph –just one example 5. what happened? Too short-sighted –cities that were next to each other were separated on the tour –should have been kept together

16. 16 TSP: Joining pairs 1.idea: join together near pairs, then do greedy on results –maybe this will work? 2.Apply this strategy to line of dots produces optimal 3.What about to this eg: Grid of dots –wider than tall –suboptimal again! 4.Clearly better tours exist: … –This alg didn’t work either Intuition: writing algorithms = raising children

17. 17 TSP: Brute force 1.So what does work? –Let’s slow down –Recall: wanted best ordering –How many orderings? Only so many 2.Following must work: run through all possible, measure each one 3.Code: Best = -1 Cost = infinity I = 0 While (I < n) If (best == -1 or cost[i] < cost[best]) Best = I; Return best;

18. 18 TSP: Brute force 1.Correct? –yes, must be 2.efficient? –NO, very inefficient –number of steps roughly n! –very bad: 5! = 120, 10! = 3,628,800, 20! = 10^18, 33! = 8^36 3.Q: can we do better? –Well, somewhat—basically clever software engineering, using info from special cases, clever coding 4.But no essentially faster alg –There’s no fast alg (so far as anyone knows!) –No proof of its nonexistence, but would be a huge surprise –In other cases, we’ll have better luck

19. 19 Measuring running time Logn N N^2 N^k 2^n N! O(f(n)) – “big O”, Th(f(n)) – theta Big-O matters, coeffs don’t Nn^22^n 5 10 20

20. Log review df: logbX = y  b^y = x –intuit: Y is number of b’s multiplied together yield X –i.e., log is inverse of exponentiation function Log2(x)= y  2^y = x 2^10 = 1024 = klog1024 2^8 = 256… 2^16 = 64k 2^20 = M 2^30 = G

21. 21 search Sequential search –Alg? –Cmplx? Binary search –Alg? –Analogy: phonebook –Cmplx? Q: how many times to /2? Nlogn 1024 …

22. 22 Order of magnitude calculations?