How to Think about Algorithms Recursion Code Stack of Stack Frames Tree of Stack Frames Friends and Strong Induction Towers of Hanoi Check List Merge & Quick Sort Simple Recursion on Trees Generalizing the Problem Things not to do Heap Sort & Priority Queues Trees Representing Equations Pretty Print Parsing Recursive Images

Divide And Conquer (an approach to faster algorithms) DIVIDE my instance to the problem into smaller instances to the same problem. Have a friend (recursively) solve them. Do not worry about it yourself. GLUE the answers together so as to obtain the answer to your larger instance.

Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time.

Code Representation of an Algorithm MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Pros and Cons?

Code Representation of an Algorithm Pros: Cons: Runs on computers Precise and succinct Perception that being able to code is the only thing needed to get a job. (false) I am not a computer I need a higher level of intuition. Prone to bugs Language dependent

Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time.

Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = bd = (a+b)(c+d) = XY = Stack Frame: A particular execution of one routine on one particular input instance.

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = bd = (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY=

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY = 4

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = X = 1 Y = 3 XY = 3

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = X = 3 Y = 5 XY = 15

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = ?

Stack of Stack Frames X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483

Stack of Stack Frames X = 2133 Y = 2312 ac = 483 bd = (a+b)(c+d) = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = X = 3 Y = 1 XY = 3

Stack of Stack Frames X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY =

Stack of Stack Frames X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = X = 3 Y = 2 XY = 6

Stack of Stack Frames An so on …. X = 2133 Y = 2312 ac = 483 bd = ? MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = ? XY = An so on ….

Stack of Stack Frames Representation of an Algorithm Pros: Cons: Trace what actually occurs in the computer Concrete. It is what students attempt to describe when told not to give code. Described in words it is impossible to follow who is doing what. Does not explain why it works. Demonstrates for only one of many inputs.

Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time.

Tree of Stack Frames X = 2133 Y = 2312 ac = 483 bd = 396 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72

Stack of Stack Frames Representation of an Algorithm Pros: Cons: View the entire computation. Good for computing the running time. Must describe entire tree. For each stack frame input instance computation solution returned. who calls who Structure of tree may be complex.

Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time.

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f One Friend for each stack frame. Each worries only about his job. X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Worry about one step at a time. Imagine that you are one specific friend. X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Consider your input instance X = 33 Y = 12 ac = bd = (a+b)(c+d) = XY =

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Consider your input instance Allocate work Construct one or more sub-instances X = 33 Y = 12 ac = ? bd = ? (a+b)(c+d) = XY = X = 3 Y = 1 XY = ? X = 3 Y = 2 XY = ? X = 6 Y = 3 XY = ?

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Consider your input instance Allocate work Construct one or more sub-instances Assume by magic your friends give you the answer for these. X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = X = 3 Y = 1 XY = 3 X = 3 Y = 2 XY = 6 X = 6 Y = 3 XY = 18

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Consider your input instance Allocate work Construct one or more sub-instances Assume by magic your friends give you the answer for these. Use this help to solve your own instance. X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 3 Y = 1 XY = 3 X = 3 Y = 2 XY = 6 X = 6 Y = 3 XY = 18

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Consider your input instance Allocate work Construct one or more sub-instances Assume by magic your friends give you the answer for these. Use this help to solve your own instance. Do not worry about anything else. Who your boss is… How your friends solve their instance… X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 3 Y = 1 XY = 3 X = 3 Y = 2 XY = 6 X = 6 Y = 3 XY = 18

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f This technique is often referred to as Divide and Conquer X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 3 Y = 1 XY = 3 X = 3 Y = 2 XY = 6 X = 6 Y = 3 XY = 18

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Consider generic instances. MULT(X,Y): Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Remember! Do not worry about Who your boss is. How your friends solve their instance. ac bd (a+b)(c+d)

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f This solves the problem for every possible instance. X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72

Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works.

Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. If I could get in, I could get the key. Then I could unlock the door so that I can get in. Circular Argument!

Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. To get into my house I must get the key from a smaller house

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Allocate work Construct one or more sub-instances X = 33 Y = 12 ac = ? bd = ? (a+b)(c+d) = XY = Each sub-instance must be a smaller instance to the same problem. X = 3 Y = 1 XY = ? X = 3 Y = 2 XY = ? X = 6 Y = 3 XY = ?

Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. Use brute force to get into the smallest house.

Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Use brute force to solve the base case instances.

Friends & Strong Induction Carefully write the specifications for the problem. <preCond>: Set of legal instances (inputs) Why? <postCond>: Required output 78yhy77uuuuuuuuuuu7

Friends & Strong Induction Carefully write the specifications for the problem. <preCond>: Set of legal instances (inputs) To be sure that we solve the problem for every legal instance. So that we know what we can give to a friend. <postCond>: Required output what is expected of us. what we can expect from our friend. 78yhy77uuuuuuuuuuu7 Related to Loop Invariants

Induction Strong Induction

Friends & Strong Induction Induction Hypothesis: ``The recursive algorithm works for every instance of size n.'' Base case ``The algorithm works for every instance of any size.'' size i

Friends & Strong Induction Representation of an Algorithm Pros: Cons: Worry about one step at a time. An abstraction within which to develop, think about, and describe algorithms in such way that their correctness is transparent. You expect too much from your friends. Each sub-instance must be a smaller instance to the same problem. You expect too much from your boss. You only know your instance. You do not know your boss’s instance.

Try explaining what this algorithm does by tracing it out. Output Try explaining what this algorithm does by tracing it out. It is much easier to TAKE ONE STEP AT A TIME

Output n=1 X n=2 Y

Output n=1 X n=2 Y n=3 A ? B ? C

Output n=1 X n=2 Y n=3 A Y B X C

Output n=1 X n=2 Y n=3 AYBXC

Output n=1 X n=2 Y n=3 AYBXC n=4 A ? B ? C

Output n=1 X n=2 Y n=3 AYBXC n=4 A AYBXC B Y C

Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC

Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 A ? B ? C

Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 AAAYBXCBYCBAYBXCC

Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 AAAYBXCBYCBAYBXCC

Time: T(1) = T(2) = 1 1 T(n) = T(n-1) + T(n-2) + 3  2T(n-1) +3 = 2Q(n) defined?

This job of mine is a bit daunting. Towers of Hanoi This job of mine is a bit daunting. Where do I start? And I am lazy.

At some point, the biggest disk moves. Towers of Hanoi At some point, the biggest disk moves. I will do that job.

To do this, the other disks must be in the middle. Towers of Hanoi To do this, the other disks must be in the middle.

Towers of Hanoi How will these move? I will get a friend to do it. And another to move these. I only move the big disk.

Towers of Hanoi

Towers of Hanoi Get a job at the Towers of Hanoi company at level n=1,2,3,4,….. Or get transferred in as the president & you decide what your vice president does.

Towers of Hanoi Time: T(1) = 1, T(n) = ≈ 2(2T(n-2)) ≈ 4(2T(n-3)) ≈ 2i T(n-i) ≈ 2n

Evaluating: T(n) = aT(n-b)+f(n) 2 1 Level Instance size Work in stack frame # stack frames Work in Level n T(0) f(n-ib) f(n-2b) f(n-b) f(n) a ai a2 1 n/b a · T(0) ai · f(n-ib) a2 · f(n-2b) a · f(n-b) 1 · f(n) n/b n-b n-hb n-ib n-2b h = ? h = n/b Exponential Likely dominated by base cases |base case| = 0 = n-hb h = n/b

Evaluating: T(n) = 1T(n-b)+f(n) 2 1 Level Instance size Work in stack frame # stack frames Work in Level n-b n n-hb n-ib n-2b T(0) f(n-ib) f(n-2b) f(n-b) f(n) h = n/b 1 f(0) f(n-ib) f(n-2b) f(n-b) f(n) h = ? Total Work T(n) = ∑i=0..h f(b·i) = θ(f(n)) or θ(n·f(n))

Evaluating: T(n) = aT(n/b)+f(n)

Check Lists for Recursive Programs This is the format of “all” recursive programs. Don’t deviate from this. Or else!

Check Lists for Recursive Programs This is input instance is your mission. You must return this output. (An x, y, & z in every computation path.) Focus on your mission.

Check Lists for Recursive Programs The <preCond> & <postCond> must document these variables and what you must do.

Check Lists for Recursive Programs To get help, you construct an instance for each friend It must be valid and smaller. And pass them to them.

Check Lists for Recursive Programs Each friend returns a solution his sub-instance. Trust him to do this. But don’t expect him to do or know anything else.

Check Lists for Recursive Programs Combine their solutions to construct a solution to your instance. Return your solution.

Check Lists for Recursive Programs If your solution is too small to get solved by the general technique solve it yourself. Be sure you handle every legal instance.

Check Lists for Recursive Programs You rarely need to change the values of these variable beyond their initial setting (this is not an iterative algorithm.) Your rarely need additional input, output, or local variable. But it you do document them.

Check Lists for Recursive Programs Have NO global variables. If you change the value of a local variable n, neither your friends nor your boss get that value.

Sorting Problem Specification <preCond>: The input is a list of n values with the same value possibly repeated. <postCond>: The output is a list consisting of the same n values in non-decreasing order. 88 14 98 25 62 52 79 30 23 31 14,23,25,30,31,52,62,79,88,98

Recursive Sorts Given list of objects to be sorted Split the list into two sub-lists. Recursively have a friend sort the two sub-lists. Combine the two sorted sub-lists into one entirely sorted list.

Four Recursive Sorts Size of Sub-lists n-1,1 n/2,n/2 Merge Sort Insertion Sort Quick Sort Selection Sort Minimal effort splitting Lots of effort recombining Lots of effort splitting Minimal effort recombining

Merge Sort 88 14 98 25 62 52 79 30 23 31 Divide and Conquer

Merge Sort (no real work) Split Set into Two 88 14 98 25 62 52 79 30 23 31 (no real work) Split Set into Two 25,31,52,88,98 Get one friend to sort the first half. 14,23,30,62,79 Get one friend to sort the second half.

Merge Sort Merge two sorted lists into one 25,31,52,88,98 14,23,25,30,31,52,62,79,88,98 14,23,30,62,79

Total work at any level = Q(n) Merge Sort Sort Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Total work at any level = Q(n) # levels = Q(log(n))

Evaluating: T(n) = aT(n/b)+f(n) 2T(n/2) + Q(n) Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level n f(n) 1 1 · f(n) n/b f(n/b) a a · f(n/b) 2 n/b2 f(n/b2) a2 a2 · f(n/b2) i n/bi f(n/bi) ai ai · f(n/bi) h = log n/log b n/bh T(1) 2i·Q(n/2i) = Q(n) n log a/log b n · T(1) log a/log b All levels basically the same. Total Work T(n) = ∑i=0..h ai×f(n/bi) = Q(n) · log(n)

Quick Sort 88 14 98 25 62 52 79 30 23 31 Divide and Conquer

Quick Sort Partition set into two using randomly chosen pivot 88 14 98 25 62 52 79 30 23 31 14 25 30 23 31 88 98 62 79 ≤ 52 ≤

Quick Sort Get one friend to sort the first half. 14 25 30 23 31 88 98 62 79 ≤ 52 ≤ 14,23,25,30,31 Get one friend to sort the first half. 62,79,98,88 Get one friend to sort the second half.

Quick Sort Glue pieces together. (No real work) 14,23,25,30,31 52 62,79,98,88 Glue pieces together. (No real work) 14,23,25,30,31,52,62,79,88,98

Quick Sort Let pivot be the first element in the list? 88 14 98 25 62 52 79 30 23 31 14 88 98 30 ≤ 31 ≤ 62 23 25 52 79

Quick Sort 14,23,25,30,31,52,62,79,88,98 23,25,30,31,52,62,79,88,98 ≤ 14 ≤ If the list is already sorted, then the slit is worst case unbalanced.

Quick Sort T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Best Time: Worst Time: Expected Time:

Quick Sort T(n) = T(0) + T(n-1) + Q(n) Best Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Worst Time: = Q(n2) Expected Time:

Quick Sort Best Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Worst Time: T(n) = T(0) + T(n-1) + Q(n) = Q(n2) Expected Time: T(n) = T(1/3n) + T(2/3n) + Q(n) = Q(n log(n))

Quick Sort Expected Time: T(n) = T(1/3n) + T(2/3n) + Q(n) = Q(n log(n)) Top work Q(n) 2nd level Q(n) Q(n) 3rd level 1/3 2/3 3/2 log (n) # levels = = Q(log(n))

Kth Element Problem Specification <preCond>: The input is a list of n values and an integer k. <postCond>: The kth smallest in the list. k=3 88 14 98 25 62 52 79 30 23 31 Output: 25 14,23,25,30,31,52,62,79,88,98

Kth Element Problem Specification Partition set into two using randomly chosen pivot 88 14 98 25 62 52 79 30 23 31 14 25 30 23 31 88 98 62 79 ≤ 52 ≤

Kth Element Problem Specification 14 25 30 23 31 88 98 ≤ 52 ≤ 62 79 k=3 r=5 elements on left 14,23,25,30,31 Get one friend to find k=3rd element in first half. Output: 25

Kth Element Problem Specification 14 25 30 23 31 88 98 ≤ 52 ≤ 62 79 k=8 r=5 elements on left Get one friend to find k=8-5=3rd element in second half. Output: 79 52,62,79,98,88

Kth Element Problem Specification Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) n log a/log b # of base cases = Worst Time: n log 1/log 2 = = n0 = 1 n +n/2 +n/4 +n/8 … +1 Expected Time:

Kth Element Problem Specification Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) Worst Time: T(n) = 1 T(n-1) + Q(n) ≈ n+T(n-1) ≈ n+(n-1)+T(n-2) ≈ n+(n-1)+(n-2)+T(n-3) ≈ n+(n-1)+(n-2)+(n-3)+…+1 ≈ Q(n2) Expected Time:

Kth Element Problem Specification Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) Worst Time: T(n) = 1 T(n-1) + Q(n) = Q(n2) Expected Time: T(n) = 1 T(2/3n) + Q(n) = Q(n)

T(N) = 2T(N/2) + 1 = (N) Size = log(b) + log(N) = 2(n) b7 = b3 × b4 N=7 b4 b3 N=4 N=3 N=2 N=1

T(N) = 1T(N/2) + 1 = (log(N)) Size = log(b) + log(N) = (n) b7 = (b3)2 × b N=7 b3 N=3 N=1

Recursion on Trees 3 8 1 2 7 6 5 9 4

Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. tree because 3 8 1 2 7 6 5 9 4

Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. node tree 3 8 1 2 7 6 5 9 4 tree

Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 tree because

Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 node tree tree

Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 tree because

Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 node tree tree

Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 tree because empty

Recursion on Trees number of nodes = ? Get help from friends 5 6 3 8 1 2 7 6 5 9 4

Recursion on Trees number of nodes = number on left + number on right + 1 = 6 + 5 + 1 = 12 5 3 8 1 2 7 6 5 9 4 6

Recursion on Trees number of nodes Base Case ? 1 Are we done? 3 8 1 2 7 6 5 9 4 1 Are we done?

Recursion on Trees number of nodes Base Case ? 3 8 1 2 7 6 5 9 4 ?

Recursion on Trees number of nodes Base Case ? Better base case! 3 8 1 2 7 6 5 9 4 Better base case!

Recursion on Trees number of nodes Base Case ? 1 3 8 1 2 7 6 5 9 4 1 Does this still need to be a base case?

Recursion on Trees number of nodes = number on left + number on right + 1 = 0 + 0 + 1 = 1 3 8 1 2 7 6 5 9 4 No!

Recursion on Trees Most programmers don’t use the empty tree as a base case. This causes a lot more work. 3 8 1 2 7 6 5 9 4

Recursion on Trees

Recursion on Trees Designing Program/Test Cases Try same code n1 n2 n1 n1 + 1 3 3 generic generic generic Same code works! Try same code 0+0+1=1 3 Same code works!

Recursion on Trees Time: T(n) = T(left) + T(right) + Q(1) = Q(n) One stack frame for each node in the tree

Recursion on Trees sum of nodes = ? Get help from friends 25 23 3 8 1 7 6 5 9 4

Recursion on Trees sum of nodes = sum on left + sum on right + value at root = 23 + 25 + 3 = 51 25 3 8 1 2 7 6 5 9 4 23

Recursion on Trees sum of nodes = ? Base Case ? 3 8 1 2 7 6 5 9 4

Recursion on Trees

Recursion on Trees Designing Program/Test Cases s1 s2 s1 + s2 + 3 s1 s1 s1 + 3 3 3 generic generic generic 0+0+ 3 =3 3

Recursion on Trees max of nodes = ? Get help from friends 9 8 3 8 1 2 7 6 5 9 4

Recursion on Trees max of nodes = max( max on left, max on right, value at root) = max( 8, 9, 3) = 9 9 3 8 1 2 7 6 5 9 4 8

Recursion on Trees max of nodes = ? Base Case ? 3 8 1 2 7 6 5 9 4 ?

Recursion on Trees 3+4+2+8+… Sum so far is 17 Sum so far is NewSum = OldSum + nextElement = 0 + 3 = 3

Recursion on Trees 3*4*2*3*… Product so far is 72 Product so far is 1 NewProd = OldProd × nextElement = 1 × 3 = 3

Recursion on Trees Conclusion so far is True True and True and False and … Conclusion so far is True Conclusion so far is True NewProd = OldProd and nextElement = True and True = True

Recursion on Trees -¥ Max(3,4,2,3,… Max so far is 4 Max so far is NewMax = max(OldMax, next Element) = max( -¥ , 3 ) = 3

Recursion on Trees

Recursion on Trees height of tree = ? Get help from friends 4 3 3 8 1 2 7 6 5 9 4

Recursion on Trees height of tree = max(height of left,height on right) + 1 = max(3,4) + 1 = 5 4 3 8 1 2 7 6 5 9 4 3

Recursion on Trees height of tree Base Case ? 3 8 1 2 7 6 5 9 4 ?

Recursion on Trees height of tree? 2 nodes or 1 edge Base Case ? 0 nodes ? edges Base Case ? 3 8 1 2 7 6 5 9 4

Recursion on Trees Work it out backwards height = 0 edge = max(height of left,height on right) + 1 = max(?,?) + 1 ? 3 8 1 2 7 6 5 9 4 ? edges

Recursion on Trees Work it out backwards height = 0 edge = max(height of left,height on right) + 1 = max(-1,-1) + 1 3 8 1 2 7 6 5 9 4 -1 edges -1 -1

Recursion on Trees

Recursion on Trees number of leaves = ? Get help from friends 2 3 3 8 1 2 7 6 5 9 4

Recursion on Trees number of leaves = number on left + number on right = 3 + 2 = 5 2 3 8 1 2 7 6 5 9 4 3

Recursion on Trees number of leaves Base Case ? 3 8 1 2 7 6 5 9 4

Recursion on Trees Designing Program/Test Cases L1 L2 L1 + L2 L1 L1 3 3 generic generic generic 1 No! 0+0 = 0 3 Needs to be another base case.

Recursion on Trees

Recursion on Trees

Recursion on Trees

Recursion on Trees ? Drops the tree.

Define Problem: Binary Search Tree <preCond> Key 25 A binary search tree. <postCond> Find key in BST (if there). 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71

Its left children ≤ Any node ≤ Its right children Binary Search Tree Its left children ≤ Any node ≤ Its right children 38 ≤ 25 ≤ 51 17 31 42 63 4 21 28 35 40 49 55 71

Define Loop Invariant Maintain a sub-tree. If the key is contained in the original tree, then the key is contained in the sub-tree. 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71 key 17

Define Step Cut sub-tree in half. Determine which half the key would be in. Keep that half. 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71 key 17 If key < root, then key is in left half. If key = root, then key is found If key > root, then key is in right half.

Recursion on Trees

Recursion on Trees

Recursion on Trees Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n)) If tree very unbalanced? T(n) = 1T(n-1) + (n) = (n2) Computing max is too much work.

Recursion on Trees Extra info from below

Recursion on Trees 1 Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n)) n Computing max is too much work.

Recursion on Trees Extra info from above min   max min   max min  38 min   max min   root root   max -¥, ¥

Things to Remember Things to Do and Things NOT TO DO

Friends - Strong Induction View of Recursion. I am obsessed with the Friends - Strong Induction View of Recursion. Trust your friends to solve sub-instances. The sub-instance given must be smaller and must be an instance to the same problem. Combine solution given by friend to construct your own solution for your instance. Focus on one step. Do not talk of their friends friends friends. Solve small instances on your own. Do it!

Don't have inputs or outputs that are not explained! Typical Test Answer Define pre & post conditions Don't have inputs or outputs that are not explained!

Typical Test Answer Call recursively on the correct types of input k,num,v

returning the correct types of output Typical Test Answer Call recursively returning the correct types of output Save the results (or don't bother calling)

Combine solutions given by friends to construct your own solution. Typical Test Answer Combine solutions given by friends to construct your own solution.

Return things of the correct types. Typical Test Answer Return things of the correct types. Not an element the root? In every path through code

Sub-instances need to be smaller. Typical Test Answer Sub-instances need to be smaller. “Size” is size of the tree Wrong base case

What is the value of n here? Typical Test Answer No Global Variables. What is the value of n here? Still zero. The friend has his own variable n.

Maybe they mean to pass the new n in and out. Typical Test Answer No Global Variables. Maybe they mean to pass the new n in and out. Please don’t use these to pass out values when I am marking.

Looks like an iterative algorithm Typical Test Answer Looks like an iterative algorithm Looks like an recursive algorithm Which is it?

Hard to write an iterative program to traverse on a recursive structure tree 3 8 1 2 7 6 5 9 4 Need pointers from each node to its parent.

Hard to write a recursive program that implements an iterative algorithm.

Writing a Recursive Program Call recursively on the correct types of input

on sub-instances which will give useful information. Writing a Recursive Program Call recursively on sub-instances which will give useful information. ? node in sub-tree 3 8 1 2 7 6 5 9 4

on sub-instances which will give useful information. Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree node in whole tree ? 3 8 1 2 7 6 5 9 4

on sub-instances which will give useful information. Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree nl node in whole tree +1 +3rd 3 8 1 2 7 6 5 9 4

on sub-instances which will give useful information. Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree nl node in whole tree +1 +3rd 3 8 1 2 7 6 5 9 4

Writing a Recursive Program But who will count nodes in left sub-tree? 3 8 1 2 7 6 5 9 4 nl nl My friend

Writing a Recursive Program But who will count nodes in left sub-tree? 3 8 1 2 7 6 5 9 4 nl nl My friend

returning the correct types of output Writing a Recursive Program Call recursively returning the correct types of output Save the results (or don't bother calling)

Combine solutions given by friends to construct your own solution. Writing a Recursive Program Combine solutions given by friends to construct your own solution.

Writing a Recursive Program Sub-instances need to be smaller. “Size” is # nodes in the tree

Writing a Recursive Program Sub-instances need to be smaller. When the instance sufficiently small solve on your own.

Return things of the correct types. Writing a Recursive Program Return things of the correct types.

Not possible because must count # nodes Writing a Recursive Program Running Time T(n) = 2T(n/2) + (1) = (n) Faster? Not possible because must count # nodes in left tree.

Writing a Recursive Program Running Time T(n) = 2T(n/2) + (n) = (n logn) If tree very unbalanced? T(n) = 1T(n-1) + (n) = (n2)

Heaps, Heap Sort, & Priority Queues

Heap Definition Completely Balanced Binary Tree The value of each node ³ each of the node's children. Left or right child could be larger. Where can 9 go? Maximum is at root. Where can 1 go? Where can 8 go?

Completely Balanced Binary Tree Implemented by an Array Heap Data Structure Completely Balanced Binary Tree Implemented by an Array

Make Heap Get help from friends

Maximum needs to be at root. Heapify Where is the maximum? Maximum needs to be at root. ?

Heapify Find the maximum. Put it in place ? Repeat

Heapify Heap Running Time:

Make Heap Get help from friends Heapify Running time: T(n) = 2T(n/2) + log(n) = Q(n)

Another algorithm ? Heap Heaps

? Heap

? Heap

?

Heap

Running Time: i log(n) -i 2log(n) -i

Priority Queues

Selection Sort < Largest i values are sorted on side. Remaining values are off to side. 6,7,8,9 < 3 4 1 5 2 Exit 79 km 75 km Max is easier to find if a heap.

Heap Sort Largest i values are sorted on side. Remaining values are in a heap. 79 km 75 km Exit Exit

Heap Data Structure 9 8 7 6 Heap Array Heap Array 5 3 4 2 1

Heap Sort ? Largest i values are sorted on side. Remaining values are in a heap. 79 km 75 km Exit ? Put next value where it belongs. Heap Exit

Heap Sort

Heap Sort ? ? ? ? ? ? ? Sorted

Heap Sort Running Time:

Recursion on Trees Evaluate Equation Tree = ? Get help from friends 7 12 7 Get help from friends

Recursion on Trees Evaluate Equation Tree = rootop( value on left, value on right ) = rootop(12,7) = 12 + 7 = 19 7 12

Recursion on Trees Evaluate Equation Tree Base Case ? 7

Derivatives Input: a function f. Output: Its derivative df/dx.

Derivatives

Derivatives Input: a function f. Output: Its derivative df/dx.

Derivatives Input: a function f. Output: Its derivative df/dx.

Derivatives Output: Its derivative df/dx. Input: a function f. g

Simplify Input: a function f. Output: f simplified.

Simplify

Recursion on Trees Printing a Tree When I taught this one year, a student needed just this algorithm for his job.

Recursion on Trees Get help from friends Typing line by line?

Recursion on Trees One stack frame prints: His input gives: his tree sting for each line whether left or right

Recursion on Trees He gives his friends: their trees string for each line whether left or right

Parsing Input: s=6*8+((2+42)*(5+12)+987*7*123+15*54) Output:

Parsing

Parsing

Parsing Algorithm: GetExp( s, i ) Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid expression j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54)

Parsing Algorithm: GetTerm( s, i ) Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid term j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54)

Parsing Algorithm: GetFact( s, i ) Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid factor j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54)

Algorithm: GetExp( s, i ) p<term,1> p<term,2> p<term,3>

Algorithm: GetExp( s, i ) … p<term,1> p<term,2> p<term,k> +

Algorithm: GetTerm( s, i ) p<fact,1> p<fact,2>

Algorithm: GetTerm( s, i ) … p<fact,1> p<fact,2> p<fact,k> *

Parsing Algorithm: GetFact( s, i )

Parsing Algorithm: GetFact( s, i ) Fact 42

Algorithm: GetFact( s, i ) p<exp>

Algorithm: GetFact( s, i ) p<exp> ( )

Parsing Stackframes  nodes in parse tree

Recursive Images if n=1 if n=0, draw n=0 else draw And recursively Draw here with n-1

Recursive Images if n=2 if n=0, draw n=1 else draw And recursively Draw here with n-1

Recursive Images if n=3 if n=0, draw n=2 else draw And recursively Draw here with n-1

Recursive Images if n=30 if n=0, draw else draw And recursively Draw here with n-1

Recursive Images if n=0 if n=5 if n=4 if n=3 if n=1 if n=2

Recursive Images if n=0 if n=5 if n=4 if n=2 if n=1 if n=3

Recursive Images Þ ¥ L(n) = 4/3 L(n-1) = (4/3)n

Ackermann’s Function How big is A(5,5)?

Ackermann’s Function

Ackermann’s Function

Ackermann’s Function 3

Ackermann’s Function

Ackermann’s Function

Ackermann’s Function

Ackermann’s Function

Ackermann’s Function