Jeff Edmonds York University

Jeff Edmonds York University
Thinking about Algorithms Abstractly Recursion Multiplying Recurrence Relations Code Stack of Stack Frames Tree of Stack Frames Friends and Strong Induction Towers of Hanoi Check List Merge & Quick Sort Simple Recursion on Trees Generalizing the Problem Things not to do Heap Sort & Priority Queues Trees Representing Equations Pretty Print Parsing Recursive Images Ackermann's Function Jeff Edmonds York University Lecture 3 COSC 3101

Precondition Postcondition On Step At a Time It can be difficult to understand where computation go. I implored you to not worry about the entire computation. x/4  3y (x-3x1)  y Strange(x,y): x1 = x/4; y1 = 3y; f1 = Strange( x1, y1 ); x2 = x - 3x1; y2 = y; f2 = Strange( x2, y2 ); return( f1+f2 ); next i-1 i T+1 next

Friends & Strong Induction
Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. If I could get in, I could get the key. Then I could unlock the door so that I can get in. Circular Argument!

Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. To get into my house I must get the key from a smaller house

Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. Use brute force to get into the smallest house.

Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. Advice: Do not trace it out. Trust your friend. Do your job. Use brute force to get into the smallest house.

Representations of Recursive Algorithms
MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) Views Pros Cons Code Tracing out stack of stack frames Tree of stack frames Friends (strong induction) How implement for a computer How runs on computer View entire computation Worry about one step at a time I need a higher level of understanding I am not a computer Crazy making Too much Must trust.

Recursion Friends & Strong Induction
MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) Know Precond: Postcond: ints x,y product xy Consider your input instance If it is small enough solve it on your own. Allocate work Construct one or more sub-instances It must be smaller and meet the precondition Assume by magic your friends give you the answer for these. Use this help to solve your own instance. Do not worry about anything else. Micro-manage friends by tracing out what they and their friend’s friends do. Who your boss is. X = 33 Y = 12 ac = 31 bd = 32 (a+b)(c+d) =63 = 3 = 6 =18 XY = 396 X = 3 Y = 1 XY = 3 Y = 2 XY = 6 X = 6 Y = 3 XY = 18

Recursion Friends & Strong Induction
MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) This technique is often referred to as Divide and Conquer X = 33 Y = 12 ac = 31 bd = 32 (a+b)(c+d) =63 = 3 = 6 =18 XY = 396 X = 3 Y = 1 XY = 3 Y = 2 XY = 6 X = 6 Y = 3 XY = 18

Complex Numbers Remember how to multiply 2 complex numbers?
(a+bi)(c+di) = [ac –bd] + [ad + bc] i Input: a,b,c,d Output: ac-bd, ad+bc If a real multiplication costs 1 and an addition costs a penny. What is the cheapest way to obtain the output from the input? Can you do better than 4.02?

Gauss’ $3.05 Method: Input: a,b,c,d Output: ac-bd, ad+bc
m1 = ac m2 = bd A1 = m1 – m2 = ac-bd m3 = (a+b)(c+d) = ac + ad + bc + bd A2 = m3 – m1 – m2 = ad+bc

Question: The Gauss “hack” saves one multiplication out of four. It requires 25% less work. Could there be a context where performing 3 multiplications for every 4 provides a more dramatic savings?

How to add 2 n-digit numbers.
Running Time How to add 2 n-digit numbers. * * * * * * * * * * * * * * * * * * * * * * + Decimal Digits or Binary Bits I don’t care. Tom Lehrer New Math

Running Time How to add 2 n-digit numbers. * * * * * * * * * * * * * * * * * * * * * * * * +

Running Time How to add 2 n-digit numbers. * * * * * * * * * * * * * * * * * * * * * * * * * * +

Running Time How to add 2 n-digit numbers. * * * * * * * * * * * * * * * * * * * * * * * * * * * * +

Running Time How to add 2 n-digit numbers. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * +

Running Time How to add 2 n-digit numbers. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * +

Running Time How to add 2 n-digit numbers. * * * * * * + On any reasonable computer adding 3 digits can be done in constant time. T(n) = The amount of time grade school addition uses to add two n-digit numbers ≤ θ(n) = linear time. Means some constant (eg 10) times n.

Running Time How to add 2 n-digit numbers. QUESTION: Is there an algorithm to add two n-digit numbers whose time grows sub-linearly in n? No! You must read the input to know the answer And this takes time n!

Running Time How to add 2 n-digit numbers.
So any algorithm for addition must use time at least linear in the size of the numbers. Grade school addition is essentially as good as it can be.

How to multiply 2 n-bit numbers.
Running Time How to multiply 2 n-bit numbers. * * * * * * * *  * * * * * * * * * * * * * * * * n2 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

For big inputs, even 100000 n << 0.000001 n2
Running Time Grade School Addition: Linear time Grade School Multiplication: Quadratic time Neat! We have demonstrated that as things scale multiplication is a harder problem than addition. For big inputs, even n << n2 # of bits in numbers t ime

Don’t jump to conclusions! We compared two algorithms.
Running Time Grade School Addition: Linear time Grade School Multiplication: Quadratic time Neat! We have demonstrated that as things scale multiplication is a harder problem than addition. Don’t jump to conclusions! We compared two algorithms. We did not prove that no possible multiplication algorithm runs in linear time. Is there a clever algorithm to multiply two numbers in less than quadratic time?

Grade School vs Kindergarten
Running Time Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * n2 a × b = a + a + a a b T(n) = Time multiply T(n) = Time multiply = θ(n2) = quadratic time. = θ(b) = linear time. Which is faster? × 9 Minutes? Add a digit? Which would you use? Centuries? Exponential?

Running Time On Line Banking? Given a value N find
N = a × b loop b = 2..N check if N/b T(n) = Time factor = θ(N) = linear time. 9 Minutes? Add a digit? Centuries? Exponential?

# of bits = log2(Value) Value = 2# of bits
Size of Input Instance 2’’ 83920 5 1’’ Size of paper # of bits # of digits Value - n = 2 in2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive Formal Reasonable Unreasonable # of bits = log2(Value) Value = 2# of bits

Running Time Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * n2 a × b = a + a + a a b T(n) = Time multiply T(n) = Time multiply = θ(n2) = quadratic time. = θ(b) = linear time. Which is faster? × b = value = Time ≈ n = # digits = 20 Time ≈ 202 ≈ 400

Running Time Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * n2 a × b = a + a + a a b T(n) = Time multiply T(n) = Time multiply = θ(n2) = quadratic time. = θ(b) = linear time. Which is faster? × b = value n = # digits = 20 Time ≈ 202 ≈ 400 ≈ 10n Time ≈ 10n ≈ exponential!!!

Running Time Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * n2 a × b = a + a + a a b T(n) = Time multiply T(n) = Time multiply = θ(n2) = quadratic time. = θ(b) = linear time. Which is faster? × 9 n = # digits = 20 Time ≈ 202 ≈ 400 Adding a single digit multiplies the time by 10!

Kindergarten Multiplication: Exponential time
Grade School Addition: Linear time Grade School Multiplication: Quadratic time Oops this was worse! Kindergarten Multiplication: Exponential time For big inputs, n << n2 << 2n # of bits in numbers t ime

Divide And Conquer (an approach to faster algorithms)
DIVIDE my instance to the problem into smaller instances to the same problem. Have a friend (recursively) solve them. Do not worry about it yourself. GLUE the answers together so as to obtain the answer to your larger instance.

Multiplication of 2 n-bit numbers
The input X can be thought of in two ways: as an integer X = 2134 as a string X = “2134” of n=4 digits. A recursive program gives each “friend” an smaller instance. Simply cut this string into two halves a and b. These can be thought of in two ways: as strings a = “21” and b= “34” with n/2 = 2 digits each. as integers a = and b= 34. Note: X = = 21 = a10n/2 + b X = a b = a 2n/ b Y = c d = c 2n/ d

X = a 2n/2 + b and Y = c 2n/2 + d X  Y = (a 2n/2 + b)  (c 2n/2 + d) = ac 2n + (ad+bc) 2n/2 + cd Example: X = 23 and Y = 12 ac 10n + ( ad + bc ) 10n/2 + cd = 2 (22+31) 2 X  Y =

X = Y = XY = ac 2n + (ad+bc) 2n/2 + bd a b c d MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

T(n) = time taken by MULT on two n-bit numbers What is T(n)? What is its growth rate? Is it θ(n2)? MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

Recurrence Relation T(1) = k for some constant k
T(n) = 4 T(n/2) + k’ n + k’’ (for some constants k’ and k’’) MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

Recurrence Relation Lets be concrete T(1) = 1 T(n) = 4 T(n/2) + n
How do we unravel T(n)? MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

Technique 1 Guess and Verify
Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = 2n2 – n Verify: Left Hand Side Right Hand Side T(1) = 2(1)2 – 1 T(n) = 2n2 – n 1 4T(n/2) + n = 4 [2(n/2)2 – (n/2)] + n

Technique 2: Decorate The Tree
= 1 T(n) = n + 4 T(n/2) T(n) T(n) = = n n T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2)

T(n) = n T(n/2) T(n/2) T(n/2) T(n/2)

T(n) = n n/2 T(n/4) T(n/2) T(n/2) T(n/2)

T(n) = n n/2 T(n/4) n/2 T(n/4) n/2 T(n/4) n/2 T(n/4)

T(n) = n n/2 n/2 n/2 n/2 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4
……………………

Level i is the sum of 4i copies of n/2i
n/ n/ n/ n/2 Level i is the sum of 4i copies of n/2i 1 2 n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 i log2n

Level i is the sum of 4i copies of n/2i
= 4i ×(n/2i) Level i is the sum of 4i copies of n/2i Increases by a factor of 2. = 4logn×(n/2logn) log2n alogn = (2loga)logn = 2(loga · logn) = (2logn)loga = nloga = nlog4 × 1 = n2 Total: ?

Geometric Sum

Geometric Increasing ∑i=0..n ri = r0 + r1 + r2 +. . . + rn
= Q(biggest term) True whenever terms increase quickly

Adding Made Easy

Increases by a factor of 2.
Total = biggest =1×n = 4×n/2 = 16×n/4 = 4i ×n/2i Increases by a factor of 2. = 4logn×n/2logn = nlog4 × 1 = n2 Total: θ(n2)

All that work for nothing!
Divide and Conquer MULT: θ(n2) time Grade School Multiplication: θ(n2) time All that work for nothing! Let’s understand the math better.

Evaluating: T(n) = aT(n/b)+f(n)

Recurrence Relations » Time of Recursive Program
T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) With all this recursing and looping, how do we determine the running time of this recursive program? Recurrence relations.

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) Let n be the “size” of our input.

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) Let T(n) be the # of “Hi”s printed by me

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) Let T(n) be the # of “Hi”s printed by me And by all of my friends.

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) If my input is sufficiently small (according to my measure of size) then I print (1) “Hi”s.

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) I personally output f(n) “Hi”s, i.e. top level stack frame.

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) I have a friends i.e. recurse a times.

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) Let b be the factor by which I shrink the input before giving it to a friend. i.e. if my input has size n then my friends are of size n/b.

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) By definition of T, the number of “Hi”s printed by one of my friends and all of his friends is T(n/b).

T(n) = a T(n/b) + f(n) procedure Eg(In) n = |In| if(n£1) then put “Hi” else loop i=1..f(n) loop i=1..a In/b = In cut in b pieces Eg(In/b) Hence, the total number printed by me and my a friends is T(n) = a T(n/b) + f(n)

Level 1 2 i h

Level Instance size 1 2 i h

Level Instance size n 1 2 i h

Level Instance size n 1 n/b 2 i h

Level Instance size n 1 n/b 2 n/b2 i h

Level Instance size n 1 n/b 2 n/b2 i n/bi h n/bh

Level Instance size n 1 n/b 2 n/b2 i n/bi h n/bh = 1 base case

Level Instance size n 1 n/b 2 n/b2 i n/bi h n/bh = 1

Level Instance size n 1 n/b 2 n/b2 i n/bi h = log n/log b n/bh = 1 bh = n h log b = log n h = log n/log b

Level Instance size Work in stack frame n 1 n/b 2 n/b2 i n/bi h = log n/log b

Level Instance size Work in stack frame n f(n) 1 n/b f(n/b) 2 n/b2 f(n/b2) i n/bi f(n/bi) h = log n/log b T(1)

Level Instance size Work in stack frame # stack frames n f(n) 1 n/b f(n/b) 2 n/b2 f(n/b2) i n/bi f(n/bi) h = log n/log b n/bh T(1)

Level Instance size Work in stack frame # stack frames n f(n) 1 n/b f(n/b) a 2 n/b2 f(n/b2) a2 i n/bi f(n/bi) ai h = log n/log b n/bh T(1) ah

Level Instance size Work in stack frame # stack frames n f(n) 1 n/b f(n/b) a 2 n/b2 f(n/b2) a2 i n/bi f(n/bi) ai h = log n/log b n/bh T(1) ah ah = a = n log n/log b log a/log b

Level Instance size Work in stack frame # stack frames Work in Level n f(n) 1 n/b f(n/b) a 2 n/b2 f(n/b2) a2 i n/bi f(n/bi) ai h = log n/log b n/bh T(1) n log a/log b

Level Instance size Work in stack frame # stack frames Work in Level n f(n) 1 1 · f(n) n/b f(n/b) a a · f(n/b) 2 n/b2 f(n/b2) a2 a2 · f(n/b2) i n/bi f(n/bi) ai ai · f(n/bi) h = log n/log b n/bh T(1) n log a/log b n · T(1) log a/log b Total Work T(n) = ∑i=0..h ai×f(n/bi)

= ∑i=0..h ai×f(n/bi) If a Geometric Sum ∑i=0..n xi = θ(max(first term, last term))

Level Instance size Work in stack frame # stack frames Work in Level n f(n) 1 1 · f(n) n/b f(n/b) a a · f(n/b) 2 n/b2 f(n/b2) a2 a2 · f(n/b2) i n/bi f(n/bi) ai ai · f(n/bi) h = log n/log b n/bh T(1) n log a/log b n · T(1) log a/log b Dominated by Top Level or Base Cases

Evaluating: T(n) = aT(n/b) + nc = 4T(n/2) + n
1 Time for top level: Time for base cases: Dominated?: c = 1 < 2 = log a/log b θ(n ) = log a/log b θ(n ) = θ(n2) log 4/log 2 Hence, T(n) = ? = θ(base cases) = θ(n ) = θ(n2). log a/log b If we reduce the number of friends from 4 to 3 is the savings just 25%?

T(n) = n n/2 T(n/4) n/2 T(n/4) n/2 T(n/4) n/2 T(n/4)

1 Time for top level: Time for base cases: Dominated?: c = 1 < 1.58 = log a/log b θ(n ) = log a/log b θ(n ) = θ(n1.58) log 3/log 2 Hence, T(n) = ? = θ(base cases) = θ(n ) = θ(n1.58). log a/log b Not just a 25% savings! θ(n2) vs θ(n1.58..)

Time for top level: n2, c=2 Time for base cases: Dominated?: c = = log a/log b θ(n ) = log a/log b θ(n ) = θ(n1.58) log 3/log 2 2 > Hence, T(n) = ? = θ(top level) = θ(n2).

1.58 Time for top level: n1.58, c=1.58 Time for base cases: Dominated?: c = 1.58 = log a/log b Hence, all θ(logn) layers require a total of θ(n1.58) work. The sum of these layers is no longer geometric. Hence T(n) = θ(n1.58 logn) θ(n ) = log a/log b θ(n ) = θ(n1.58) log 3/log 2

Level Instance size Work in stack frame # stack frames Work in Level n f(n) 1 1 · f(n) n/b f(n/b) a a · f(n/b) 2 n/b2 f(n/b2) a2 a2 · f(n/b2) i n/bi f(n/bi) ai ai · f(n/bi) h = log n/log b n/bh T(1) All levels the same: Top Level to Base Cases n log a/log b n · T(1) log a/log b

Logs log 27 = log 9

Logs log? 27 = log? 9 Which base?

Logs log2 27 log2 c · logc 27 logc 27 = = log2 9 log2 c · logc 9
It does not matter. Which is easiest?

Logs log3 27 log3 33 3 = = log3 9 log3 32 2 Please no calculators in exams. And I wont ask log 25 log 9

Divide and Conquer MULT: θ(n2) time Grade School Multiplication: θ(n2) time All that work for nothing! Let’s try to multiply faster.

MULT revisited T(1) = 1 T(n) = 4 T(n/2) + n
MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) ) We likely can’t reduce our work n. Later we will consider reducing the size n/2 of our friend’s instance. Can you see a way to reduce the number 4 of friends?

Gauss’ Hack: Input: a,b,c,d Output: ac, ad+bc, bd
A1 = ac A3 = bd m3 = (a+b)(c+d) A2 = m3 – A1- A3 = ad + bc 2 additions and one multiplication = ac + ad + bc + bd

Gaussified MULT - Karatsuba 1962)
MULT(X,Y) If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e2n + (MULT(a+b, c+d) – e - f) 2n/2 + f ) T(n) = 3 T(n/2) + n Hence T(n) = θ(n1.58)

Cutting into d pieces. X = Y = X = i=0..d-1ai 2i/d
ad-1 a2 a1 a0 bd-1 b2 b1 b0 X = Y = … X = i=0..d-1ai 2i/d Y = j=0..d-1bj 2j/d XY = i=0..d-1j=0,d-1.aibj 2(i+j)/d Instances for friends?

Cutting into d pieces. Y\X i=0..d-1j=0,d-1.aibj 2(i+j)/d XY = ad-1
bd-1 b2 b1 b0 Y\X aibj … bj ai XY = i=0..d-1j=0,d-1.aibj 2(i+j)/d Instances for friends Number of friends is d2.

Evaluating: T(n) = aT(n/b) + nc = T(n/ ) +
Cutting into d pieces. Evaluating: T(n) = aT(n/b) + nc = T(n/ ) + n1 d2 d Time for top level: Time for base cases: Dominated?: c = 1 < 2 = log a/log b n c = = n1 1 θ(n ) = log a/log b θ(n ) log d^2/log d = θ(n2) Hence, T(n) = ? = θ(base cases) = θ(n ) = θ(n2). log a/log b

Cutting into d pieces. All that work for nothing!

Cutting into d pieces. Y\X ad-1 a2 a1 a0 bd-1 b2 ai
… … a2 a1 a0 bd-1 = k=0..2d-2 (j aj bk-j) 2k/d … bj aibj … b2 b1 b0 XY = i=0..d-1j=0,d-1.aibj 2(i+j)/d Values needed Number of friends is 2d-1 Instance for friend Fancy Gaussian trick

Cutting into d pieces. Evaluating: T(n) = aT(n/b) + nc = T(n/ ) + n1 2d-1 d log 2d-1/log d log a/log b = ≈ log 2d/log d = log(d) +1/log d → 1 Say d=logn

Cutting into d pieces. Evaluating: T(n) = aT(n/b) + nc = T(n/ ) + n1 2d-1 d n c = = n1 1 Time for top level: Time for base cases: Dominated?: c = 1 ≈ log a/log b θ(n ) log a/log b ≈ θ(n1) Hence, T(n) = ? = θ(n log n). Actually θ(n log n log log n).

Multiplication Algorithms
Kindergarten ? n2n Grade School n2 Karatsuba n1.58… Fastest Known n logn 3*4= 107

The Goal is to UNDERSTAND Recursive Algorithms
Fundamentally to their core

Rudich www.discretemath.com
Representation: Understand the relationship between different representations of the same information or idea 1 2 3 Rudich

MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) Views Pros Cons Code Tracing out stack of stack frames Tree of stack frames Friends (strong induction) How implement for a computer How runs on computer View entire computation Worry about one step at a time I need a higher level of understanding I am not a computer Crazy making Too much Must trust.

Code Representation Pros: Cons: Runs on computers Precise and succinct
MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) Pros: Cons: Runs on computers Precise and succinct Perception that being able to code is the only thing needed to get a job. (false) I am not a computer I need a higher level of intuition. Prone to bugs Language dependent

Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = bd = (a+b)(c+d) = XY = Stack Frame: A particular execution of one routine on one particular input instance.

Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then return( XY )
Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = bd = (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY=

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY = 4

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = X = 1 Y = 3 XY = 3

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = X = 3 Y = 5 XY = 15

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = ?

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = X = 3 Y = 1 XY = 3

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY =

Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = X = 3 Y = 2 XY = 6

Stack of Stack Frames An so on …. MULT(X,Y):
If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = ? XY = An so on ….

Stack of Stack Frames Pros: Cons:
Trace what actually occurs in the computer Concrete. It is what students attempt to describe when told not to give code. Described in words it is impossible to follow who is doing what. Does not explain why it works. Demonstrates for only one of many inputs.

Tree of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then return( XY )
Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72

Tree of Stack Frames Pros: Cons: View the entire computation.
Good for computing the running time. Must describe entire tree. For each stack frame input instance computation solution returned. who calls who Structure of tree may be complex.

MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) Views Pros Cons Code Tracing out stack of stack frames Tree of stack frames Friends (strong induction) Worry about one step at a time Must trust.

MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = One Friend for each stack frame. Each worries only about his job. Imagine that you are one. X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72

MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) Know Precond: Postcond: ints x,y product xy Consider your input instance If it is small enough solve it on your own. Allocate work Construct one or more sub-instances It must be smaller and meet the precondition Assume by magic your friends give you the answer for these. Use this help to solve your own instance. Do not worry about anything else. Micro-manage friends by tracing out what they and their friend’s friends do. Who your boss is. X = 33 Y = 12 ac = 31 bd = 32 (a+b)(c+d) =63 = 3 = 6 =18 XY = 396 X = 3 Y = 1 XY = 3 Y = 2 XY = 6 X = 6 Y = 3 XY = 18

MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Consider generic instances. ac bd (a+b)(c+d) MULT(X,Y): Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Remember! Do not worry about Who your boss is. How your friends solve their instance.

MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f This solves the problem for every possible instance. X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72

Pros: Cons: Worry about one step at a time. An abstraction within which to develop, think about, and describe algorithms in such way that their correctness is transparent. Students resist it. Students expect too much from their friends. Each sub-instance must be a smaller instance to the same problem. Students expect too much from their boss. You only know your instance. You do not know your boss’s instance.

Carefully write the specifications for the problem. Required output <postCond>: Set of legal instances (inputs) <preCond>: To be sure that we solve the problem for every legal instance. So that we know what we can give to a friend. So that we know what is expected of us. what we can expect from our friend.

If you want more from your friends, change the pre and/or postconditions Be sure to document it. Be sure you and all of your friends are using the same pre/postconditions. Be sure to handle these changes yourself.

Induction Hypothesis: ``The recursive algorithm works for every instance of size n.'' Base case ``The algorithm works for every instance of any size.'' size i

Give and solve the recurrence relation for the Running Time of this algorithm. T(n) = aT(n/b)+f(n)

Input Size A Measure of Size, Size is a function
that takes as input the computation’s input I and that outputs Eg Input is <n,m>, Size(<n,m>) = n or 3n+7m Algorithm Designer gets to decide! According to this measure Your friend’s instance must be smaller than yours. You must solve sufficiently small instances on your own. a real number

Input Size Does the program always halt? m keeps getting bigger!
Algorithm Designer gets to define the measure of size. Size(<n,m>) = n Size = 3 2 1 I = <3,5> <2,10> <1,20> <0,30> Sufficiently small to halt.

Input Size Does the program always halt? Each subinstance is smaller.
But according to what measure of size? Size(<n,m>) = ? I = <4,2> <3,2> <4,1> <3,1> <4,0> <3,1> <4,-1> …. There is an infinite path!

Input Size Does the program always halt? Each subinstance is smaller.
But according to what measure of size? Size(<n,m>) = n+m Every friend’s instance is smaller. I = <4,2> <3,2> <4,1> When it gets sufficiently small the computation must stop. <3,1> <4,0> <3,1> <4,-1> If n+m0, then Maximum dept = n+m

Input Size Does the program always halt?
According to what single measure of size are both instances smaller? Size(<n,m>) = 5n+2m I = <n,m> <n-1,m+2> <n+1,m-3> size = 5(n-1)+2(m+2) = 5n+2m -1 size = 5(n+1)+2(m-3) = 5n+2m -1 Every friend’s instance is smaller. Maximum dept = 5n+2m

Input Size Does the program always halt?
According to what single measure of size are both instances smaller? Size(<n,m>) = I = <n,m> <n-4,m+2> <n-8,m+4> <n-12,m+6> <n-6,m+3> <n,m> There is an infinite path! ….

Try explaining what this algorithm does by tracing it out.
Output Try explaining what this algorithm does by tracing it out. I dare you!!! It is much easier to TAKE ONE STEP AT A TIME

Output n=1 X n=2 Y

Output n=1 X n=2 Y n=3 A ? B ? C

Output n=1 X n=2 Y n=3 A Y B X C

Output n=1 X n=2 Y n=3 AYBXC

Output n=1 X n=2 Y n=3 AYBXC n=4 A ? B ? C

Output n=1 X n=2 Y n=3 AYBXC n=4 A AYBXC B Y C

Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC

Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 A ? B ? C

Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 AAAYBXCBYCBAYBXCC

Time: T(1) = T(2) = 1 1 T(n) = T(n-1) + T(n-2) + 3  2T(n-1) +3 = 2Q(n) defined?

This job of mine is a bit daunting.
Towers of Hanoi This job of mine is a bit daunting. Where do I start? And I am lazy.

At some point, the biggest disk moves.
Towers of Hanoi At some point, the biggest disk moves. I will do that job.

To do this, the other disks must be in the middle.
Towers of Hanoi To do this, the other disks must be in the middle.

Towers of Hanoi How will these move? I will get a friend to do it.
And another to move these. I only move the big disk.

Towers of Hanoi

Towers of Hanoi Get a job at the Towers of Hanoi company
at level n=1,2,3,4,….. Or get transferred in as the president & you decide what your vice president does.

Towers of Hanoi Time: T(1) = 1, T(n) = ≈ 2(2T(n-2)) ≈ 4(2T(n-3))
≈ 2i T(n-i) ≈ 2n

Evaluating: T(n) = aT(n-b)+f(n)
2 1 Level Instance size Work in stack frame # stack frames Work in Level n T(0) f(n-ib) f(n-2b) f(n-b) f(n) a ai a2 1 n/b a · T(0) ai · f(n-ib) a2 · f(n-2b) a · f(n-b) 1 · f(n) n/b n-b n-hb n-ib n-2b h = ? h = n/b Exponential Likely dominated by base cases |base case| = 0 = n-hb h = n/b

Evaluating: T(n) = 1T(n-b)+f(n)
2 1 Level Instance size Work in stack frame # stack frames Work in Level n-b n n-hb n-ib n-2b T(0) f(n-ib) f(n-2b) f(n-b) f(n) h = n/b 1 f(0) f(n-ib) f(n-2b) f(n-b) f(n) h = ? Total Work T(n) = ∑i=0..h f(b·i) = θ(f(n)) or θ(n·f(n))

Check Lists for Recursive Programs
This is the format of “all” recursive programs. Don’t deviate from this. Or else!

This is input instance is your mission. You must return this output. (An x, y, & z in every computation path.) Focus on your mission.

The <preCond> & <postCond> must document these variables and what you must do.

To get help, you construct an instance for each friend It must be valid and smaller. And pass them to them.

Each friend returns a solution his sub-instance. Trust him to do this. But don’t expect him to do or know anything else.

Combine their solutions to construct a solution to your instance. Return your solution.

If your solution is too small to get solved by the general technique solve it yourself. Be sure you handle every legal instance.

You rarely need to change the values of these variable beyond their initial setting (this is not an iterative algorithm.) Your rarely need additional input, output, or local variable. But it you do document them.

Have NO global variables. If you change the value of a local variable n, neither your friends nor your boss get that value.

Sorting Problem Specification
<preCond>: The input is a list of n values with the same value possibly repeated. <postCond>: The output is a list consisting of the same n values in non-decreasing order. 88 14 98 25 62 52 79 30 23 31 14,23,25,30,31,52,62,79,88,98

Recursive Sorts Given list of objects to be sorted
Split the list into two sub-lists. Recursively have a friend sort the two sub-lists. Combine the two sorted sub-lists into one entirely sorted list.

Four Recursive Sorts Size of Sub-lists n-1,1 n/2,n/2 Merge Sort
Insertion Sort Quick Sort Selection Sort Minimal effort splitting Lots of effort recombining Lots of effort splitting Minimal effort recombining

Merge Sort 88 14 98 25 62 52 79 30 23 31 Divide and Conquer

Merge Sort (no real work) Split Set into Two
88 14 98 25 62 52 79 30 23 31 (no real work) Split Set into Two 25,31,52,88,98 Get one friend to sort the first half. 14,23,30,62,79 Get one friend to sort the second half.

Merge Sort Merge two sorted lists into one 25,31,52,88,98
14,23,25,30,31,52,62,79,88,98 14,23,30,62,79

Total work at any level = Q(n)
Merge Sort Sort Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Total work at any level = Q(n) # levels = Q(log(n))

2T(n/2) + Q(n) Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level n f(n) 1 1 · f(n) n/b f(n/b) a a · f(n/b) 2 n/b2 f(n/b2) a2 a2 · f(n/b2) i n/bi f(n/bi) ai ai · f(n/bi) h = log n/log b n/bh T(1) 2i·Q(n/2i) = Q(n) n log a/log b n · T(1) log a/log b All levels basically the same. Total Work T(n) = ∑i=0..h ai×f(n/bi) = Q(n) · log(n)

Quick Sort 88 14 98 25 62 52 79 30 23 31 Divide and Conquer

Quick Sort Partition set into two using randomly chosen pivot 88 14 98
25 62 52 79 30 23 31 14 25 30 23 31 88 98 62 79 ≤ 52 ≤

Quick Sort Get one friend to sort the first half.
14 25 30 23 31 88 98 62 79 ≤ 52 ≤ 14,23,25,30,31 Get one friend to sort the first half. 62,79,98,88 Get one friend to sort the second half.

Quick Sort Glue pieces together. (No real work) 14,23,25,30,31 52
62,79,98,88 Glue pieces together. (No real work) 14,23,25,30,31,52,62,79,88,98

Quick Sort Let pivot be the first element in the list? 88 14 98 25 62
52 79 30 23 31 14 88 98 30 ≤ 31 ≤ 62 23 25 52 79

Quick Sort 14,23,25,30,31,52,62,79,88,98 23,25,30,31,52,62,79,88,98 ≤ 14 ≤ If the list is already sorted, then the slit is worst case unbalanced.

Quick Sort T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Best Time: Worst Time:
Expected Time:

Quick Sort T(n) = T(0) + T(n-1) + Q(n) Best Time:
T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Worst Time: = Q(n2) Expected Time:

Quick Sort Best Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Worst Time:
T(n) = T(0) + T(n-1) + Q(n) = Q(n2) Expected Time: T(n) = T(1/3n) + T(2/3n) + Q(n) = Q(n log(n))

Quick Sort Expected Time: T(n) = T(1/3n) + T(2/3n) + Q(n)
= Q(n log(n)) Top work Q(n) 2nd level Q(n) Q(n) 3rd level 1/3 2/3 3/2 log (n) # levels = = Q(log(n))

Kth Element Problem Specification
<preCond>: The input is a list of n values and an integer k. <postCond>: The kth smallest in the list. k=3 88 14 98 25 62 52 79 30 23 31 Output: 25 14,23,25,30,31,52,62,79,88,98

Partition set into two using randomly chosen pivot 88 14 98 25 62 52 79 30 23 31 14 25 30 23 31 88 98 62 79 ≤ 52 ≤

14 25 30 23 31 88 98 ≤ 52 ≤ 62 79 k=3 r=5 elements on left 14,23,25,30,31 Get one friend to find k=3rd element in first half. Output: 25

14 25 30 23 31 88 98 ≤ 52 ≤ 62 79 k=8 r=5 elements on left Get one friend to find k=8-5=3rd element in second half. Output: 79 52,62,79,98,88

Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) n log a/log b # of base cases = Worst Time: n log 1/log 2 = = n0 = 1 n +n/2 +n/4 +n/8 … +1 Expected Time:

Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) Worst Time: T(n) = 1 T(n-1) + Q(n) ≈ n+T(n-1) ≈ n+(n-1)+T(n-2) ≈ n+(n-1)+(n-2)+T(n-3) ≈ n+(n-1)+(n-2)+(n-3)+…+1 ≈ Q(n2) Expected Time:

Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) Worst Time: T(n) = 1 T(n-1) + Q(n) = Q(n2) Expected Time: T(n) = 1 T(2/3n) + Q(n) = Q(n)

Power T(N) = 2T(N/2) + 1 = (N) Size = log(b) + log(N) = 2(n)
bN T(N) = 2T(N/2) + 1 = (N) Size = log(b) + log(N) = 2(n) b7 = b3 × b4 N=7 b4 b3 N=4 N=3 N=2 N=1

Power T(N) = 1T(N/2) + 1 = (log(N)) Size = log(b) + log(N) = (n)
b7 = (b3)2 × b N=7 b3 N=3 N=1

Recursion on Trees (define) 3 8 1 2 7 6 5 9 4

Recursion on Trees A binary tree is: - the empty tree
(define) A binary tree is: - the empty tree - a node with a right and a left sub-tree. tree because 3 8 1 2 7 6 5 9 4

(define) A binary tree is: - the empty tree - a node with a right and a left sub-tree. node tree 3 8 1 2 7 6 5 9 4 tree

(define) A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 tree because

(define) A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 node tree tree

(define) A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 tree because

(define) A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 node tree tree

(define) A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 2 7 6 5 9 4 tree because empty

Recursion on Trees number of nodes = ? Get help from friends 5 6
3 8 1 2 7 6 5 9 4

Recursion on Trees number of nodes
(friends) number of nodes = number on left + number on right + 1 = = 12 5 3 8 1 2 7 6 5 9 4 6

Recursion on Trees 6 (communication)
Being lazy, I will only consider my root node and my communication with my friends. I will never look at my children subtrees but will trust them to my friends. 3 8 1 2 7 6 5 9 4 6

We will only communicate through these walls Q(1) info via the pre & post-conditions. 3 8 1 2 7 6 5 9 4 6

Recursion on Trees (communication)
I know only what you tell me via the pre-condition. I tell you only what the post-condition instructs me to. 3 8 1 2 7 6 5 9 4

I certainly will tell you the answer for my subtree to the computational problem at hand 3 8 1 2 7 6 5 9 4

I can also tell you anything else about this subtree. Simply ask by changing the post-condition. 3 8 1 2 7 6 5 9 4

But remember, I know nothing about the outside world (root, other subtree, whether I am left or right.) except what you tell me. 3 8 1 2 7 6 5 9 4

I tell you extra information by changing the pre-condition. (Info about left subtree, the root, things my boss told me, …) 3 8 1 2 7 6 5 9 4 6

Of course if the pre or post-conditions change then I must fill this contract with my boss too. 3 8 1 2 7 6 5 9 4 6

Recursion on Trees number of nodes Base Case ? 1 Are we done?
3 8 1 2 7 6 5 9 4 1 Are we done?

Recursion on Trees number of nodes Base Case ? ? (base case) 3 8 1 2 7
6 5 9 4 ?

Recursion on Trees number of nodes Base Case ? Better base case!
3 8 1 2 7 6 5 9 4 Better base case!

Recursion on Trees number of nodes Base Case ? 1
3 8 1 2 7 6 5 9 4 1 Does this still need to be a base case?

Recursion on Trees number of nodes
(base case) number of nodes = number on left + number on right + 1 = = 1 3 8 1 2 7 6 5 9 4 No!

Recursion on Trees (base case) Most programmers don’t use the empty tree as a base case. This causes a lot more work. 3 8 1 2 7 6 5 9 4

Recursion on Trees (code)

Recursion on Trees Designing Program/Test Cases Try same code n1 n2
n1 n1+0+1 3 Same code works! 3 generic generic generic Try same code 0+0+1=1 Same code works! 3 Base Case

Recursion on Trees aT(n/b) + Q(nc) Time: T(n) = = 2T(n/2) + Q(1)
= T(left) + T(right) + Q(1) = ?? Time: T(n) = This only works for balanced tree. We don’t know how many nodes on left vs right. How do we solve this?

Recursion on Trees ∑stack frame Work done by stack frame Time: T(n) =
X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 Y = 1 XY=6 X = 6 XY=18 X = 5 X = 4 XY=20 X = 9 Y = 8 XY=72

Recursion on Trees ∑stack frame Work done by stack frame Time: T(n) =
= Q(n) × Q(1) = Q(n) One stack frame for each node in the tree and for empty trees hang off And constant work per stack frame.

Recursion on Trees Many Children number of nodes =
(mult-children) Recursion on Trees (friends) Many Children number of nodes = = 13 One friend for each sub-tree. 4 2 3 8 1 2 7 6 5 9 4

(mult-children) Recursion on Trees (code)

Recursion on Trees Designing Program/Test Cases Try same code n1 n3
(mult-children) Recursion on Trees (cases) Designing Program/Test Cases Try same code n1 n3 n1 + n2 + n3 + 1 n2 n1 n1 + 1 3 3 Same code works! generic generic generic generic But is this needed (if not the input) Try same code +1=1 3 Same code works!

(mult-children) Recursion on Trees (code)

Recursion on Trees ∑stack frame Work done by stack frame
(mult-children) Recursion on Trees (time) ∑stack frame Work done by stack frame Time: T(n) = = Q(n) × Q(1) = Q(n) Q(n) = Q(n2) One stack frame for each node in the tree And constant work per stack frame.

Recursion on Trees ∑stack frame Work done by stack frame
(mult-children) Recursion on Trees (time) ∑stack frame Work done by stack frame Time: T(n) = = ∑stack frame Q(# subroutine calls) = ∑node Q(# edge in node) = Q(edgeTree) Q(nodeTree) = Q(n)

Recursion on Trees We pass the recursive program a “binary tree”.
class LinkedBinaryTree { class Node { E element; Node parent; Node left; Node right; } Node root = null; Tree We pass the recursive program a “binary tree”. But what type is it really? This confused Jeff at first.

Recursion on Trees One would think tree is of type LinkedBinaryTree.
class LinkedBinaryTree { class Node { E element; Node parent; Node left; Node right; } Node root = null; Tree (LinkedBinaryTree tree) One would think tree is of type LinkedBinaryTree. Then getting its left subtree, would be confusing. left_Tree

Recursion on Trees It is easier to have tree be of type Node.
class LinkedBinaryTree { class Node { E element; Node parent; Node left; Node right; } Node root = null; Tree tree (Node tree) It is easier to have tree be of type Node. But it is thought of as the subtree rooted at the node pointed at. The left child is then or tree.left tree.Getleft() or or Tree.leftSub(tree) leftSub(tree)

Recursion on Trees class LinkedBinaryTree { class Node { E element; Node parent; Node left; Node right; } Node root = null; public int NumberNodes() { return NumberNodesRec( root ); } Tree tree private int NumberNodesRec (Node tree) NumberNodesRec But the outside user does not know about pointers to nodes.

Recursion on Trees sum of nodes = ? Get help from friends 25 23 (sum)
8 1 2 7 6 5 9 4

Recursion on Trees sum of nodes
= sum on left + sum on right + value at root = = 51 25 3 8 1 2 7 6 5 9 4 23

Recursion on Trees sum of nodes = ? Base Case ? (sum) 3 8 1 2 7 6 5 9
4

Recursion on Trees (sum)

Recursion on Trees Designing Program/Test Cases s1 s2 s1 + s2 + 3 s1
(sum) Designing Program/Test Cases s1 s2 s1 + s2 + 3 s1 s1 + 3 3 3 generic generic generic =3 3

Recursion on Trees max of nodes = ? Get help from friends 9 8 (max) 3
1 2 7 6 5 9 4

Recursion on Trees max of nodes
= max( max on left, max on right, value at root) = max( 8, 9, 3) = 9 9 3 8 1 2 7 6 5 9 4 8

Recursion on Trees max of nodes = ? Base Case ? ? (max) 3 8 1 2 7 6 5
9 4 ?

Recursion on Trees 3+4+2+8+… Sum so far is 17 Sum so far is
(max) … Sum so far is 17 Sum so far is NewSum = OldSum + nextElement = =

Recursion on Trees 3*4*2*3*… Product so far is 72 Product so far is 1
(max) 3*4*2*3*… Product so far is 72 Product so far is 1 NewProd = OldProd × nextElement = × =

Recursion on Trees Conclusion so far is True
(max) True and True and False and … Conclusion so far is True Conclusion so far is True NewProd = OldProd and nextElement = True and True = True

Recursion on Trees -¥ Max(3,4,2,3,… Max so far is 4 Max so far is
NewMax = max(OldMax, next Element) = max( -¥ , ) =

Recursion on Trees (max)

Recursion on Trees height of tree = ? Get help from friends 4 3
8 1 2 7 6 5 9 4

Recursion on Trees height of tree
= max(height of left,height on right) + 1 = max(3,4) + 1 = 5 4 3 8 1 2 7 6 5 9 4 3

Recursion on Trees height of tree Base Case ? ? (height) 3 8 1 2 7 6 5
9 4 ?

Recursion on Trees height of tree? 2 nodes or 1 edge Base Case ?
0 nodes ? edges Base Case ? 3 8 1 2 7 6 5 9 4

Recursion on Trees Work it out backwards height = 0 edge
= max(height of left,height on right) + 1 = max(?,?) + 1 ? 3 8 1 2 7 6 5 9 4 ? edges

Recursion on Trees Work it out backwards height = 0 edge
= max(height of left,height on right) + 1 = max(-1,-1) + 1 3 8 1 2 7 6 5 9 4 -1 edges -1 -1

Recursion on Trees (height)

Recursion on Trees number of leaves = ? Get help from friends 2 3
8 1 2 7 6 5 9 4

Recursion on Trees number of leaves = number on left + number on right
= = 5 2 3 8 1 2 7 6 5 9 4 3

Recursion on Trees number of leaves Base Case ? (#leaves) 3 8 1 2 7 6
5 9 4

Recursion on Trees Designing Program/Test Cases L1 L2 L1 + L2 L1
(#leaves) Designing Program/Test Cases L1 L2 L1 + L2 L1 3 3 generic generic generic 1 No! 0+0 = 0 3 Needs to be another base case.

Recursion on Trees (#leaves)

Recursion on Trees (travers)

Recursion on Trees (copy)

Recursion on Trees (deallocate) ? Drops the tree.

Define Problem: Binary Search Tree
Recursion on Trees (BST) Define Problem: Binary Search Tree <preCond> Key A binary search tree. <postCond> Find key in BST (if there). 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71

Its left children ≤ Any node ≤ Its right children
Binary Search Tree (BST) Its left children ≤ Any node ≤ Its right children 38 ≤ 25 ≤ 51 17 31 42 63 4 21 28 35 40 49 55 71

Define Loop Invariant Maintain a sub-tree.
(BST) Maintain a sub-tree. If the key is contained in the original tree, then the key is contained in the sub-tree. 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71 key 17

Define Step Cut sub-tree in half.
(BST) Cut sub-tree in half. Determine which half the key would be in. Keep that half. 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71 key 17 If key < root, then key is in left half. If key = root, then key is found If key > root, then key is in right half.

Recursion on Trees (BST)

Recursion on Trees (IsBST)

Recursion on Trees Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n))
(IsBST) Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n)) If tree very unbalanced? T(n) = 1T(n-1) + (n) = (n2) Computing max is too much work.

Recursion on Trees (IsBST) Extra info from below

Recursion on Trees 1 Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n)) n
(IsBST) 1 n Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n)) Computing max is too much work.

Recursion on Trees Extra info from above (IsBST) min   max min 
38 min   max min   root root   max -¥, ¥

Things to Remember Things to Do and Things NOT TO DO

Friends - Strong Induction View of Recursion.
I am obsessed with the Friends - Strong Induction View of Recursion. Trust your friends to solve sub-instances. The sub-instance given must be smaller and must be an instance to the same problem. Combine solution given by friend to construct your own solution for your instance. Focus on one step. Do not talk of their friends friends friends. Solve small instances on your own. Do it!

Don't have inputs or outputs that are not explained!
Typical Test Answer Define pre & post conditions Don't have inputs or outputs that are not explained!

what that should look like.
Typical Test Answer “You did NOT ask for code in the question so I am explaining it as it is easier.” Yes, but I asked you to give a recursive algorithm and said very strongly in my Unit steps what that should look like. “We start at the leaf nodes and work up the tree.” But this is not recursion!

Typical Test Answer Call recursively on the correct types of input
k,num,v

returning the correct types of output
Typical Test Answer Call recursively returning the correct types of output Save the results (or don't bother calling)

Combine solutions given by friends to construct your own solution.
Typical Test Answer Combine solutions given by friends to construct your own solution.

Return things of the correct types.
Typical Test Answer Return things of the correct types. Not an element the root? In every path through code

Sub-instances need to be smaller.
Typical Test Answer Sub-instances need to be smaller. “Size” is size of the tree Wrong base case

What is the value of n here?
Typical Test Answer No Global Variables. What is the value of n here? Still zero. The friend has his own variable n.

Maybe they mean to pass the new n in and out.
Typical Test Answer No Global Variables. Maybe they mean to pass the new n in and out. Please don’t use these to pass out values when I am marking.

Looks like an iterative algorithm
Typical Test Answer Looks like an iterative algorithm Looks like an recursive algorithm Which is it?

Heaps, Heap Sort, & Priority Queues

Heap Definition Completely Balanced Binary Tree The value of each node
³ each of the node's children. Left or right child could be larger. Where can 9 go? Maximum is at root. Where can 1 go? Where can 8 go?

Completely Balanced Binary Tree Implemented by an Array
Heap Data Structure Completely Balanced Binary Tree Implemented by an Array

Maximum needs to be at root.
Heapify Where is the maximum? Maximum needs to be at root. ?

Heapify Find the maximum. Put it in place ? Repeat Time = O(logn)

Make Heap Get help from friends Heapify Running time:
T(n) = 2T(n/2) + log(n) = Q(n)

Priority Queues Sorted List Unsorted List Heap
Items arrive with a priority. O(n) O(1) O(logn) Item removed is that with highest priority.

Heap Sort Put number in a priority queue - O(n) time
Remove them one at a time in order - n  O(logn)

Recursion on Trees  + 3+47 (3+4)7 Evaluate Equation Tree = ?
12 7 Get help from friends  + 3+47 (3+4)7

Recursion on Trees  + (3+4)7 Evaluate Equation Tree
= rootop( value on left, value on right ) = rootop(7,7) = 7  7 = 49  7 7 + (3+4)7

Recursion on Trees Evaluate Equation Tree Base Case ?  7 + (3+4)7

Derivatives Input: a function f. Output: Its derivative df/dx.

Derivatives

Derivatives Input: a function f. Output: Its derivative df/dx.

Derivatives Output: Its derivative df/dx. Input: a function f. g

Simplify Input: a function f. Output: f simplified.

Simplify

Recursion on Trees Printing a Tree
When I taught this one year, a student needed just this algorithm for his job.

Recursion on Trees Get help from friends Typing line by line?

Recursion on Trees One stack frame prints: His input gives: his tree
sting for each line whether left or right

Recursion on Trees He gives his friends: their trees
string for each line whether left or right

Parsing Input: s=6*8+((2+42)*(5+12)+987*7*123+15*54) Output:

Context Free Grammars (3+4)7 exp term fact  ( exp ) 7 term + fact

Parsing/Compiling Input: Output: Java Code
Machine Code simulating the Java code.

Parsing/Compiling Input: Output: Java Code
Machine Code simulating the Java code. Challenge: Keep track of three algorithms simultaneously The compiler The Java code being compiled The MARIE code being produced.

Parsing

Parsing Algorithm: GetExp( s, i )
Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid expression j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54)

Parsing Algorithm: GetTerm( s, i )
Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid term j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54)

Parsing Algorithm: GetFact( s, i )
Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid factor j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54)

Algorithm: GetExp( s, i )
p<term,1> p<term,2> p<term,3>

Algorithm: GetExp( s, i )
… p<term,1> p<term,2> p<term,k> +

Algorithm: GetTerm( s, i )
p<fact,1> p<fact,2>

Algorithm: GetTerm( s, i )
… p<fact,1> p<fact,2> p<fact,k> *

Parsing Algorithm: GetFact( s, i )

Parsing Algorithm: GetFact( s, i ) Fact 42

Algorithm: GetFact( s, i )
p<exp>

Algorithm: GetFact( s, i )
p<exp> ( )

Parsing/Compiling This parsing algorithm only works for Look Ahead One Grammars. Look Ahead One: A grammar is said to be look ahead one if, given any two rules for the same non-terminal, the ﬁrst place that the rules diﬀer in actual characters. A ⇒ B ’u’ C ’w’ E A ⇒ B ’u’ C ’x’ F A ⇒ B ’u’ C A ⇒ B ’v’ G H ? (and next character is not ‘w’ or ‘x’) (Ok if first character(s) within a B is different than in a F.) A ⇒ F G This feature allows our parsing algorithm to look only at the next token in order to decide what to do next. Thus the algorithm runs in linear time.

Parsing/Compiling This parsing algorithm only works for Look Ahead One Grammars. A ⇒ a A a A ⇒  A ⇒ ( A ) A ⇒  ? (next character is ‘a’) Generates (((()))) Generates aaaaaaaa Not Look Ahead One GetA(s,i) if( s[i] = ‘(‘ ) pA,jA, = GetA(s,i+1) if( s[jA] = ‘)‘ ) return(  ‘(‘ pA ’)’, jA+1) else return( error ) return(,i) Parser can’t find middle. A ⇒ A B A ⇒ A C Recurses forever.

Parsing/Compiling This parsing algorithm only works for Look Ahead One Grammars. A ⇒ BC A ⇒ DE B ⇒ bbb .... D ⇒ bbb .... A ⇒ BC A ⇒ DE B ⇒ b .... D ⇒ d .... GetA(s,i) if( s[i] = ‘b‘ ) pB,jB, = GetB(s,i) pC,jC, = GetC(s,jB) return(  pB pC , jC ) elseif( s[i] = ‘d‘ ) pD,jD, = GetD(s,i) pE,jE, = GetE(s,jD) return(  pD pE , jE ) Not Look Ahead One Don’t know whether to call GetB or GetD

Parsing Stackframes  nodes in parse tree

Recursive Images if n=1 if n=0, draw n=0 else draw And recursively
Draw here with n-1 n=0

Draw here with n-1

Recursive Images if n=30 if n=0, draw else draw And recursively
Draw here with n-1

Recursive Images if n=0 if n=5 if n=4 if n=3 if n=1 if n=2

Recursive Images if n=0 if n=5 if n=4 if n=2 if n=1 if n=3

Recursive Images Þ ¥ L(n) = 4/3 L(n-1) = (4/3)n

Ackermann’s Function How big is A(5,5)? n applications 364

Ackermann’s Function n applications n applications 365

Ackermann’s Function n applications n applications 366

Ackermann’s Function n applications 367

Ackermann’s Function

Ackermann’s Function 372

Please give me feedback so that I can better serve you.
Please feel free to ask questions! Please give me feedback so that I can better serve you. Thanks for the feedback that you have given me already.

Cut Out

Hard to write an iterative program
to traverse on a recursive structure tree 3 8 1 2 7 6 5 9 4 Need pointers from each node to its parent.

Hard to write a recursive program that implements an iterative algorithm.

Writing a Recursive Program
Call recursively on the correct types of input

on sub-instances which will give useful information.
Writing a Recursive Program Call recursively on sub-instances which will give useful information. ? node in sub-tree 3 8 1 2 7 6 5 9 4

Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree node in whole tree ? 3 8 1 2 7 6 5 9 4

Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree nl node in whole tree +1 +3rd 3 8 1 2 7 6 5 9 4

But who will count nodes in left sub-tree? 3 8 1 2 7 6 5 9 4 nl nl My friend

returning the correct types of output
Writing a Recursive Program Call recursively returning the correct types of output Save the results (or don't bother calling)

Combine solutions given by friends to construct your own solution.
Writing a Recursive Program Combine solutions given by friends to construct your own solution.

Sub-instances need to be smaller. “Size” is # nodes in the tree

Sub-instances need to be smaller. When the instance sufficiently small solve on your own.

Return things of the correct types.
Writing a Recursive Program Return things of the correct types.

Not possible because must count # nodes
Writing a Recursive Program Running Time T(n) = 2T(n/2) + (1) = (n) Faster? Not possible because must count # nodes in left tree.

Running Time T(n) = 2T(n/2) + (n) = (n logn) If tree very unbalanced? T(n) = 1T(n-1) + (n) = (n2)

End Recursive Algorithms
Math Review Graphs Searching Algorithms

Heaps, Heap Sort, & Priority Queues 393

Heap Definition Completely Balanced Binary Tree The value of each node
³ each of the node's children. Left or right child could be larger. Where can 9 go? Maximum is at root. Where can 1 go? Where can 8 go? 394

Completely Balanced Binary Tree Implemented by an Array
Heap Data Structure Completely Balanced Binary Tree Implemented by an Array

Make Heap Get help from friends 396

Maximum needs to be at root.
Heapify Where is the maximum? Maximum needs to be at root. ?

Heapify Find the maximum. Put it in place ? Repeat 398

Heapify Heap Running Time:

Make Heap Get help from friends Heapify Running time:
T(n) = 2T(n/2) + log(n) = Q(n) 402

Another algorithm ? Heap Heaps

? Heap 404

? Heap

Running Time: i log(n) -i 2log(n) -i 408

Priority Queues

Selection Sort < Largest i values are sorted on side.
Remaining values are off to side. 6,7,8,9 < 3 4 1 5 2 Exit 79 km 75 km Max is easier to find if a heap. 410

Heap Sort Largest i values are sorted on side.
Remaining values are in a heap. 79 km 75 km Exit Exit

Heap Data Structure 9 8 7 6 Heap Array Heap Array 5 3 4 2 1 412

Heap Sort ? Largest i values are sorted on side. Remaining values are
in a heap. 79 km 75 km Exit ? Put next value where it belongs. Heap Exit

Heap Sort 414

Heap Sort ? ? ? ? ? ? ? Sorted

Heap Sort Running Time: 416

Jeff Edmonds York University

Similar presentations

Presentation on theme: "Jeff Edmonds York University"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Jeff Edmonds York University

Similar presentations

Presentation on theme: "Jeff Edmonds York University"— Presentation transcript:

Similar presentations

About project

Feedback