Great Theoretical Ideas in Computer Science

X1 X1 X2 X2 + + X3 X3 Lecture 8 (September 17, 2009) Counting III: Generating Functions

Recap

The Binomial Formula (1+X) 0 = (1+X) 1 = (1+X) 2 = (1+X) 3 = (1+X) 4 = X 1 + 2X + 1X X + 3X 2 + 1X X + 6X 2 + 4X 3 + 1X 4 Pascal’s Triangle: k th row are coefficients of (1+X) k Inductive definition of k th entry of n th row: Pascal(n,0) = Pascal (n,n) = 1; Pascal(n,k) = Pascal(n-1,k-1) + Pascal(n-1,k)

Pascal’s Triangle “It is extraordinary how fertile in properties the triangle is. Everyone can try his hand”

Summing on k th Avenue  i = k n i k n+1 k+1 =

Fibonacci Numbers = 2 = 3 = 5 = 8 = 13

Sums of Squares n n n k  k = 0 n 2 =

All these properties can be proved inductively and algebraically. We gave combinatorial proofs using the Manhattan block walking representation of binomial coefficients

How many shortest routes from A to B? B A 10 5

Manhattan j th streetk th avenue There areshortest routes from (0,0) to (j,k) j+k k

Manhattan Level nk th avenue There areshortest routes from (0,0) to (n-k,k) n k

Manhattan Level nk th avenue There are shortest routes from (0,0) to n k level n and k th avenue

Level nk th avenue n k n-1 k-1 n-1 k = +

Level nk th avenue n k 2 How many ways to get to via ?

Level nk th avenue n n n k  k = 0 n 2 =

Level nk th avenue How many ways to get to via ? n+1 k+1 i k

Level nk th avenue n+1 k+1 i k  i = k n = n+1 k+1

1 + X 1 + X 2 + X 3 + … + X n-2 + X n-1 = X - 1 X n – 1 Recall the Geometric Series 1 - X 1 - X n =

1 + X 1 + X 2 + X 3 + … + X n + … = 1 - X 1 the Infinite Geometric Series Also makes sense if we view the infinite sum on the left as a formal power series Holds when |X| < 1

1 + X 1 + X 2 + X 3 + … + X n + … 1 - X 1 - X 1 - X 2 - X 3 - … - X n - X n+1 - … (1- X) P(X) =  P(X) = -X * P(X) = P(X) = 1

Formal Power Series  i = 0  aiXiaiXi P(X) = There are no worries about convergence issues. This is a purely syntactic object.

Formal Power Series  i = 0  aiXiaiXi P(X) = If you want, think of as the infinite vector V = But, as you will see, thinking of as a “polynomial” is very natural.

Operations A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … adding them together (A+B)(X) = (a 0 +b 0 ) + (a 1 +b 1 ) X + (a 2 +b 2 ) X 2 + … like adding the vectors position-wise + =

Operations A(X) = a 0 X 0 + a 1 X 1 + a 2 X 2 + … multiplying by X X * A(X) = 0 X 0 + a 0 X 1 + a 1 X 2 + a 2 X 3 + … like shifting the vector entries SHIFT =

Example Example: V = n th row of Pascal’s triangle Store: V = V := ; Loop n times V := V + SHIFT(V);

Example: V := ; Loop n times V := V + SHIFT(V); V = n th row of Pascal’s triangle P V := 1; P V := P V (1+X); Example

Example: V := ; Loop n times V := V + SHIFT(V); V = n th row of Pascal’s triangle Example P V = (1+ X) n As expected, the coefficients of P V give the n th row of Pascal’s triangle

To summarize…  i = 0  aiXiaiXi P(X) = Given a sequence V = associate a formal power series with it This is the “generating function” for V

Fibonaccis i.e., the sequence is represented by the power series 0 + 1X 1 + 1X X X X X 6 +… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2

Two Representations A(X) = 0 + 1X 1 + 1X X X X X 6 +… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 Can we write A(X) more succinctly?

A(X) = F 0 + F 1 X 1 + F 2 X 2 + F 3 X 3 + … + F n X n +… 0 + F 0 X 1 + F 1 X 2 + F 2 X 3 + … + F n-1 X n +… X A(X) = X 1 + F 0 X 2 + F 1 X 3 + … + F n-2 X n +… X 2 A(X) = (1 – X – X 2 ) A(X) (F 0 – 0 – 0) + (F 1 – F 0 – 0)X = X = (1 – X – X 2 ) X A(X) =

Fibonaccis F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 (1 – X – X 2 ) X A(X) = has the generating function i.e., the coefficient of X n in A(X) is F n

1 – X – X 2 X X 2 + X 3 -(X – X 2 – X 3 ) X 2X 3 + X 4 -(X 2 – X 3 – X 4 ) + X 2 -(2X 3 – 2X 4 – 2X 5 ) + 2X 3 3X 4 + 2X 5 + 3X 4 -(3X 4 – 3X 5 – 3X 6 ) 5X 5 + 3X 6 + 5X 5 -(5X 5 – 5X 6 – 5X 7 ) 8X 6 + 5X 7 + 8X 6 -(8X 6 – 8X 7 – 8X 8 )

Two representations of the same thing… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 (1 – X – X 2 ) X A(X) =

Closed form? F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 (1 – X – X 2 ) X A(X) = let’s factor (1 – X – X 2 ) (1 – X – X 2 ) = (1 - φ X)(1 – (-1/ φ )X) where φ = 1 + √5 2

Simplify, simplify… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 X A(X) = (1 - φ X)(1 – (-1/ φ )X) A(X) = 1 (1 - φ X) 1 (1 – (-1/ φ )X) √5 1 + some elementary algebra omitted…* *you are not allowed to say this in your answers…

A(X) = 1 (1 - φ X) 1 (1 – (-1/ φ )X) √ (1 - φ X) = 1 + φ X + φ 2 X 2 + … + φ n X n + … = 1 + Y 1 + Y 2 + Y 3 + … + Y n + … 1 - Y 1 the Infinite Geometric Series

A(X) = 1 (1 - φ X) 1 (1 – (-1/ φ )X) √ (1 - φ X) = 1 + φ X + φ 2 X 2 + … + φ n X n + …  the coefficient of X n in A(X) is… √5 1 (-1/ φ ) n √5 + φnφn 1 (1 – (-1/ φ )X) = 1 + (-1/ φ ) X + … + (-1/ φ ) n X n + …

√5 1 (-1/ φ ) n √5 + φnφn F n = where φ = 1 + √5 2 J.P.M. Binet (1843) Closed form for Fibonaccis Leonhard Euler (1765) Abraham de Moivre (1730) “golden ratio”

√5 1 (-1/ φ ) n √5 + φnφn F n = Closed form for Fibonaccis √5 1 φnφn F n = closest integer to

To recap…  i = 0  aiXiaiXi P(X) = Given a sequence V = associate a formal power series with it This is the “generating function” for V We just used this for the Fibonaccis…

Multiplication A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … multiply them together (A*B)(X) = (a 0* b 0 ) + (a 0 b 1 + a 1 b 0 ) X + (a 0 b 2 + a 1 b 1 + a 2 b 0 ) X 2 + (a 0 b 3 + a 1 b 2 + a 2 b 1 + a 3 b 0 ) X 3 + … seems a bit less natural in the vector representation (it’s called a “convolution” there)

Mult.: special case A(X) = a 0 + a 1 X + a 2 X 2 + … Special case: B(X) = 1 + X + X 2 + … multiply them together (A*B)(X) = a 0 + (a 0 + a 1 ) X + (a 0 + a 1 + a 2 ) X 2 + (a 0 + a 1 + a 2 + a 3 ) X 3 + … it gives us partial sums! 1 1-X =

For example… Suppose A(X) = 1 + X + X 2 + … and B(X) = 1 + X + X 2 + … 1 1-X = 1 = then (A*B)(X) = 1 + 2X + 3X 2 + 4X 3 + … 1 1-X = 1 * 1 (1-X) 2 = Generating function for the sequence

Take 1 + 2X + 3X 2 + 4X 3 + … 1 (1-X) 2 = ¢ 1 + ¢ 2 X 1 + ¢ 3 X 2 + ¢ 4 X 3 +… 1 (1-X) 3 = multiplying through by 1/(1-X) Generating function for the triangular numbers! What happens if we again take prefix sums?

1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = What’s the pattern? 1 (1-X) n = ???

1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = (1-X) n = ??? What’s the pattern?,,,, …

1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = ,,,, … 1 (1-X) n = ??? What’s the pattern?

1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = ,,,, … (1-X) n = ??? What’s the pattern?

1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = ,,,, … (1-X) n = What’s the pattern?

1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = ,,,, … (1-X) n =  k = 0  XkXk k+n-1 n-1 What’s the pattern?

From Lecture #7  i = 0 k i n k+1 n+1 = Can be used to derive the coefficient of X k

1 (1-X) n =  k = 0  XkXk k+n-1 n-1 What’s the pattern?

What is the coefficient of X k in the expansion of: ( 1 + X + X 2 + X 3 + X ) n ? Each path in the choice tree for the cross terms has n choices of exponent e 1, e 2,..., e n ¸ 0. Each exponent can be any natural number. Coefficient of X k is the number of non-negative solutions to: e 1 + e e n = k Another way to see it…

n + k - 1 n - 1 What is the coefficient of X k in the expansion of: ( 1 + X + X 2 + X 3 + X ) n ? Another way to see it…

Recap: getting partial sums A(X) = a 0 + a 1 X + a 2 X 2 + … a 0 + (a 0 + a 1 ) X + (a 0 + a 1 + a 2 ) X 2 + (a 0 + a 1 + a 2 + a 3 ) X 3 + … dividing by (1-X) gives us partial sums! A(X) 1-X =

Here’s an interesting use… 1 (1-X) 3 =  k = 0  XkXk (k+2)(k+1) 2 subtract off and add k+2 2 3/2 (1-X) X ½ X(1+X) 2(1-X) 3 =  k = 0  XkXk k2k2 2

From the previous page… X(1+X) (1-X) 3 =  k = 0  XkXk sum of first k squares hence if we take partial sums… X(1+X) (1-X) 3 = 1 1-X *  k = 0  XkXk k2k2

Coefficient of X k in (X 2 +X)/(1-X) 4 is the sum of the first k squares:

So finally…

Finally, a different counting problem…

Let c n = number of ways to pick exactly n fruits. What is a closed form for c n ? E.g., c 5 = 6

Recall Multiplication A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … multiply them together (A*B)(X) = (a 0* b 0 ) + (a 0 b 1 + a 1 b 0 ) X + (a 0 b 2 + a 1 b 1 + a 2 b 0 ) X 2 + (a 0 b 3 + a 1 b 2 + a 2 b 1 + a 3 b 0 ) X 3 + …

So if A(x), B(x), O(x) and P(x) are the generating functions for the number of ways to fill baskets using only one kind of fruit the generating function for number of ways to fill basket using any of these fruit is given by C(x) = A(x)B(x)O(x)P(x)

Suppose we only pick bananas b n = number of ways to pick n fruits, only bananas. B(x) = 1 + x 2 + x 4 + x 6 + … 1 1-X 2 =

Suppose we only pick apples a n = number of ways to pick n fruits, only apples. A(x) = 1 + x 5 + x 10 + x 15 + … 1 1-X 5 =

Suppose we only pick oranges o n = number of ways to pick n fruits, only oranges. O(x) = 1 + x + x 2 + x 3 + x 4 1-X 5 1-X =

Suppose we only pick pears p n = number of ways to pick n fruits, only pears. P(x) = 1 + x 1-X 2 1-X =

Let c n = number of ways to pick exactly n fruits of any type  c n x n = A(x) B(x) O(x) P(x) = 1-X 2 1-X 1 1-X X 2 1-X 5 1-X = 1 (1-X) 2

1 c n is coefficient of X n in c n = n+1. Let c n = number of ways to pick exactly n fruits of any type

More operations: Differentiation A(X) = a 0 + a 1 X + a 2 X 2 + … differentiate it… A’(X) = a 1 + 2a 2 X + 3a 3 X 2 …  i = 0  (i+1)a i+1 X i A’(X) =  i = 0  ia i X i X A’(X) =

Formal Power Series Basic operations on Formal Power Series Writing the generating function for a recurrence binomial and multinomial coefficients Solving G.F. to get closed form G.F.s for common sequences Prefix sums using G.F.s Using G.F.s to solve counting problems Here’s What You Need to Know…