Transparency No. 5-1 Discrete Mathematics Chapter 5 Advanced Counting Techniques

Discrete Mathematics advanced counting techniques Transparency No. 5-2 Contents Recurrence Relations Solving Recurrence Relations Divide-and-conquer Relations The Inclusion-Exclusion principle Applications of the Inclusion-Exclusion principle

Discrete Mathematics advanced counting techniques Transparency No. 5-3 Recurrence relations #bacteria doubles every hour. Initially (t=0), there are 5 bacteria ==> #bacteria after n hours = ? sol: Let a n = #bacteria after n hours. => a 0 = (1) initial condition a n = 2 a n-1 for n > (2) recurrence relation (1) and (2) uniquely determine a n for all n  N. -- called a recurrence definition of the sequence a 0,a 1,..={a n } n  N Goal: find explicit formula for a n satisfying the recurrence relation (and initial conditions)

Discrete Mathematics advanced counting techniques Transparency No. 5-4 Definition of recurrence relations A recurrence relation for the sequence {a n } is a formula that express a n in terms of one or more of the previous terms of the sequence (i.e., some a j 's with j < n). A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation. Ex1: If {a n } satisfies (1) a n = a n-1 - a n-2 for n = 2,3,... (2) a 0 =3, a 1 = 5. => a 2 = ? and a 3 = ? Many problems can be modeled(or expressed) more naturally by recurrence relations.

Discrete Mathematics advanced counting techniques Transparency No. 5-5 recurrence relation example Ex3: deposit $10,000 in an account with 11% compound interest annually. ==> How much will be in the account after 30 years ? Sol: let P n = amount of the account after n years ==> 1. p 0 = 10, p n = p n-1 + p n-1 x ==> P n = 1.11 P n-1 = P n-2 =...= 1.11 n P 0 = 1.11 n x ==> P 30 = x = 22,892,297. Ex4: Rabbits and Fibonacci number. A pair of rabbits produce one pair per month after 2 month old. Initially there is only one pair. => How many pairs of rabbits are there after n months? sol: f n = #pairs after n months. =>1. f 0 = f 1 = 1 --initial case 2. f n = f n-1 + f n-2 for n > 1, where f n-1 : #old-rabbits and f n-2 is #new born rabbits.

Discrete Mathematics advanced counting techniques Transparency No. 5-6 More recurrence examples (The tower of Hanoi) 3 pegs: A,B,C. Initially n disks placed on peg A. Disks can be moved from peg to peg only if the disk is on top of the original disk and smaller than that on top of the target peg.The goal is to have all disks moved to the 2nd pegs. Let H n = #moves required to solve the problems with n disks => H n = ? Sol: 1. H 1 = H n = H n H n-1 = 2H n A--n-->B A--n-1-->C A--1-->B C--n-1-->B. Hence H n = 2 H n = 2 2 H n =... = 2 n-1 H n = 2 n -1. How big is H 64 ? : assume one move takes 1 sec => = 1.8 x = 500 billion years!!

Discrete Mathematics advanced counting techniques Transparency No. 5-7 more recurrence examples Ex6: #bit strings of length n not containing two consecutive 0s. sol: a n = #bit strings of such kind of length n. ==> 1. a 0 = 1, a 1 = a n = #bits string ending with 1 + #bit-strings-ending with 10 = a n-1 + a n-2. Ex7: a n = # n-digit-strings containing an even number of 0. Sol: valid strings must be in one of the forms: 1. x 1 x 2... x n-1 x n with x n = 1..9 or 2. x 1 x 2... x n-1 0 with x 1 x 2... x n-1 containing odd number of 0. ==> 1. a 0 = 1, a 1 = a n = a n-1 x 9 + (10 n-1 - a n-1 ) for n > 1.

Discrete Mathematics advanced counting techniques Transparency No Solving recurrence relations Linear homogeneous recurrence relation of degree k with constant coefficients: any recurrence of the form: a n = c 1 a n-1 + c 2 a n c k a n-k, where k > 0 and every c j is a constant. Theorem: The recurrence relation: a n = c 1 a n-1 + c 2 a n c k a n-k, and initial conditions: a 0 = t 0,..., a k-1 = t k-1 uniquely determine a sequence (satisfying both conditions) Pf: 1. Existence: trivial. (by Math Ind) 2. Uniqueness: If {  n } and {  n } satisfy the relation => {  n } = {  n } (i.e.,  n =  n for all n.) simple math. ind. Left as an exercise.

Discrete Mathematics advanced counting techniques Transparency No. 5-9 Solving linear homogeneous recurrence relations with constant coefficients a n = c 1 a n-1 + c 2 a n c k a n-k (1) : a recurrence relation => a n = r n is a solution of (1) iff r k = c 1 r k-1 + c 2 r k c k r k-k (2) (2) is called the characteristic equation of (1) a n = c 1 a n-1 + c 2 a n (3) Theorem 1: If r 2 = c 1 r + c 2 has two distinct roots r 1, r 2 then sequence {a n } is a solu of (3) iff a n = d 1 r 1 n + d 2 r 2 n for n = 0,1,2,.. (*) where d 1 and d 2 are constants. pf: <= : simple substitution. =>: Let b 0,b 1,... be any solution of (3). ==> For {b n } to satisfy (*) ==> b 0 = d 1 + d 2 ; b 1 = d 1 r 1 + d 2 r 2 ==> (by ind.) for any k > 1: b k = c 1 b k-1 + c 2 b k-2 = (by ind.hyp.) c 1 (d 1 r 1 k-1 +d 2 r 2 k-1 ) + c 2 (d 1 r 1 k-2 +d 2 r 2 k-2 ) = d 1 (c 1 r 1 k-1 +c 2 r 1 k-1 ) +d 2 (c 1 r 2 k-2 +c 2 r 2 k-2 ) = (by *) d 1 r 1 k + d 2 r 2 k. QED

Discrete Mathematics advanced counting techniques Transparency No Examples: Ex3: Find solu of a n = a n a n-2 for n > 1 and a 0 = 2 and a 1 = 7. sol: char equ: r 2 = r +2 has roots 2, -1. Hence a n = d 1 2 n + d 2 (-1) n for all n for some d 1, d 2. ==> a 0 = d 1 + d 2 a 1 = 2 d 1 - d 2 => d 1 = 3; d 2 = -1 ==> a n = 3 x 2 n - (-1) n for n  0. Ex4: find solu of the Fibonacci sequence: f 1 =f 0 =1 and f n = f n-1 + f n-2 for n > 1. Sol: The char equ: r 2 = r + 1 has roots:  =(1+rt(5)) /2,  =(1- rt(5))/2. note: 1<  <2 and -1 <  <0. Hence fn = d 1  n + d 2  n with f 0 = 1 = d 1 + d 2 and f 1 = 1 = d 1  + d 2  => d 1 = rt(5)/5 and d 2 = -rt(5)/5. => f n = d 1  n + d 2  n = O(  n ) grows exponentially.

Discrete Mathematics advanced counting techniques Transparency No Solving recurrence relation Theorem 2: If r 2 = c 1 r + c 2 has only one r 0 then any sequence {a n } is a solu of (3) iff a n = d 1 r 0 n + d 2 n r 0 n for n = 0,1,2,.. (*) where d 1 and d 2 are any constants. Pf: => : Similar to Theorem 1. <= : a n-1 = d 1 r 0 n-1 + d 2 (n-1) r 0 n-1 a n-2 = d 1 r 0 n-2 + d 2 (n-2) r 0 n-2 ==> c 1 a n-1 + c 2 a n-2 = d 1 (c 1 r 0 n-1 + c 2 r 0 n-2 ) + d 2 (c 1 (n-1) r 0 n-1 + c 2 (n-2) r 0 n-2 ) = d 1 r 0 n-2 (c 1 r o + c 2 ) + d 2 r 0 n-2 ((n-1)(c 1 r o +c 2 ) -c 2 ) = d 1 r 0 n + d 2 r 0 n-2 ((n-1) r r 0 2 ) -- since r 2 - c 1 r -c 2 = (r-r 0 ) 2. Ex5: find solu. of a n = 6a n a n-2 with a 0 = 1 and a 1 = 6. sol: the char equ has one root 3. => a n = d 1 3 n + d 2 n 3 n. => d 1 = 1 and 3d d 2 = 6 => a n = 3 n + n 3 n. for n >= 0.

Discrete Mathematics advanced counting techniques Transparency No Generalization Theorem 3': If the equ. r k = c 1 r k-1 + c 2 r k c k has solutions r 1 m1, r 2 m2,..,r s ms (with m m s = k) where m i is the multiplicity of the root r i. (i.e. r k - c 1 r k c k =  i=1..s (x-r i ) mi ) then {a n } is a solution of the recurrence relation: a n = c 1 a n c k a n-k iff a n =  i=1,s (  j=0,m i -1 d ij n j r i n ), where d ij 's are constants. Ex: The recurrence relation a n = 5 a n a n-2 -7 a n a n-4 has char. equ. r 4 = 5r 3 -9r 2 -7r + 2, which is equ. to (r-2)(r-1) 3 = 0. Hence r has roots: 2, 1 3. Then the relation has general solu: a n = d 0 2 n + e 0 1 n + e 1 n1 n + e 2 n 2 1 n. where d 0, e 0,e 1 and e 2 are constants determined by initial conditions.

Discrete Mathematics advanced counting techniques Transparency No simultaneous recurrence relations Ex24: Solving the simultaneous recurrence relations: 1. a n = 3 a n b n-1 2. b n = a n b n-1, with a 0 = 1 and b 0 = 2. sol: (1,2) can be represented in matrix form: Let Y be the eigenvector of A: (I.e., AY =  Y for some.) => (A- I)Y = 0 => det(A- I) = 0 => (3- )(2- ) - 2 = 0 => =1,4. => Y 1 = (1,-1) T and Y 2 = (2,1) T. Now assume X 0 = d 1 Y 1 + d 2 Y 2 => X n = A X n-1 =... = A n X o = A n-1 (AX 0 ) = A n-1 (d 1 AY 1 + d 2 AY 2 ) = A n-1 (d 1 1 Y 1 + d 2 2 Y 2 ) =... = d 1 1 n Y 1 + d 2 2 n Y 2.

Discrete Mathematics advanced counting techniques Transparency No Divide-and-conquer relations f(n): resources (time or space) needed to solve a problem of size n. Then f(n) = a f(n/b) + g(n) : where a : the number of subproblems n/b : size of each subproblem g(n) : cost for splitting problem and combining solutions Problem: How to estimate the size of f(n) ? f(n) = a f(n/b) +g(n) = a 2 f(n/b 2 ) + a g(n/b) + g(n) =.... = a k f(n/b k ) +  j=0,k-1 a j g(n/b j ). Hence if n = b k ==> f(n) = a k f(1) +  j=0,k-1 a j g(n/b j ) (1) Theorem 1: If f(n) = a f(n/b) + c, where a  1, b > 1 and c > 0, is an increasing function, then f(n) = O(n  (log b a)) if a > 1 and = O(log n) if a = 1.

Discrete Mathematics advanced counting techniques Transparency No proof: Pf: If n = b k, by (1), f(n) = a k f(1) +  j=0,k-1 a j c. If a = 1 ==> f(n) = f(1) +  j=0,k-1 c = f(1) + ck = O(log n). If a > 1 ==> f(n) = a k f(1) +  j=0,k-1 a j c = a k f(1) + c(a k -1) /(a-1) = a k [f(1) + c/(a-1)] - c/(a-1) = c 1 a lg b n + c 2 = c 1 n log b a + c 2 = O(n log b a ). If b k < n < b k+1 is not a power of b. ==> f(n) < f(b k+1 ) = c 1 a k+1 + c 2 < c 1 a a log b n + c 2 = O(n log b a ). Ex: If f(n) = 5 f(n/2) + 3. => a = 5 > 1; c = 3; b = 2. ==> f(n) = O(n log b a ) = O(n lg 5 ) If f(n) = 2 f(n/2) + 2 ==> f(n) = O(n lg 2 ) = O(n) is linear.

Discrete Mathematics advanced counting techniques Transparency No divede-and-conquer relations(cont'd) Theorem 2: If f(n) = a f(n/b) + cn d, where a  1, b > 1 and c,d > 0, is increasing, then f(n) = O(n d ) if a < b d = O(n d lg n) if a = b d = O(n log b a ) if a > b d. Pf: f(b k ) = a f(b k-1 ) + cb kd = a 2 f(b k-2 ) + c a b d(k-1) + c b dk. =.... = a k f(1) +  j=0,k-1 cb kd a j /b dj = f(1) a k + cb kd (1-(a/b d ) k )/(1 - (a/b d )). case 1: n = b k. (k = log b n) Hnece if a f(n) = f(1) a k + cb kd (1-(a/b d ) k )/(1 - (a/b d ))  O(b kd ) = O(b d log b n ) = O(n d ). If a = b d =>f(n) = f(1) a k + ckb kd = O(kb kd ) = O(n d lg n). If a > b d => f(n) = f(1) a k + cb kd (1-(a/b d ) k )/(1 - (a/b d )).  f(1) a k + cb kd (a/b d ) k = O(a k ) = (n log b a ).

Discrete Mathematics advanced counting techniques Transparency No The master theorem: case 2: b k < n < b k+1 : => f(b k )  f(n)  f(b k+1 ). But O(f(b k )) = O(f(b k+1 )), hence O(f(n)) =O(f(b k )). QED Ex8:Fast integer multiplication: A = (a 2n-1 a 2n-2... a 1 a 0 ) B = (b 2n-1 b 2n-2... b 1 b 0 ) A H = (a 2n-1...a n ); A L = (a n-1,...,a 0 ); B H =..., B L =... => A x B = (2 n A H + A L ) x (2 n B H + B L ) = 2 2n A H B H + 2 n (A H B L + A L B H ) + (A L B L ) => f(n) = 4f(n/2) + O(n) => f(n) = O(n  lg 4) = O(n 2 ) - no improving !! But AXB = 2 2n A H B H + 2 n (A H B L + A L B H ) + (A L B L ) = 2 2n A H B H + 2 n( (A H +A L )(B H +B L )) - A H B H - A L B L ) + (A L B L ) ==> f(n) = 3f(n/2) + O(n) => f(n) = O(n lg 3 ) < O(n 2 ).

Discrete Mathematics advanced counting techniques Transparency No More examples: Fast matrix multiplication:(Ex4 & 9) one nxn matrix multiplication can be divided into 7 (n/2)x(n/2) multiplications + 15 (n/2)x(n/2) additions. => f(n) = 7 f(n/2) + 15n 2 /4 => a = 7, b = 2, d = 2. => a > b d = 4. => f(n) = O(n lg 7 ) = O(n 2.81 ). Better than direct multiplication(=O(n 3 )) !! Although even better result (O(n )) is possible.

Discrete Mathematics advanced counting techniques Transparency No Inclusion-Exclusion principle The principle: A, B : two finite sets => |AUB| = |A| + |B| - |A  B|. Ex2: #positive integers < 1000 and dividable by 7 or 11 = ? sol: let A ={x | x < 1000 and 7 | x} B = {x | x < 1000 and 11 |x}. => |A  B| ={x | x < 1000 and 77 | x} => |AUB| = |A| + |B| - |A  B| =[1000/7]+[1000/11]-[1000/77] = 220. Problem: |AUBUC| = ? |A| +|B|+|C| -|A  B|-|A  C||B  C|+|A  B  C| |A|+|B|+|C| |A|+|B|+|C| -|A  B|-|A  C|-|B  C|

Discrete Mathematics advanced counting techniques Transparency No The general inclusion-exclusion principle Theorem 1: |A 1 U A 2...UA n | =  1  i  n |A i | -  1  i<j  n |A i  A j | +  1  i<j<k  n |A i  A j  A k | (-1) n |A 1  A 2 ...  A n | ---(*) pf: Let a be any element belonging to exactly A d1, A d2,...,A dr. ==> It is counted C(r,s) times by the sum:  0<j1<j2<...<js< n+1 |A j1  A j2 ...  A js |. (note: C(r,s) = 0 if r < s). ==> # a counted by (*) = C(r,1) - C(r,2) (-1) n C(r,n) = C(r,1) - C(r,2) (-1) r C(r,r) = - (1-1) r + C(r,0) = 1

Discrete Mathematics advanced counting techniques Transparency No Applications of the IE principle Alternative form: Ai : set of elements having property Pi. N(Pj 1,...,Pj k ) = #elements with properties Pj 1,...,Pj k. (i.e. N(Pj 1,...,Pj k ) = |Aj 1 ...  Aj k |. ) ~Pj : the negation of property Pj. N(~Pj 1,...,~Pj k )) = #elements without any property of Pj 1,..,pj k. = N(~(Pj 1 \/...\/Pj k )) = |U| - |Aj 1 ..  Aj k |. => N(~P 1,...~P n )) = N -  N(Pi) +  N(PiPj) -  N(PiPjPk) (-1) n N(P1P2...Pn). => |~(A1UA2...UAn)| = |U| -  |Ai| +  |Ai  Aj| (-1) n |A1 ...  An|.

Discrete Mathematics advanced counting techniques Transparency No Examples: Ex1: x1+x2+x3 = 11, 0  x1  3, 0  x2  4, 0  x3  6. ==> #integer solutions = ? Sol: let P1 = "x1 > 3"; P2 = "x2 > 4"; P3 = "x3 > 6". =>#sol = N(~p1/\~p2/\~p3) = N(~(P1\/P2\/P3)) = N - N(p1) - N(P2) - N(P3) +N(p1/\P2) +N(P1/\P3) + N(P2/\P3) - N(N(p1/\P2/\P3) => N = C(11+2, 2); N(p1) = #sol with x1 > 3 = #sol of "x1' + x2 + x3 = 7" = C(9,2) N(p2) = C(8,2); N(P3) = C(6,2). N(p1/\P2) = #slo with X1> 3 and X2 > 4 = #sol of "x1'+x2'+X3 = 2" = C(4,2) = 6. N(P1/\P3) = C(0+2,2) = 0; N(P2/\P3) = 0. N(P1/\P2/\P3) = 0. => #sol = = 6.

Discrete Mathematics advanced counting techniques Transparency No More example Ex2: #positive integers < 101 and dividable by 4, 5 or 6. A = {x : 4|x}; B = {x: 5|x }; C= {x: 6|x}. => |A| +|B|+|C| = [100/4] + [100/5] + [100/6] = =61. |A  B|+|A  C|+|B  C| = [100/20] +[100/12] + [100/30] =16 |A  B  C| = [100/60] = 1. => #sol = =46. Ex3': |A|= m, |B| = n, m  n. #onto function f:A -> B = ? Let B = {b 1,...,b n } and Pi = "b i is not in the range of the fun" => N(Pi) = |Ci|= |{f | f:A->B and bi not in f(A)}|. #ontos = N(~P1/\~P2.../\~Pn) = N(~(P1\/...\/Pn)) = |U| - |C1 U C2 U... U Cn| = n m -  |Ci| +  |Ci  Cj| (-1)n |C1  C2 ...  Cn| = n m - C(n,1)(n-1) m + C(n,2)(n-2) m (-1) n-1 C(n,n-1) 1 m.

Discrete Mathematics advanced counting techniques Transparency No more examples (cont'd) Ex3: #ways to assign 5 jobs to 4 people s.t. each one is assigned at least one job. sol: m =5, n = 4. #ways = 4 5 -C(4,1)3 5 + C(4,2)2 5 +C(4,3)1 5 = = 240. Ex4: [The hatcheck problem:] n hats randomly returned to the customers. => What is the probability that no one receives his own hat?

Discrete Mathematics advanced counting techniques Transparency No Derangements x 1,x 2,...,x n : a list Any permutation a 1,...,a n of {x 1,...,X n } s.t. a j  x j for all j = 1..n is called a derangement of the list. Ex: 1,2,3,4,5 has derangement but not Let D n = # derangements of n objects. => D 1 = 0, D 2 = 1, D 3 = 2. (123 has derangements: 231, 212) Theorem: D n = n![1- 1/1! + 1/2! (-1) n /n!] Pf: let P i = def "a=a 1...a n is a perm of x 1...x n s.t. a i = x i." => D n = N(~P 1 /\~P 2 /\../\~P n ) = N(~(P 1 \/P 2 \/..\/P n )) = N -  N(P i ) +  N(P i /\P j ) (-1) n N(p 1 /\../\P n ) = n! -C(n,1)(n-1)! +C(n,2)(n-2)! (-1) n C(n,n)(n-n)! = n![1 - 1/1! + 1/2!.. ]. => solu of the hatcheck problem = D n / P(n,n) = D n /n! in [0.3,0.5] = (1 -1/1! + 1/2!-...) -> e -1 = as n -> .