Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Combinatorics

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Introduction –Combinatorics is the study of the arrangement of discrete objects –Who Cares? Counting is important whenever we have finite resources Examples: –Words of memory needed to store … –Instructions needed to solve … –Number of hits per second supported by … –A password must be between six and eight characters long. The characters can be a digit or a letter (case sensitive). Each password must include at least one digit. How many passwords can we support?

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Useful Counting Cheat Sheet –Summation Notation –Summation Properties –Summation Formulas

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Basic Counting Principles (1) –Product Rule If you have two sets, A & B, |A  B| = |A| * |B| Example: How many different bit strings are there of length seven? –Answer: 2 7 Extends (by induction) to sequence of n sets –Sum Rule If you have two disjoint sets, A & B, |A  B| = |A| + |B| Example: How many ways are there to elect an ACM president if the president must be a CS faculty member (from 8) or a CS major (from 250)? –Answer: 258 Extends (by induction) to n mutually exclusive sets

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Basic Counting Principles (2) –Inclusion-Exclusion Rule If you have two arbitrary sets, A & B, |A  B| = |A| + |B| - |A  B| Example: How many bit strings of length eight start with a 1 or end with the two bits 00? –Answer: – 2 5 = 160 Subsumes the Sum Rule Extends (by induction) to n arbitrary sets (see p. 205 of text) Consider Venn Diagram

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Basic Counting Principles (3) –Extended Example: Internet Address (IPv4) Sample: (data.cs.uwec.edu) Translation: 32-bits Interpretation Restrictions – not available as netid of Class A network –All 0 s or all 1 s are not allowed as any hostid How many possible computer Internet addresses? –Note, netid for Class A is 7 bits long, netid for Class B is 14 bits long, and netid for Class C is 21 bits long if you can’t tell from the above picture. Plus some additional classes

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Basic Counting Principles (3) How many possible computer Internet addresses? Class A: ((2 7 – 1) * (2 24 – 2)) prohibited as netid Class B: (2 14 * (2 16 – 2)) + all 0’s and all 1’s prohibited Class C: (2 21 * (2 8 – 2)) = 3,737,091,842.

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Counting –E.g., there are just over 4 billion people in Utopia and the area of Utopia is slightly more than 16,000 square miles. 1. How many people are there per square mile? 2 2 * 2 30 = 2 32 people 2 4 * 2 10 = 2 14 sq. miles  2 32 / 2 14 = 2 18 people per sq. mile 2. If each square mile is divided into 16 subnets, and everyone has a computer on the subnet, how many bits are necessary to identify a computer in a particular subnet? # people per subnet = 2 18 / 2 4 = 2 14  # bits = 14

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Permutations –an ordered arrangement of n elements from a set –an r-permutation is an ordered arrangement of r elements taken from a set of n elements –the number of r-permutations of n items ( n P r ) n P r = n! / (n - r)! –Example: How many ways are there to select a 1 st, 2 nd, and 3 rd place winner from 100 different people? Answer: 100 P 3 = 970,200 –Notice: n P n = n! n P 0 = 1 n P 1 = n

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Combinations –an unordered arrangement of n elements from a set unordered means order does not matter! –an r-combination is an unordered arrangement of r elements taken from a set of n elements –the number of r-combinations of n items ( n C r ) n C r = n! / r!(n - r)! = n P r / r! –Example: Quality Control wants to test 25 microprocessor chips from the 300 manufacturers. In how many ways can this be done? Answer: 300 C 25 –Notice: n C n = 1 n C 0 = 1 n C 1 = n n C r = n C n-r

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Repetition (1) –Thus far we have assumed that items can not be used twice in one draw –What if they can? Letters in a password Selecting from bag of items with copies –Permutations with repetition n PR r = n r –Combinations with repetition n CR r = n+r-1 C r –Permutations with indistinguishable objects n PI n1,n2,…,nk = n! / (n 1 !*n 2 !*…*n k !) Same as the number of ways n distinguishable objects can be distributed into k distinguishable buckets so that n i objects are in bucket i page 240 of text

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Repetition (2) –Example: Permutations with indistinguishable objects n PI n1,n2,…,nk = n! / (n 1 !*n 2 !*…*n k !) How many different strings can be made by reordering the letters of the word “SUCCESS”? We have three Ss, two Cs, one U, and one E. Notice that the three Ss can be placed into the seven positions in C(7,3) ways, leaving four blank positions The Cs can be placed into those four positions C(4,2) ways, leaving two free positions. The U can be placed into those two positions in C(2,1) ways, leaving one free position (allowing E to be placed in C(1,1) ways). From the product rule: C(7,3) * C(4,2) * C(2,1) * C(1,1) which equals (7! * 4! * 2! * 1!) / (3! * 4! * 2! * 2! * 1! * 1! * 1! * 0!) which equals 7! / (3! * 2! * 1! * 1!) which equals 420

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Repetition (3) –Extended Example: How many ways are there to build four hands of 5 cards from the standard deck of 52 cards? Solution I: Combinations of Objects –Use Combinations for each player’s hand –Use Product Rule for total 52 C 5 * 47 C 5 * 42 C 5 * 37 C 5 = 52! / (5!5!5!5!32!) Solution II: Buckets –n distinguishable objects (52 cards) –k distinguishable buckets (4 hands + 1 undealt pile) –5 cards in each hand, 32 cards in undealt pile 52! / (5!5!5!5!32!) Solution III: Permutations of Indistinguishable Objects –5 indistinguishable objects of each of four types (one for each hand) –32 indistinguishable objects of a fifth type (undealt) 52!/(5!5!5!5!32!)

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Probability (1) –An area of mathematics that has its origin in counting Developed to study gambling games –circa 1650 by Pascal –generalized circa 1812 by Laplace –Finite Probability experiment – a procedure that yields one of a given finite set of possible outcomes sample space – the finite set of possible outcomes event – a subset of the sample space The probability of an event E, which is a subset of a finite sample space S of equally likely outcomes, is p(E) = |E| / |S|

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Probability (2) –Example: What is the probability of picking a winning combination of six numbers out of 40 (no repeats)? S : set of all six-number sequences taken from 40 E : the target number |S| = 40 C 6 = 3,838,380 |E| = 1  p(E) = 1/3,838,380 (approx ) –Example: What is the probability that a hand of five cards in poker contains four of a kind? S : set of all possible poker hands E : set of all possible four-of-a-kind hands |S| = 52 C 5 = 2,598,960 |E| = nbr of ways to pick rank * nbr of ways to pick other card = 13 C 1 * 48 C 1 = 624  p(E) = 624 / 2,598,960 (approx ) product rule

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Probability (3) –Properties 0  p(E)  1 p(S) = 1 p(E) = 1 – p(E) p(E 1  E 2 ) = p(E 1 ) + p(E 2 ) – p(E 1  E 2 ) –disjoint events »p(E 1  E 2 ) = 0 »p(E 1  E 2 ) = p(E 1 ) + p(E 2 ) p(E 1  E 2 ) = p(E 1 ) * p(E 2 | E 1 ) –independent events »p(E 2 |E 1 ) = p(E 2 ) »p(E 1  E2) = p(E 1 ) * p(E 2 ) Each outcome “equally likely” implies p(a) = 1/n where a  S and |S| = n. called uniform distribution

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Application: Expert Systems (1) –Recall expert systems use (a  (a  b))  b –Too simplistic a may or may not be known with certainty a  b may or may not be known with certainty –Alternative Approach Based on Probability Bayes’ Theorem or Creates a network of related probabilities called a Bayesian Propagation Network

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Application: Expert Systems (2) –Concrete Example: p(D) = p(Rob has cold) = 0.2 p(S|D) = p(Rob sneezed | Rob has cold) = 0.75 p(S|D) = p(Rob sneezed | Rob does not have cold) = 0.2 p(S) = p(S|D)*p(D) + p(S|D)*p(D ) = 0.31  P(D|S) = (0.75*0.2) / 0.31 =  P(D|S) = ((1 – 0.75)*0.2) / (1 – 0.31) = Thus knowledge that Rob sneezed increased p(cold) by a factor of almost 2.5 Thus knowledge that Rob did not sneeze decreased p(cold) by a factor of almost 3.

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Application: Expert Systems (3) –Previous example assumed one hypothesis and one piece of evidence –Must generalize to n pieces of evidence and m hypotheses –Generalized Bayes’ Theorem or assumes that symptoms are conditionally independent given some decease –Frequently not the case!

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Application: Expert Systems (4) –Example S 1 : patient sneezed S 2 : patient coughs –Suppose we observe S 1 p(D 2 |S 1 ) = 0.53, p(D 3 |S 1 ) = 0.06 –Now we observe S 2 p(D 1 |S 1 S 2 ) = 0.33, p(D 2 |S 1 S 2 ) = 0.67, p(D 3 |S 1 S 2 ) = 0.0 i = 1 (cold)i = 2 (allergy)i = 3 (light sens.) p(D i ) p(S 1 |D i ) p(S 2 |D i )

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Application: Monte Carlo Techniques –deterministic algorithms don’t always exist or aren’t always practical –Example: Value of  Let P = set of randomly selected points –(x,y) | 0  x  1  0  y  1 Let S = { (x,y) | (x,y)  P  } Let a = | S | / | P |   4a –Example: Genetic Algorithms motivated by genetic reproduction primary operations crossover, mutate, reproduce Let v(c) represents “value” of chromosome Let v(m) be maximum such value of a population each c survives during reproduction with p = v(c)/v(m)