CSE 221: Algorithms and Data Structures Lecture #10 Counting: Easy as 1, 2, C(3,1) Steve Wolfman 2010W2

Learning Goals After this unit, you should be able to: Apply counting principles to determine the number of arrangements or orderings of discrete objects, with or without repetition, and given various constraints. Use appropriate mathematical constructs to express a counting problem (e.g. counting passwords with various restrictions placed on the characters within). Identify problems that can be expressed and solved as a combination of smaller sub problems. When necessary, use decision trees to model more complex counting problems.

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

In CS, we often encounter situations where we want to count the number of possible outcomes of an event. For example, we may wish to determine: the number of possible paths to follow in a directed network (graph), the number of possible 5-8 character passwords, etc.

Describe the problem formally. Suppose that the customers of a bank are asked to use 3-digit PINs to protect their accounts when using the bank’s ATM machines. If the other constraints are: no 2 digits can be the same, and the only allowable digits are 1, 2, 3, & 4, how many PINs are possible? Describe the problem formally. We use as much mathematical notation as needed for a clear formalism. U = {1, 2, 3, 4} PIN = x1x2x3 where each xi ∈ U, and for each distinct pair of indexes i, j, xi != xj.

(b) Model the problem using a tree diagram.   . 1 2 3 4 234 134 124 123

Multiplication Principle (Product Rule) If an operation consists of t steps and: Step 1 can be performed in n1 ways, Step 2 can be performed in n2 ways, … Step t can be performed in nt ways, then the entire operation can be performed in n1n2…nt different ways. Example: How many postal codes begin with the letter V and end with the digit 4? __ __ __ __ __ __ One choice for step 1. 10 for step 2. 26 for step 3. 10 for step 4. 26 for step 5. 1 for step 6. 10^2*26^2

Example: Use the multiplication principle to prove that the number of subsets of a set containing n elements is 2n. For each of n elements, choose independently whether to include it or not.

Example: Suppose a programming language requires you to define variable names (identifiers) using exactly 3 different upper case characters. How many different identifiers contain either an A or a B, but not both? The identifier definitely contains exactly one A or B. So, let’s decide which. That’s a choice of: 2. Now, where should that letter go, first, second, or third. That’s a choice of: 3. Now, what letters should go in the remaining two blanks. 24 options for the left-hand one and 23 for the right hand one. That’s a choice of: 24*23. Overall: 2*3*24*23. OLD NOTES THAT DIDN’T NOTICE THE “DISTINCT” PART: It’s hard to see how to do this with the mult principle alone. There’s lots of approaches we could TRY. Here’s a simple one: There are 25 legit letters (A or B and C-Z) and three blanks. That’s 25^3. However, we aren’t allowed to use an identifier with NO A/B. So, we can eliminate all the identifiers with NO A/B: 24^3. 25^3 – 24^3 = 1801 Finally, we double that (because we could use either A or B): 3602.

Example: Let X be a set with n elements Example: Let X be a set with n elements. How many ordered pairs (A, B) satisfy all of these constraints: A ⊆ X, B ⊆ X, and A ∩ B = ∅? How many As and Bs are there such that A and B don’t overlap? Essentially, we’re choosing for each element whether to include it in A, to include it in B, or not to include it in either: 3^n

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

Addition Principle (Sum Rule) If X1, X2, …, Xt are pairwise disjoint sets (i.e., Xi ∩ Xj = ∅ ∀i ∀j s.t. i, j∈{1, 2, …, t }, i ≠ j), then the number of ways to select an element from any of X1 or X2 or … or Xt is: | X1 | + | X2 | + … + | Xt |

Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters. How many different passwords are possible? Emphasize mapping to addition principle. Sets are: set of 6-letter passwords, set of 7-letter passwords, and set of 8-letter passwords. Call it 62 possibilities. So, 62^6 + 62^7 + 62^8.

Back-of-the-envelope calculation Back-of-the-envelope calculation. For the previous example, roughly how long would it take for a program to perform a brute-force attack to discover a user’s password, if we assume (guess?) that a million passwords can be tested each second? Best case? Average case? Worst case? 62 is about 64, which is 2^6. (2^6)^6 = 2^36. So, we have 2^36 + 2^42 + 2^48. The smaller terms really don’t matter. So, 2^48. One million is about 2^20. So, that’s 2^28 millions (or about 250 million millions). Best case: one-millionth of a second. Average case: about half or 125 million seconds. There are about 30 million seconds in a year; so, about 4 years. Worst case: about 8 years. Side thought: Talking about how small a search space is when dealing with simple passwords. So, say you knew users didn’t use any caps. That cuts down each letter from about 64 to about 32. 32 = 2^5. So, we have (2^5)^6 + (2^5)^7 + (2^5)^8 = 2^30 + 2^35 + 2^40, which is about 2^40. That’s about one million million. So, it would take about 500,000 seconds (half of one million) on average to crack the password, which is about 1/60th of a year or a bit less than a week. Since this is a highly parallelizable task, we should be VERY concerned about that result! Throw 10 computers at the problem and you have the password in well under a day!

How many possible combinations are there this time? Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters, but this time, at least 1 of the chars. has to be a number. How many possible combinations are there this time? Subtract out only letter combos. (52^6 + 52^7 + 52^8)

Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters, but this time, at least 1 of the chars. has to be a number. How many possible combinations are there if we know users are “minimally complex” in their password choices? Subtract out only letter combos. (52^6 + 52^7 + 52^8)

Now, will the passwords be more secure or less secure? Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters, but this time, at least 1 of the chars. has to be a number. Now, will the passwords be more secure or less secure? Subtract out only letter combos. (52^6 + 52^7 + 52^8)

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

Inclusion-Exclusion Principle If an object can be found in 2 or more sets at the same time, then we cannot use the addition principle. Why not? If A and B are sets, then the total number of elements in either A or B is given by: | A ∪ B | = | A | + | B | – | A ∩ B | Example using a Venn Diagram:

Example: How many 8-bit strings either start with “1” or end with “00”? [1 + 7(0/1)] or [6(0/1) + 00] but don’t double-count [1 + 5(0/1) + 00]: 2^7 + 2^6 – 2^5

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

Permutations & Combinations How many different outcomes are there if you choose r balls from a barrel containing n different balls?

r-permutations r-combinations r-permutations w/ rep. Example: r = 3 Barrel = {A,B,C,D} Order matters ABC ≠ CAB Order doesn’t ABC = CAB Repetition not OK e.g., AAB is not counted ABC, ABD, ACB, ACD, ADB, ADC, BAC, BAD, BCA, BCD, BDA, BDC, CAB, CAD, CBA, CBD, CDA, CDB, DAB, DAC, DBA, DBC, DCA, DCB r-permutations ABC, ABD, ACD, BCD r-combinations Repetition OK e.g., AAB is counted AAA, AAB, AAC, AAD, ABA, ABB, ABC, ABD, … r-permutations w/ rep. AAA, AAB, AAC, AAD, ABB, ABC, ABD, ACC, ACD, ADD, BBB, BBC, BBD, BCC, BCD, BDD, CCC, CCD, CDD, DDD r-combinations w/ rep. How many different outcomes are there if you choose r balls from a barrel containing n different balls?

(1) r-Permutations (Order matters, and repetition is not allowed.) An r-permutation of n distinct elements x1, x2, …, xn is an ordering of an r-element subset of {x1, x2, …, xn}. The number of r-permutations is: P(n, r) = n(r) = Derivation:

Example 1: In how many ways can 5 electoral candidates finish in an election, assuming no ties? Example 2: In how many ways can 7 girls and 3 boys line up, if the boys must stand next to each other? Ex 2 is really about ordering 8 “things” and then choosing an order for the boys inside the one “big thing”. 8!*3!

Example 3: Suppose an operating system has a queue of 3 low priority and 5 high priority processes ready to run. In how many ways could these processes be ordered for execution if 2 low priority processes are not allowed to be executed back to back? 5! To get HHHHH Then _H_H_H_H_H_ 6p3 for the 3 spots where we put the Ls. TODO: fix error below (7C2 is wrong) 5!3! (8c3 – 7C2 – 6C1) 8*7 – 7*3 – 6 = 56 – 21 – 6 = 29

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

(2) r-Combinations (Order doesn’t matter, and repetition is not allowed.) An r-combination of n distinct elements x1, x2, …, xn is an r-element subset of {x1, x2, …, xn}. The number of r-combinations is: C(n, r) = ( ) = Example 4: A donut shop has 10 kinds of donuts. In how many ways can 6 distinct kinds of donuts be selected? Example 5: Show how to derive a relationship between r- permutations and r-combinations. Ex 5: there are r! ways to reorder an r-combination. So, nCr is nPr/r!, which is correct!

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

(3) r-Permutations with Repetition (Generalized r-Permutations) Here, order matters, and repetition is allowed. Suppose we have a set of n distinct elements x1, x2, …, xn and we select a sequence of r elements, allowing repeats. How many different sequences are possible?

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

(4) r-Combinations with Repetition (Generalized r-Combinations) Here, order doesn’t matter, and repetition is allowed. Suppose we have a set of n distinct elements x1, x2, …, xn. The number of unordered r-element selections from this set, with repetition allowed, is: All that really matters here is how many of each different type of thing you select. And, the number of selections of each type must add up to r. So, one way to think of it is that you have r boxes labelled: Box1 box2 box3 ... Boxn And, you’re putting items into the boxes. Putting an item in a box MAKES IT that type of item (magically): .. . ..... .... Now, we have 3 “item1s”, 1 “item2”, 5 “item3s”, ..., and 4 “itemns” It’s hard to imagine a formula as things stand, but remember that there really are r things across the bottom of that diagram. If we squished them all together, we’d get r dots: ............... But, now we can’t tell which box everything ended up in. So, let’s put in dividers to split up the boxes: ...|.|.....|..||.... That’s: 3 thing1s, 1 thing2, 5 thing3s, 2 thing4s, 0 thing5s, 4 thing6s. In this case, what we’re really asking is how to arrange n-1 dividers among the r items. So, there’s r+n-1 different places where a dot or a divider can fall. N-1 of those places are chosen to be dividers. At that point, we know exactly how many of each type of thing we have. So, # of arrangements is: (r+n-1)c(n-1) Equivalently, we could say: (r+n-1)cr

Example 8: If a donut shop sells 3 kinds of donuts: plain, glazed, and jelly, then how many possible selections of 7 donuts are there? We have 9 slots: _ _ _ _ _ _ _ _ _ For example we could put dividers here: | _ _ _ | _ _ _ _ To have 0 plain, 3 glazed, and 4 jelly. (d + t – 1)c(t – 1) (d + t – 1)c(d)

( ) ( ) Summary of Formulas Order Matters & Repetition is not Allowed Order does not Matter & Repetition is not Allowed Order Matters & Repetition is Allowed Order does not Matter & Repetition is Allowed ( ) n n(r) r ( ) r+n-1 nr n-1

Today’s Outline Multiplication Principle Addition Principle Inclusion/Exclusion Principle Combinations and Permutations r-permutations, no repetition r-combinations, no repetition r-permutation, with repetition r-combinations, with repetition variants

Permutations of Indistinguishable Objects Multinomial Theorem: The number of different permutations of n objects where there are: n1 indistinguishable type 1 objects n2 indistinguishable type 2 objects … nk indistinguishable type k objects and n1 + n2 + … + nk = n is: Proof: n!/(product of ni! for all i) This is really the same as the combinations idea, except that we don’t care about several different types of orderings. So, there are n! orderings if we number the elements of each indistinguishable set, but then we “overcount” the indistinguishable sets by the number of permutations of that set.

Example 9: How many strings can be made by reordering the letters PEPPER? 6!/(3!2!1!) = 6!/(3!2!) = 60

Theorem: The number of ways to place n distinguishable objects into r distinguishable boxes so that ni objects are placed into box i (where i = 1, 2, …, r and n1 + n2 + … + nr = n) is: Example 10: How many ways are there to distribute hands of 5 cards to each of 4 players from a deck of 52 cards? What we’re really doing is taking all the possible permutations and getting rid of the “overcounts” within each box. 52!/(32!5!5!5!5!)

Learning Goals After this unit, you should be able to: Apply counting principles to determine the number of arrangements or orderings of discrete objects, with or without repetition, and given various constraints. Use appropriate mathematical constructs to express a counting problem (e.g. counting passwords with various restrictions placed on the characters within). Identify problems that can be expressed and solved as a combination of smaller sub problems. When necessary, use decision trees to model more complex counting problems.

Coming Up Final Exam! Yay!