1 Algorithms CSCI 235, Fall 2012 Lecture 9 Probability

2 Permutations How many ways can you take k items from a set of n items? Order matters (ab is counted separately from ba) No duplicates allowed (don't count aa) Example: S = {a, b, c, d}, how many ways can we take 2 items? ab ac ad ba bc bd ca cb cd da db dc Total pairs = 12 4 ways to pick first 3 ways to pick second In general: The number of permutations of k items from a set of n is:

3 Combinations Combinations are like permutations, except order doesn't matter. ab and ba are considered the same. ab ac ad bc bdcd Combination of 4 things taken 2 at a time: Each set of k items has k! possible permutations. number of combinations = number of permutations divided by k!

4 k-tuples and k-selections k-tuple (or k-string): Like a permutation, but can have duplicates. aa ab ac ad ba bb bc bd ca cb cc cd da db dc dd n choices for 1st item n choices for 2nd item, etc. Total possible k-tuples for k items taken from a group of n items is n k k-selection: Like combination, but can have duplicates. aa ab ac ad bb bc bd cc cd dd Formula:

5 Sample Spaces A sample space is the set of all possible outcomes for an experiment. Experiment A: flip 3 coins Sample space = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT} # of possibilities: ? Experiment B: Flip a coin until you get heads. Sample space = {H, TH, TTH, TTTH,...} 2323

6 Diagramming sample spaces Experiment A: (flip 3 coins)

7 Diagramming sample spaces Experiment A: (flip 3 coins) H3H1H2H3T3H1H2T3H3H1T2H3T3H1T2T3H3T1H2H3T3T1H2T3H3T1T2H3T3T1T2T3H3H1H2H3T3H1H2T3H3H1T2H3T3H1T2T3H3T1H2H3T3T1H2T3H3T1T2H3T3T1T2T3 T1T1 T2T2 H2H2 T2T2 H2H2 H1H1 Experiment B: (flip until heads) Sample space (the set of all possible outcomes)

8 Diagramming sample spaces Experiment A: (flip 3 coins) H3H1H2H3T3H1H2T3H3H1T2H3T3H1T2T3H3T1H2H3T3T1H2T3H3T1T2H3T3T1T2T3H3H1H2H3T3H1H2T3H3H1T2H3T3H1T2T3H3T1H2H3T3T1H2T3H3T1T2H3T3T1T2T3 T1T1 T2T2 H2H2 T2T2 H2H2 H1H1 Experiment B: (flip until heads) H 1 T 1 H 2 T 1 T 2 H 3 T 1 T 2 T 3 H 4 H 1 H 2 H 3 H 4 T 1 T 2 T 3 T 4... Sample space (the set of all possible outcomes)...

9 Events An event is a subset of the sample space: Experiment A (flip 3 coins): Event A1: First flip is head = {HHH, HHT, HTH, HTT} Event A2: Second flip is tail = {HTH, HTT, TTH, TTT} Event A3: Exactly 2 tails = {HTT, THT, TTH} Event A4: Two consecutive flips are the same= {HHH, HHT, HTT, THH, TTH, TTT} Experiment B (flip until get heads): Event B1: First flip is head = {H} Event B2: First flip is tail = {TH, TTH, TTTH...} Event B3: Even number of flips = {TH, TTTH, TTTTTH...}

10 Definitions If A and B are events in sample space S, then "A and B" is translated "A or B" is translated "not A" is translated Two events are mutually exclusive if

11 Examples 1. Of the four events in Experiment A, which pairs are mutually exclusive? 2. Of the three events in Experiment B, which pairs are mutually exclusive?

12 Recall events An event is a subset of the sample space: Experiment A (flip 3 coins): Event A1: First flip is head = {HHH, HHT, HTH, HTT} Event A2: Second flip is tail = {HTH, HTT, TTH, TTT} Event A3: Exactly 2 tails = {HTT, THT, TTH} Event A4: Two consecutive flips are the same= {HHH, HHT, HTT, THH, TTH, TTT} Experiment B (flip until get heads): Event B1: First flip is head = {H} Event B2: First flip is tail = {TH, TTH, TTTH...} Event B3: Even number of flips = {TH, TTTH, TTTTTH}

13 Probability Distribution A probability distribution Pr{ } on sample space S is any mapping from events of S to [0...1] such that the following axioms hold: 1) Pr{A} >= 0 for any event A 2) Pr{S} = 1 Example: Flip two coins: S= {HH, HT, TH, TT} Event A = {HT}Event B = {TH} What is

14 Some useful theorems Example: Flip two coins. Event A = {HH, HT, TH} Event B = {HT, TH, TT} What is

15 Working with probability trees If events at distinct stages of a probability tree are independent, then the probability of a leaf is the product of the probabilities on the path to the leaf. Experiment A (flip 3 weighted coins): H 1 =1/3, T 1 = 2/3; H 2 =1/4, T 2 =3/4; H 3 =1/5, T 3 = 4/5

16 Working with probability trees If events at distinct stages of a probability tree are independent, then the probability of a leaf is the product of the probabilities on the path to the leaf. H3H1H2H3T3H1H2T3H3H1T2H3T3H1T2T3H3T1H2H3T3T1H2T3H3T1T2H3T3T1T2T3H3H1H2H3T3H1H2T3H3H1T2H3T3H1T2T3H3T1H2H3T3T1H2T3H3T1T2H3T3T1T2T3 T1T1 T2T2 H2H2 T2T2 H2H2 H1H1 Experiment A (flip 3 weighted coins): H 1 =1/3, T 1 = 2/3; H 2 =1/4, T 2 =3/4; H 3 =1/5, T 3 = 4/5 (1/3)(1/4)(1/5) = 1/60 (1/3)(1/4)(4/5) = 4/60 = 1/15 (1/3)(3/4)(1/5) = 3/60 = 1/20 (1/3)(3/4)(4/5) = 12/60 = 1/5 (2/3)(1/4)(1/5) = 2/60 = 1/30 (2/3)(1/4)(4/5) = 8/60 = 2/15 (2/3)(3/4)(1/5) = 6/60 = 1/10 (2/3)(3/4)(4/5) = 24/60 = 2/5

17 Another example Experiment B (flip a weighted coin until heads): H i = 1/3, T i = 2/3 H 1 T 1 H 2 T 1 T 2 H 3 T 1 T 2 T 3 H 4 H 1 H 2 H 3 H 4 T 1 T 2 T 3 T /3 (2/3)(1/3) = 2/9 (2/3)(2/3)(1/3) = 4/27 (2/3)(2/3)(2/3)(1/3) = 8/81 Probability of S =1/3 + 2/9 + 4/27 + 8/ =