Introduction to Algorithms Probability Review – Appendix C CIS 670

Notions of Discrete Probability

Comp 122 Discrete probability The language of probability helps count all possible outcomes.The language of probability helps count all possible outcomes. Definitions:Definitions: Random Experiment (or Process)Random Experiment (or Process) Result (outcome) is not fixed. Multiple outcomes are possible.Result (outcome) is not fixed. Multiple outcomes are possible. Ex: Throwing a fair die.Ex: Throwing a fair die. Sample Space SSample Space S Set of all possible outcomes of a random experiment.Set of all possible outcomes of a random experiment. Ex: {1, 2, 3, 4, 5, 6} when a die is thrown.Ex: {1, 2, 3, 4, 5, 6} when a die is thrown. Elementary EventElementary Event A possible outcome, element of S, x  S;A possible outcome, element of S, x  S; Ex: 2 – Throw of fair die resulting in 2.Ex: 2 – Throw of fair die resulting in 2. Event EEvent E Subset of S, E  S;Subset of S, E  S; Ex: Throw of die resulting in {x > 3} = {4, 5, 6}Ex: Throw of die resulting in {x > 3} = {4, 5, 6} Certain event : SCertain event : S Null event : Null event :  Mutual ExclusionMutual Exclusion Events A and B are mutually exclusive if A  B= .Events A and B are mutually exclusive if A  B= .

Flip a coin Running example: flip a coin twice and write down the outcomes (heads or tails).Running example: flip a coin twice and write down the outcomes (heads or tails). Sample space = elementary events = possible outcomes of experiment = {HH,HT, TH, TT}.Sample space = elementary events = possible outcomes of experiment = {HH,HT, TH, TT}. Events = subsets of sample space. For example {HH,HT, TH} (at least one heads).Events = subsets of sample space. For example {HH,HT, TH} (at least one heads). Each event has a certain probability.Each event has a certain probability. Uniform probability distribution: all elementary events have equal probability: Pr[HH] = Pr[HT] = Pr[TH] = Pr[TT] = 1/4.Uniform probability distribution: all elementary events have equal probability: Pr[HH] = Pr[HT] = Pr[TH] = Pr[TT] = 1/4.

Axioms of Probability A probability distribution Pr{} on a sample space S is a mapping from events of S to real numbers such that the following are satisfied:

Discrete Probability -Example

Axioms of Probability & Conclusions

Conditional Probability

Sometimes we have some prior partial knowledge about the outcome of an experiment. For example, suppose that a friend has flipped two fair coins and has told you that at least one of the coins showed a head. What is the probability that both coins are heads?Sometimes we have some prior partial knowledge about the outcome of an experiment. For example, suppose that a friend has flipped two fair coins and has told you that at least one of the coins showed a head. What is the probability that both coins are heads? In the example above, A is the event that both coins are heads, and B is the event that at least one coin is a head. Thus, Pr{A | B} = (1/4)/(3/4) = 1/3.In the example above, A is the event that both coins are heads, and B is the event that at least one coin is a head. Thus, Pr{A | B} = (1/4)/(3/4) = 1/3.

Independent Events Example 1: Experiment: Rolling two independent dice.Example 1: Experiment: Rolling two independent dice. Event A: Die 1 < 3 Event A: Die 1 < 3 Event B: Die 2 > 3 Event B: Die 2 > 3 A and B are independent. A and B are independent. Example 2: For example, suppose that two fair coins are flipped and that the outcomes are independent. Then the probability of two heads is (1/2)(1/2) = 1/4.Example 2: For example, suppose that two fair coins are flipped and that the outcomes are independent. Then the probability of two heads is (1/2)(1/2) = 1/4.

Discrete Random Variables A random variable X is a function from a sample space S to the real numbers. If the space is finite or countably infinite, a random variable X is called a discrete random variable. Maps each possible outcome of an experiment to a real number.

Comp 122 Discrete Random Variables Example:Example: Rolling 2 dice.Rolling 2 dice. X: Sum of the values on the two dice.X: Sum of the values on the two dice. Pr{X=7} = Pr{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)}Pr{X=7} = Pr{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)} = 6/36 = 1/6. = 6/36 = 1/6.

Expected Value (Mean) of a Random Variable

Expected Value -Properties

For mutually independent random variablesFor mutually independent random variables X 1, X 2, …, X n E[X 1 X 2 … X n ] = E[X 1 ] E[X 2 ]…E[X n ]E[X 1 X 2 … X n ] = E[X 1 ] E[X 2 ]…E[X n ]

Indicator Random Variables In order to analyze many algorithms, including the hiring problem, we will use indicator random variables. Indicator random variables provide a convenient method for converting between probabilities and expectationsIn order to analyze many algorithms, including the hiring problem, we will use indicator random variables. Indicator random variables provide a convenient method for converting between probabilities and expectations A simple yet powerful technique for computing the expected value of a random variable.A simple yet powerful technique for computing the expected value of a random variable. Takes only 2 values, 1 and 0.Takes only 2 values, 1 and 0. Suppose we are given a sample space S and an event A. Then the indicator random variable I {A} associated with event A is defined as:Suppose we are given a sample space S and an event A. Then the indicator random variable I {A} associated with event A is defined as:

Indicator Random Variable - Example