Constraints on Credence

Conditional Probability

Conditional Probability Question: What is the probability of rolling a prime number (2, 3, 5) when one rolls a 6 sided die?

Conditional Probability Answer: since all the sides are equal: 3/6, or 1/2.

Conditional Probability POSSIBLE ROLLS 1 2 3 4 5 6

Conditional Probability Question: What is the probability of rolling a prime number (2, 3, 5), given that you know the number is even?

STEP 1: Ignore Odd Rolls POSSIBLE ROLLS 1 2 3 4 5 6

STEP 2: Find the Prime Rolls POSSIBLE ROLLS 1 2 3 4 5 6

STEP 3: Count the Prime-and-Even Rolls as a Proportion of the Even Rolls POSSIBLE ROLLS 1 2 3 4 5 6

Conditional Probability Answer: 1/3 This is the probability of prime conditional on rolling even.

Definition of Conditional Probability Pr( p / q ) = Pr( p & q ) Pr( q )

Exercise 1 (from the book) Suppose: Pr( wind ) = 0.6 Pr( rain ) = 0.5 Pr( wind & rain ) = 0.4 What is Pr( wind / rain) and Pr( rain / wind) ?

Pr( wind / rain ) = ? Pr( p / q ) = Pr( p & q ) Pr( q )

Pr( wind / rain ) = ? Pr( wind / rain ) = Pr( wind & rain ) Pr( rain )

Pr( wind / rain ) = ? Pr( wind / rain ) = 0.4 Pr( rain )

Pr( wind / rain ) = ? Pr( wind / rain ) = 0.4 0.5 Pr( wind ) = 0.6 Pr( rain ) = 0.5 Pr( wind & rain ) = 0.4

Pr( wind / rain ) = ? Pr( wind / rain ) = = 4/5 0.4 0.5 Pr( rain ) = 0.5 Pr( wind & rain ) = 0.4

Pr( rain / wind ) = ? Pr( p / q ) = Pr( p & q ) Pr( q )

Pr( rain / wind ) = ? Pr( rain / wind ) = Pr( rain & wind ) Pr( wind ) Pr( wind & rain ) = 0.4

Pr( rain / wind ) = ? Pr( rain / wind ) = Pr( rain & wind ) Pr( wind ) Pr( wind & rain ) = 0.4

Pr( rain / wind ) = ? Pr( rain / wind ) = 0.4 Pr( wind ) Pr( wind & rain ) = 0.4

Pr( rain / wind ) = ? Pr( rain / wind ) = 0.4 0.6 Pr( wind ) = 0.6 Pr( wind & rain ) = 0.4

Pr( rain / wind ) = ? Pr( rain / wind ) = = 2/3 0.4 0.6 Pr( wind ) = 0.6 Pr( rain ) = 0.5 Pr( wind & rain ) = 0.4

Bayes’ Theorem

Reverend Thomas Bayes Relatively obscure British mathematician. Proved a (more specific) instance of the theorem that bears his name

Bayes’ Theorem Pr( p / q ) = Pr( q / p ) x Pr( p ) Pr( q )

Proof 1. Pr( a / b ) = Pr( a & b ) ÷ Pr( b ) by Definition of Conditional Probability 2. Pr( a / b ) x Pr( b ) = Pr( a & b ) multiplying both sides by Pr( b ) 3. Pr( a & b ) = Pr( a / b ) x Pr( b ) symmetry of = 4. Pr( q & p ) = Pr( q / p ) x Pr( p ) substitution

Area(Z & B) = Area(Z) x Area(B/ Z) Formula from Last Time Area(Z & B) = Area(Z) x Area(B/ Z) This is the proportion of Z that is taken up by B

Proof Part 2 1. Pr( p / q ) = Pr( p & q ) ÷ Pr( q ) by Definition of Conditional Probability 2. p & q = q & p by logic 3. Pr( p / q ) = Pr( q & p ) ÷ Pr( q ) by 1, 2, and substitution 4. Pr( p / q ) = [ Pr( q / p ) x Pr( p ) ] ÷ Pr( q ) substitution from last slide

Base Rate Fallacy

Elements of Bayes’ Theorem Pr( p / q ) = “likelihood” “prior probability” Pr( q / p ) x Pr( p ) Pr( q )

Bayes’ Theorem Baye’s theorem lets us calculate the probability of A conditional on B when we have the probability of B conditional on A.

Base Rate Fallacy There are ½ million people in Russia are affected by HIV/ AIDS. There are 150 million people in Russia.

Base Rate Fallacy Imagine that the government decides this is bad and that they should test everyone for HIV/ AIDS.

The Test If someone has HIV/ AIDS, then : 95% of the time the test will be positive (correct) 5% of the time will it be negative (incorrect)

The Test If someone does not have HIV/ AIDS, then: 95% of the time the test will be negative (correct) 5% of the time will it be positive (incorrect)

Suppose you test positive Suppose you test positive. We’re interested in the conditional probability: what is the probability you have HIV assuming that you test positive. We’re interested in Pr(HIV = yes/ test = pos)

Known: Pr(sick) = 1/300 Known: Pr(positive/ sick) = 95% Known: Pr(positive/ not-sick) = 5% Unknown: Pr(positive) Unknown: Pr(sick/ positive)

Pr(positive) Pr(positive) = True positives + false positives = [Pr(positive/ sick) x Pr(sick)] + [Pr(positive/ not-sick) x Pr(not-sick)] = [95% x 1/300] + [5% x 299/300] = 5.3% Known: Pr(sick) = 1/300 Known: Pr(positive/ sick) = 95% Known: Pr(positive/ not-sick) = 5%

Known: Pr(sick) = 1/300 Known: Pr(positive/ sick) = 95% Known: Pr(positive) = 5.3% Unknown: Pr(sick/ positive)

Pr(sick/ positive) Pr(A/ B) = [Pr(B/ A) x Pr(A)] ÷ Pr(B) Pr(sick/ positive) = [Pr(positive/ sick) x Pr(sick)] ÷ Pr(positive) = [95% x Pr(sick)] ÷ Pr(positive) = [95% x 1/300] ÷ Pr(positive) = [95% x 1/300] ÷ 5.3% = 5.975% Known: Pr(sick) = 1/300 Known: Pr(positive/ sick) = 95% Known: Pr(positive) = 5.3%

Conditionalization

Confirming Hypotheses Suppose there is a certain hypothesis H for which you are collecting evidence E. For example, the hypothesis could be that some person is or is not HIV- positive.

Elements of Bayes’ Theorem Pr( H / E ) = “likelihood” “prior probability” Pr( E / H ) x Pr( H ) Pr( E )

Elements of Bayes’ Theorem Pr( H ) is the prior probability of your hypothesis H being true– that is, prior to collecting any evidence. Pr( E / H ) is the likelihood that you will observe evidence E, on the assumption that your hypothesis H is true.

Confirming Hypotheses Now suppose you collect the evidence and you know that E is true. Your degree of belief in your hypothesis H should now change (higher if E supports it, lower if it supports not-H).

Bayesianism, i.e. Conditionalization PrNEW( H ) = PrOLD( H / E )

Diachronic Constraint The axioms of probability are synchronic constraints on your degrees of belief: they are how those degrees should relate to one another at any moment. A bunch of different probability functions all satisfy these axioms.

Diachronic Constraint Therefore, for all the axioms care, PrNEW(H) can be anything at all, so long as, for example PrNEW(not-H) is 1 – PrNEW(H), etc. The axioms don’t tell you how your degrees of belief should relate to each other over time.

Conditionalization Does! PrNEW( H ) = PrOLD( H / E )

Exercise 3 (from the book) Suppose you have good reason to believe that Pr( H ) = 0.1 Pr( E ) = 0.2 Pr( E / H ) = 0.8 Then you learn E. What probability should you now attach to H?

Start with Bayes’ Theorem Pr( H / E ) = Pr( E / H ) x Pr( H ) Pr( E ) Pr( H ) = 0.1 Pr( E ) = 0.2 Pr( E / H ) = 0.8

Fill in What You Know Pr( H / E ) = 0.8 x 0.1 0.2 Pr( H ) = 0.1 Pr( E ) = 0.2 Pr( E / H ) = 0.8

Calculate! Pr( H / E ) = = 0.4 0.8 x 0.1 0.2 Pr( H ) = 0.1 Pr( E ) = 0.2 Pr( E / H ) = 0.8

Become a Bayesian! PrNEW( H ) = PrOLD( H / E ) = = 0.4 0.8 x 0.1 0.2 Pr( H ) = 0.1 Pr( E ) = 0.2 Pr( E / H ) = 0.8

Problem #4 & The Theorem of Total Probability

Problem #4 Warning: NOT ON THE TEST! You have a 10% degree of belief that a coin is not fair but has a 75% bias in favor of heads. You toss it twice and see two heads. What should be your degree of belief that it is fair?

Problem #4 Warning: NOT ON THE TEST! You have a 10% degree of belief that a coin is not fair but has a 75% bias in favor of heads. You toss it twice and see two heads. What should be your degree of belief that it is fair? ASSUMPTION: You have 90% degree of belief that it is fair.

What Do We Know? Hypothesis H: Coin is biased 75% in favor of heads. Pr( H ) = 10%

What Do We Know? Evidence E = Coin lands heads twice in a row. Pr( E ) = ???

What Do We Know What is Pr( E / H )? Well, if the coin is biased 75% towards heads, then there is a 75% probability that it lands heads on the first toss, and a 75% probability that it lands heads on the second toss. Since the tosses are independent: Pr( both heads ) = Pr ( heads first ) x Pr (heads second) = 3/4 x 3/4 = 9/16.

How Do We Solve? we know this we know this Pr( H / E ) = Pr( E / H ) x Pr( H ) Pr( E ) we don’t know this

Logic to the Rescue Theorem: P is equivalent to [ ( P & Q ) or ( P & not-Q) ] Proof part 1: Suppose P. Now either Q is true or it is false. But you know P is true. So you are either living in a world where P and Q are both true, or you are living in a world where P is true but Q is false. Proof part 2: Suppose that one of the following two possibilities is true (a) P and Q (b) P and not-Q. If the first possibility is true, then P. If the second possibility is true, also P. So either way, P.

Using the Theorem Pr( E ) = Pr[ ( E & H ) or ( E & not-H ) ] = Pr( E & H ) + Pr( E & not-H ) because ‘E & H’ and ‘E & not-H’ are exclusive = Pr( E / H ) x Pr( H ) + Pr( E / not-H) x Pr( not-H ) by the Definition of Conditional Probability

Definition of Conditional Probability Pr( E / H ) = Pr( E & H ) Pr( H )

Using the Theorem Pr( E ) = Pr[ ( E & H ) or ( E & not-H ) ] = Pr( E & H ) + Pr( E & not-H ) because ‘E & H’ and ‘E & not-H’ are exclusive = Pr( E / H ) x Pr( H ) + Pr( E / not-H) x Pr( not-H ) by the Definition of Conditional Probability = 9/16 x 1/10 + Pr( E / not-H ) x 9/10 earlier calculations = 9/16 x 1/10 + 1/4 x 9/10 = 9/32

How Do We Solve? we know this we know this Pr( H / E ) = Pr( E / H ) x Pr( H ) Pr( E ) now we know this

How Do We Solve? we know this we know this Pr( H / E ) = 9/16 x 1/10 9/32 now we know this

How Do We Solve? we know this we know this Pr( H / E ) = = 0.2 9/16 x 1/10 9/32 now we know this

The Principal Principle

Cr( p / Pr(p) = x) = x