1. PROBABILITY AND BAYES THEOREM 1

2. 2 POPULATION SAMPLE PROBABILITY STATISTICAL INFERENCE

3. 3 PROBABILITY: A numerical value expressing the degree of uncertainty regarding the occurrence of an event. A measure of uncertainty. STATISTICAL INFERENCE: The science of drawing inferences about the population based only on a part of the population, sample.

4. 4 PROBABILITY CLASSICAL INTERPRETATION If a random experiment is repeated an infinite number of times, the relative frequency for any given outcome is the probability of this outcome. Probability of an event: Relative frequency of the occurrence of the event in the long run. –Example: Probability of observing a head in a fair coin toss is 0.5 (if coin is tossed long enough). SUBJECTIVE INTERPRETATION The assignment of probabilities to event of interest is subjective –Example: I am guessing there is 50% chance of rain today.

5. 5 PROBABILITY Random experiment –a random experiment is a process or course of action, whose outcome is uncertain. Examples ExperimentOutcomes Flip a coinHeads and Tails Record a statistics test marksNumbers between 0 and 100 Measure the time to assembleNumbers from zero and above a computer

6. 6 Performing the same random experiment repeatedly, may result in different outcomes, therefore, the best we can do is consider the probability of occurrence of a certain outcome. To determine the probabilities, first we need to define and list the possible outcomes PROBABILITY

7. 7 Determining the outcomes. –Build an exhaustive list of all possible outcomes. –Make sure the listed outcomes are mutually exclusive. The set of all possible outcomes of an experiment is called a sample space and denoted by S. Sample Space

8. 8 Countable Uncountable (Continuous ) Finite number of elements Infinite number of elements

9. 9 EXAMPLES Countable sample space examples: –Tossing a coin experiment S : {Head, Tail} –Rolling a dice experiment S : {1, 2, 3, 4, 5, 6} –Determination of the sex of a newborn child S : {girl, boy} Uncountable sample space examples: –Life time of a light bulb S : [0, ∞) –Closing daily prices of a stock S : [0, ∞)

10. Sample Space Multiple sample spaces for the same experiment are possible E.g. with 5 coin tosses we can take: S={HHHHH, HHHHT, …} or if we are only interested in the number of heads we can take S*={0,1,2,3,4,5} 10

11. 11 EXAMPLES Examine 3 fuses in sequence and note the results of each experiment, then an outcome for the entire experiment is any sequence of N’s (non-defectives) and D’s (defectives) of length 3. Hence, the sample space is S : { NNN, NND, NDN, DNN, NDD, DND, DDN, DDD}

12. 12 –Given a sample space S ={O 1,O 2,…,O k }, the following characteristics for the probability P(O i ) of the simple event O i must hold: –Probability of an event: The probability P(A), of event A is the sum of the probabilities assigned to the simple events contained in A. Assigning Probabilities

13. 13 Assigning Probabilities P(A) is the proportion of times the event A is observed.

14. Set theory: Definitions Set: a set A is a collection of elements (or outcomes) Membership: x A (x is in A), or x A (x is not in A) Complement: Union: Intersection: Difference: Subset: A is contained in B Equality: Symmetric difference: 14

15. Algebraic laws commutative: A ∪ B = B ∪ A A ∩ B = B ∩ A associative: (A ∪ B) ∪ C = A ∪ (B ∪ C) A ∩ (B ∩ C) = (A ∩ B) ∩ C distributive: A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) DeMorgan’s: (A ∪ B)' = A' ∩ B' (' is complement) (A ∩ B)' = A' ∪ B' 15

16. 16 Intersection The intersection of event A and B is the event that occurs when both A and B occur. The intersection of events A and B is denoted by (A and B) or A  B. The joint probability of A and B is the probability of the intersection of A and B, which is denoted by P(A and B) or P(A  B).

17. 17 Union The union event of A and B is the event that occurs when either A or B or both occur. At least one of the events occur. It is denoted “A or B” OR A  B

18. 18 For any two events A and B P(A  B) = P(A) + P(B) - P(A  B) Addition Rule

19. 19 Complement Rule The complement of event A (denoted by A C ) is the event that occurs when event A does not occur. The probability of the complement event is calculated by P(A C ) = 1 - P(A) A and A C consist of all the simple events in the sample space. Therefore, P(A) + P(A C ) = 1

20. 20 MUTUALLY EXCLUSIVE EVENTS Two events A and B are said to be mutually exclusive or disjoint, if A and B have no common outcomes. That is, A and B =  (empty set) The events A 1,A 2,… are pairwise mutually exclusive (disjoint), if A i  A j =  for all i  j.

21. 21 EXAMPLE The number of spots turning up when a six-sided dice is tossed is observed. Consider the following events. A: The number observed is at most 2. B: The number observed is an even number. C: The number 4 turns up.

22. 22 VENN DIAGRAM A graphical representation of the sample space. 2 S 1 3 5 4 6 A B C ABAB 1 4 6 A B 2 ABAB 1 4 6 A B 2 2 A  C =  A and C are mutually exclusive

23. 23 AXIOMS OF PROBABILTY (KOLMOGOROV AXIOMS) Given a sample space S, the probability function is a function P that satisfies 1) For any event A, 0  P(A)  1. 2) P(S) = 1. 3) If A 1, A 2,… are pairwise disjoint, then

24. Probability P : S  [0,1] Probability domain range function 24

25. 25 THE CALCULUS OF PROBABILITIES If P is a probability function and A is any set, then a. P(  )=0 b. P(A)  1 c. P(A C )=1  P(A)

26. 26 THE CALCULUS OF PROBABILITIES If P is a probability function and A and B any sets, then a.P(B  A C ) = P(B)  P(A  B) b.If A  B, then P(A)  P(B) c. P(A  B)  P(A)+P(B)  1 (Bonferroni Inequality) d. (Boole’s Inequality)

27. Principle of Inclusion-Exclusion A generalization of addition rule Proof by induction 27

28. 28 EQUALLY LIKELY OUTCOMES The same probability is assigned to each simple event in the sample space, S. Suppose that S={s 1,…,s N } is a finite sample space. If all the outcomes are equally likely, then P({s i })=1/N for every outcome s i.

29. 29 ODDS The odds of an event A is defined by It tells us how much more likely to see the occurrence of event A. P(A)=3/4  P(A C )=1/4  P(A)/P(A C ) = 3. That is, the odds is 3. It is 3 times more likely that A occurs as it is that it does not.

30. ODDS RATIO OR is the ratio of two odds. Useful for comparing the odds under two different conditions or for two different groups, e.g. odds for males versus females. If odds of event A is 4.2 for males and 2 for females, then odds ratio is 2.1. The odds of observing event A is 2.1 times higher for males compared to females. 30

31. CONDITIONAL PROBABILITY (Marginal) Probability: P(A): How likely is it that an event A will occur when an experiment is performed? Conditional Probability: P(A|B): How will the probability of event A be affected by the knowledge of the occurrence or nonoccurrence of event B? If two events are independent, then P(A|B)=P(A) 31

32. CONDITIONAL PROBABILITY 32

33. Example Roll two dice S=all possible pairs ={(1,1),(1,2),…,(6,6)} Let A=first roll is 1; B=sum is 7; C=sum is 8 P(A|B)=?; P(A|C)=? Solution: P(A|B)=P(A and B)/P(B) P(B)=P({1,6} or {2,5} or {3,4} or {4,3} or {5,2} or {6,1}) = 6/36=1/6 P(A|B)= P({1,6})/(1/6)=1/6 =P(A) A and B are independent 33

34. Example P(A|C)=P(A and C)/P(C)=P(Ø)/P(C)=0 A and C are disjoint Out of curiosity: P(C)=P({2,6} or {3,5} or {4,4} or {5,3} or {6,2}) = 5/36

35. CONDITIONAL PROBABILITY 35

36. Example Suppose we pick 4 cards at random from a deck of 52 cards containing 4 aces. A=event that we pick 4 aces A i =event that ith pick is an ace (i=1,2,3,4) 36

37. BAYES THEOREM Suppose you have P(B|A), but need P(A|B). 37

38. Example Let: –D: Event that person has the disease; –T: Event that medical test results positive Given: –Previous research shows that 0.3 % of all Turkish population carries this disease; i.e., P(D)= 0.3 % = 0.003 –Probability of observing a positive test result for someone with the disease is 95%; i.e., P(T|D)=0.95 –Probability of observing a positive test result for someone without the disease is 4%; i.e. P(T| )= 0.04 Find: probability of a randomly chosen person having the disease given that the test result is positive. 38

39. Example Solution: Need P(D|T). Use Bayes Thm. P(D|T)=P(T|D)*P(D)/P(T) P(T)=P(D and T)+P( and T) = 0.95*0.003+0.04*0.997 = 0.04273 P(D|T) =0.95*0.003 / 0.04273 = 6.67 % Test is not very reliable! 39

40. BAYES THEOREM Can be generalized to more than two events. If A i is a partition of S, then, Can be rewritten in terms of odds –Suppose A 1,A 2,… are competing hypotheses and B is evidence or data relevant to choosing the correct hypothesis Posterior odds = likelihood ratio x prior odds 40

41. Independence A and B are independent iff –P(A|B)=P(A) or P(B|A)=P(B) –P(AB)=P(A)P(B) A 1, A 2, …, A n are mutually independent iff for every subset j of {1,2,…,n} E.g. for n=3, A1, A2, A3 are mutually independent iff P(A 1 A 2 A 3 )=P(A 1 )P(A 2 )P(A 3 ) and P(A 1 A 2 )=P(A 1 )P(A 2 ) and P(A 1 A 3 )=P(A 1 )P(A 3 ) and P(A 2 A 3 )=P(A 2 )P(A 3 ) 41

42. Independence If n=4, then the number of conditions for independence is Find these conditions. 42

43. Sequences of events A sequence of events A 1, A 2, … is increasing iff A sequence of events A 1, A 2, … is decreasing iff If {A n } is increasing, then If {A n } is decreasing, then 43

44. Examples Let S=(0,1) and A n =(1/n,1) {A n } is increasing. What is limit of A n as n goes to infinity? Let S=(0,1) and B n =(0,1/n) {B n } is decreasing. What is limit of B n as n goes to infinity? 44

45. Problems 1. Show that two nonempty events cannot be disjoint and independent at the same time. Hint: First, prove that if they are disjoint, then they are not independent. Second, prove that if they are independent, then they are not disjoint.

46. Problems 2. If P(A)=1/3 and P(B c )=1/4, can A and B be disjoint? Explain.

47. Problems 3. Either prove the statement is true or disprove it: If P(B|A)=P(B|A C ), then A and B are independent.

48. Problems 4. An insurance company has three types of customers – high risk, medium risk, and low risk. Twenty percent of its customers are high risk, and 30% are medium risk. Also, the probability that a customer has at least one accident in the current year is 0.25 for high risk, 0.16 for medium risk, and 0.1 for low risk. a) Find the probability that a customer chosen at random will have at least one accident in the current year. b) Find the probability that a customer is high risk, given that the person has had at least one accident during the current year.

49. Problems 5. Eleven poker chips are numbered consecutively 1 through 10, with two of them labeled with a 6 and placed in a jar. A chip is drawn at random. i)Find the probability of drawing a 6. ii)Find the odds of drawing a 6 from the jar. iii)Find the odds of not drawing a 6. 49

50. Problems 6. If the odds in favor of winning a horse race are 3:5, find the probability of winning the race. 50

51. Problems 7. In a hypothetical clinical study, the following results were obtained. Find the odds ratio and interpret. 51 Treatment Total number of patients treated Number who achieved at least 50% pain relief Number who did not achieve at least 50% pain relief Ibuprofen 400 mg 402218 Placebo407 33