1. PROBABILITY AND STATISTICS PROBABILITY

2. Probability IIntroduction to Probability ASatisfactory outcomes vs. total outcomes BBasic Properties CTerminology IICombinatory Probability AThe Addition Rule – “Or” 1. The special addition rule (mutually exclusive events) 2. The general addition rule (non-mutually exclusive events) BThe Multiplication Rule – “And” 1. The special multiplication rule (for independent events) 2. The general multiplication rule (for non-independent events)

3. 3 OBJECTIV ES develop sound understanding of probability values used in subsequent chapters develop basic skills necessary to solve simple probability problems

4. 4 Event - any collection of results or outcomes from some procedure Simple event - any outcome or event that cannot be broken down into simpler components Sample space - all possible simple events Experiment - a procedure that yields one of a given set of possible outcomes. Definitions

5. Basic Properties of Probabilities Property 1: The probability of an event is always between 0 and 1, inclusive. Property 2: The probability of an event that cannot occur is 0. (An event that cannot occur is called an impossible event.) Property 3: The probability of an event that must occur is 1. (An event that must occur is called a certain event.)

6. Probability for Equally Likely Outcomes Suppose an experiment has N possible outcomes, all equally likely. Then the probability that a specified event occurs equals the number of ways, f, that the event can occur, divided by the total number of possible outcomes. In symbols: Probability of a given event = Number of ways a given event can occur Total of all possible outcomes

7. 7 Notation P – denotes a probability A, B,... – denote specific events P(A) – denotes the probability of event A occurring

8. 8 Rule 1: Relative Frequency Approximation Conduct (or observe) an experiment a large number of times, and count the number of times event A actually occurs, then an estimate of P(A) is P(A) = number of times A occurred number of times trial was repeated Basic Rules for Computing Probability

9. 9 Rule 2: Classical approach (requires equally likely outcomes) If a procedure has n different simple events, each with an equal chance of occurring, and s is the number of ways event A can occur, then P(A) = number of ways A can occur number of different simple events s n =

10. 10 Rule 3: Subjective Probabilities P(A), the probability of A, is found by simply guessing or estimating its value based on knowledge of the relevant circumstances. Basic Rules for Computing Probability

11. 11 Rule 1 The relative frequency approach is an educated approximation. Rule 2 The classical approach is the actual probability. Rule 3 The relative frequency approach is an absolute guess.

12. 12 Law of Large Numbers As a procedure is repeated again and again, the relative frequency probability (from Rule 1) of an event tends to approach the actual probability.

13. Two computer simulations of tossing a balanced coin 100 times Law of Large Numbers

14. 14 The sample space consists of two simple events: the person is struck by lightning or is not. Because these simple events are not equally likely, we can use the relative frequency approximation (Rule 1) or subjectively estimate the probability (Rule 3). Using Rule 1, we can research past events to determine that in a recent year 377 people were struck by lightning in the US, which has a population of about 274,037,295. Therefore, Example: Find the probability that a randomly selected person will be struck by lightning this year. P(struck by lightning in a year) ≈ 377 / 274,037,295 ≈ 1 / 727,000

15. 15 Example: On an ACT or SAT test, a typical multiple-choice question has 5 possible answers. If you make a random guess on one such question, what is the probability that your response is wrong? There are 5 possible outcomes or answers, and there are 4 ways to answer incorrectly. Random guessing implies that the outcomes in the sample space are equally likely, so we apply the classical approach (Rule 2) to get: P(wrong answer) = 4 / 5 = 0.8

16. 16 Probability Limits The probability of an impossible event is 0. The probability of an event that is certain to occur is 1. 0 ≤ P(A) ≤ 1 Impossible to occur Certain to occur

17. 17 Possible Values for Probabilities Certain Likely 50-50 Chance Unlikely Impossible 1 0.5 0

18. 18 Discrete Probability Example I: An urn contains four blue balls and five red balls. What is the probability that a ball chosen from the urn is blue? Solution: There are nine possible outcomes, and the event “blue ball is chosen” comprises four of these outcomes. Therefore, the probability of this event is 4 / 9 or approximately 44.44%.

19. 19 Discrete Probability Example II: What is the probability of winning the lottery 6/49, that is, picking the correct set of six numbers out of 49? Solution: There are C(49, 6) possible outcomes. Only one of these outcomes will actually make us win the lottery. p(E) = 1 / C(49, 6) = 1 / 13,983,816

20. 20 What happens if the outcomes of an experiment are not equally likely? In that case, we assign a probability p(s) to each outcome s ∈ S, where S is the sample space. Two conditions have to be met: (1): 0 ≤ p(s) ≤ 1 for each s ∈ S, and (2): ∑ s ∈ S p(s) = 1 This means, as we already know, that (1) each probability must be a value between 0 and 1, and (2) the probabilities must add up to 1, because one of the outcomes is guaranteed to occur. DISCRETE PROBABILITY

21. 21 How can we obtain these probabilities p(s) ? The probability p(s) assigned to an outcome s equals the limit of the number of times s occurs divided by the number of times the experiment is performed. Once we know the probabilities p(s), we can compute the probability of an event E as follows: p(E) = ∑ s ∈ E p(s) DISCRETE PROBABILITY

22. 22 Example I: A die is biased so that the number 3 appears twice as often as each other number. What are the probabilities of all possible outcomes? Solution: There are 6 possible outcomes s 1, …, s 6. p(s 1 ) = p(s 2 ) = p(s 4 ) = p(s 5 ) = p(s 6 ) and p(s 3 ) = 2p(s 1 ) Since the probabilities must add up to 1, we have: 5p(s 1 ) + 2p(s 1 ) = 1 7p(s 1 ) = 1 p(s 1 ) = p(s 2 ) = p(s 4 ) = p(s 5 ) = p(s 6 ) = 1 / 7, p(s 3 ) = 2 / 7 DISCRETE PROBABILITY

23. 23 Example II: For the biased die from Example I, what is the probability that an odd number appears when we roll the die? Solution: E odd = {s 1, s 3, s 5 } Remember the formula p(E) = ∑ s ∈ E p(s). p(E odd ) = ∑ s ∈ E odd p(s) = p(s 1 ) + p(s 3 ) + p(s 5 ) p(E odd ) = 1 / 7 + 2 / 7 + 1 / 7 = 4 / 7 = 57.14% DISCRETE PROBABILITY

24. 24 COMPLEMENTARY EVENTS P(A)or P(~A)(read “not A”) The complement of event A, denoted by A, or ~A consists of all outcomes in which event A does NOT occur.

25. 25 Because 20 of the 50 subjects are men, it follows that 30 of the 50 subjects are women so, P(not selecting a man) = P(~man) = P(woman) = 30 / 50 = 0.6 Example: Testing Corvettes The General Motors Corporation wants to conduct a test of a new model of Corvette. A pool of 50 drivers has been recruited, 20 of whom are men. When the first person is selected from this pool, what is the probability of not getting a male driver?

26. 26 Complimentary Events Let E be an event in a sample space S. The probability of an event ~E, the complimentary event of E, is given by p(~E) = 1 – p(E). This can easily be shown: p(~E) = (|S| – |E|)/|S| = 1 – |E|/|S| = 1 – p(E). This rule is useful if it is easier to determine the probability of the complimentary event than the probability of the event itself.

27. 27 Example I: A sequence of 10 bits is randomly generated. What is the probability that at least one of these bits is zero? Solution: There are 2 10 = 1024 possible outcomes of generating such a sequence. The event ~E, “none of the bits is zero”, includes only one of these outcomes, namely the sequence 1111111111. Therefore, p(~E) = 1 / 1024. Now p(E) can easily be computed as p(E) = 1 – p(~E) = 1 – 1 / 1024 = 1023 / 1024. COMPLIMENTARY EVENTS

28. 28 Example II: What is the probability that at least two out of 36 people have the same birthday? Solution: The sample space S encompasses all possibilities for the birthdays of the 36 people, so |S| = 365 36. Let us consider the event ~E (“no two people out of 36 have the same birthday”). ~E includes P(365, 36) outcomes (365 possibilities for the first person’s birthday, 364 for the second, and so on). Then p(~E) = P(365, 36)/365 36 = 0.168, so p(E) = 0.832 or 83.2% COMPLIMENTARY EVENTS

29. 29 Odds actual odds against event A occurring is the ratio P(~A)/P(A), usually expressed in the form of a:b (or a to b), where a and b are integers with no common factors actual odds in favor of event A are the reciprocal of the odds against that event, P(A)/P(~A) or b:a (or b to a)

30. 30 Odds The payoff odds against event A represent the ratio of net profit (if you win) to the amount of the bet. Payoff odds against event A = (net profit):(amount bet)

31. Frequency distribution of annual income for U.S. families

32. Probability from Frequency Distributions What is the probability of having an income between $15,000 and $24,999 10,040 / 69,597 = 0.1443

33. Frequency distribution for students’ ages N = 40 What is the likelihood of randomly selecting a student who is 21? What is the likelihood of selecting a student who’s age is an odd number? What is the likelihood of selecting a student who is either 21 or 23? 7 / 40 = 0.175 21 / 40 = 0.525 10 / 40 = 1 / 4 = 0.25

34. Sample space for rolling a die once

35. Possible outcomes for rolling a pair of dice

36. Probabilities of 2 throws of the die What is the probability of a 1 and a 3? What is the probability of two sixes? What is the probability of at least one 3? 2 / 36 or 1 / 18 1 / 36 12 / 36 or 1 / 3

37. 1 2 3 6 4 5 1 2 3 6 4 5 1 2 3 6 4 5 1 2 3 6 4 5 1 2 3 6 4 5 1 2 3 6 4 5 1 2 3 6 4 5 TREE DIAGRAM The probability of rolling a 1 and a 3. Represents two paths out of thirty six possible paths. 2 / 36 or 1 / 18

38. The Sum of Two Die Tosses What is the probability that the sum will be 5? 7? What is the probability that the sum will be 10 or more? What is the probability that the sum will be either 3 or less or 11 or more? 4 / 36 or 1 / 9 6 / 36 or 1 / 6 3 / 36 + 3 / 36 = 6 / 36 or 1 / 6 6 / 36 or 1 / 6

39. A deck of playing cards

40. A gambling experiment Everyone in the room takes 2 cards from the deck (keep face down) Rules, most to least valuable: Pair of the same color (both red or both black) Mixed-color pair (1 red, 1 black) Any two cards of the same suit Any two cards of the same color In the event of a tie, highest card wins (ace is top)

41. What do you want to bet? Look at your two cards. Will you fold or bet? What is the most rational strategy given your hand?

42. Rational strategy There are N people in the room What are the chances that someone in the room has a better hand than you? Need to know the probabilities of different scenarios

43. The Probability that the king of hearts is selected 1 / 52

44. The probability that a king is selected 4 / 52 = 1 / 13

45. 13 / 52 = 1 / 4 THE PROBABILITY THAT A HEART IS SELECTED

46. The probability that a face card is selected 12 / 52 = 3 / 13

47. Venn diagram for event E

48. Relationships Among Events (~ E): The event that “E does not occur.” (A & B): The event that “both A and B occur.” (A or B): The event that “either A or B or both occur.”

49. If E is the probability of drawing a face card, P(~ E ). P(~E) = 40 / 52 = 10 / 13 But recall, P(E) = 12 / 52 = 3 / 13 and 1 – 3 / 13 = 13 / 13 – 3 / 13 = 10 / 13

50. An event and its complement

51. The Complementation Rule For any event E, P(E) = 1 – P(~E) In words, the probability that an event occurs equals 1 minus the probability that it does not occur or P(~E) = 1 – P(E) In words, the probability that an event does not occur equals 1 minus the probability that it does occur.

52. Combinations of Events The Addition Rule – “Or” The special addition rule (mutually exclusive events) The general addition rule (non-mutually exclusive events) The Multiplication Rule – “And” The special multiplication rule (for independent events) The general multiplication rule (for non-independent events)

53. Venn diagrams for (a) event (~E ) (b) event (A & B) (c) event (A or B)

54. P(King & Heart) A King of Hearts 1 / 13 X 1 / 4 = 1 / 52

55. P( Face Card & Heart ) A Face Card that is a Heart 3 / 13 X 1 / 4 = 3 / 52

56. Mutually Exclusive Events Two or more events are said to be mutually exclusive if at most one of them can occur when the experiment is performed, that is, if no two of them have outcomes in common

57. Two mutually exclusive events

58. (a) Two mutually exclusive events (b) Two non-mutually exclusive events

59. (a) Three mutually exclusive events (b) Three non-mutually exclusive events (c) Three non-mutually exclusive events

60. The Special Addition Rule If event A and event B are mutually exclusive, then P(A or B) = P(A) + P(B) More generally, if events A, B, C, … are mutually exclusive, then P(A or B or C...) = P(A) + P(B) + P(C) +... That is, for mutually exclusive events, the probability that at least one of the events occurs is equal to the sum of the individual probabilities.

61. Non-mutually exclusive events

62. The General Addition Rule If A and B are any two events, then P(A or B) = P(A) + P(B) – P(A & B) or p(A ∪ B) = p(A) + p(B) - p(A ∩ B). In words, for any two events, the probability that one or the other occurs equals the sum of the individual probabilities less the probability that both occur.

63. P(king or heart) A King or a Heart 4 / 52 + 13 / 52 – 1 / 52 = 16 / 52 = 4 / 13

64. P(spade or face card) P (spade) + P (face card) – P (spade & face card) = 1 / 4 + 3 / 13 – 3 / 52 = 22 / 52 = 11 / 26

65. 65 Example: What is the probability of a positive integer selected at random from the set of positive integers not exceeding 100 to be divisible by 2 or 5? Solution: E 2 : “integer is divisible by 2” ; E 5 : “integer is divisible by 5” E 2 = {2, 4, 6, …, 100} |E 2 | = 50 p(E 2 ) = 0.5 E 5 = {5, 10, 15, …, 100} |E 5 | = 20 p(E 5 ) = 0.2 E 2 ∩ E 5 = {10, 20, 30, …, 100} |E 2 ∩ E 5 | = 10 p(E 2 ∩ E 5 ) = 0.1 p(E 2 ∪ E 5 ) = p(E 2 ) + p(E 5 ) – p(E 2 ∩ E 5 ) p(E 2 ∪ E 5 ) = 0.5 + 0.2 – 0.1 = 0.6 THE GENERAL ADDITION RULE

66. The Special Multiplication Rule (for independent events) If events A, B, C,... are independent, then P(A & B & C &...) = P(A) P(B) P(C)... What is the probability of all of these events occurring: Flip a coin and get a head Draw a card and get an ace Throw a die and get a 1 P(A & B & C ) = P(A)· P(B)· P(C) = 1 / 2 X 1 / 13 X 1 / 6 = 1 / 156

67. 67 Conditional Probability If we toss a coin three times, what is the probability that an odd number of tails appears (event E), if the first toss is a tail (event F) ? If the first toss is a tail, the possible sequences are TTT, TTH, THT, and THH. In two out of these four cases, there is an odd number of tails. Therefore, the probability of E, under the condition that F occurs, is 0.5. We call this conditional probability.

68. Conditional Probability: For non- independent events The probability that event B occurs given that event A has occurred is called a conditional probability. It is denoted by the symbol P(B | A), which is read “the probability of B given A.” We call A the given event.

69. 69 Example: What is the probability of a random bit string of length four contains at least two consecutive 0s, given that its first bit is a 0 ? Solution: E: “bit string contains at least two consecutive 0s” F: “first bit of the string is a 0” We know the formula p(E | F) = p(E ∩ F)/p(F). E ∩ F = {0000, 0001, 0010, 0011, 0100} p(E ∩ F) = 5 / 16 p(F) = 8 / 16 = 1 / 2 p(E | F) = ( 5 / 16 )/( 1 / 2 ) = 5 / 32 = 0.15625 CONDITIONAL PROBABILITY

70. Contingency table for age and rank of faculty members (using frequencies)

71. The Conditional-Probability Rule If A and B are any two events, then In words, for any two events, the conditional probability that one event occurs given that the other event has occurred equals the joint probability of the two events divided by the probability of the given event.

72. The Conditional-Probability Rule P(R 3 |A 4 ) = 36 / 253 = 0.142 P(A 4 |R 3 ) = 36 / 320 = 0.112 What is the probability that someone is an Assistant Professor given that they are 50-59 years old? What is the probability that someone is 50- 59 years old given that they are an Assistant Professor?

73. Joint probability distribution (using proportions) P(R 3 |A 4 ) = 0.031 / 0.217 = 0.142 P( A 4 | R 3 ) = 0.031 / 0.275 = 0.112 What is the probability that someone is an Assistant Professor given that they are 50-59 years old? What is the probability that someone is 50- 59 years old given that they are an Assistant Professor?

74. Contingency table of marital status and sex (using proportions)

75. Joint probability distribution (using proportions)

76. The General Multiplication Rule If A and B are any two events, then P(A & B) = P(A) P(B | A). In words, for any two events, their joint probability equals the probability that one of the events occurs times the conditional probability of the other event given that event. Note: Either 1) The events are independent and then P(A & B) = P(A) · P(B). Or 2) The events are not independent and then a contingency table must be used

77. 77 Independence Let us return to the example of tossing a coin three times. Does the probability of event E (odd number of tails) depend on the occurrence of event F (first toss is a tail) ? In other words, is it the case that p(E | F) ≠ p(E) ? We actually find that p(E | F) = 0.5 and p(E) = 0.5, so we say that E and F are independent events.

78. 78 Because we have p(E | F) = p(E ∩ F)/p(F), p(E | F) = p(E) if and only if p(E ∩ F) = p(E)p(F). Definition: The events E and F are said to be independent if and only if p(E ∩ F) = p(E)p(F). Obviously, this definition is symmetrical for E and F. If we have p(E ∩ F) = p(E)p(F), then it is also true that p(F | E) = p(F). INDEPENDENC E

79. Independent Events Event B is said to be independent of event A if the occurrence of event A does not affect the probability that event B occurs. In symbols, P(B | A) = P(B). This means that knowing whether event A has occurred provides no probabilistic information about the occurrence of event B. Class Fr So Ju Se Male 40 50 50 40| 180 Female80 100 100 80| 360 120 150 150 120| 540

80. 80 Bernoulli Trials Suppose an experiment with two possible outcomes, such as tossing a coin. Each performance of such an experiment is called a Bernoulli trial. We will call the two possible outcomes a success or a failure, respectively. If p is the probability of a success and q is the probability of a failure, it is obvious that p + q = 1.

81. 81 Often we are interested in the probability of exactly k successes when an experiment consists of n independent Bernoulli trials. Example: A coin is biased so that the probability of head is 2/3. What is the probability of exactly four heads to come up when the coin is tossed seven times? BERNOULLI TRIALS

82. 82 Solution: There are 2 7 = 128 possible outcomes. The number of possibilities for four heads among the seven trials is C(7, 4). The seven trials are independent, so the probability of each of these outcomes is ( 2 / 3 ) 4 ( 1 / 3 ) 3. Consequently, the probability of exactly four heads to appear is C(7, 4)( 2 / 3 ) 4 ( 1 / 3 ) 3 = 560 / 2187 = 25.61% BERNOULLI TRIALS

83. Probability and the Normal Distribution What is the probability of randomly selecting an individual with an I.Q. between 95 and 115? Mean 100, S.D. 15. Find the z-score for 95 and 115 and compute the area between

84. Probability and Experimental Design Using probability to evaluate a treatment effect. Values that are extremely unlikely to be obtained from the original population are viewed as evidence of a treatment effect.

85. A Preview of Sampling Distributions XX What is the probability of randomly selecting a sample of three individuals, all of whom have an I.Q. of 135 or more? Find the z-score of 135, compute the tail region and raise it to the 3 rd power. This concept is critical to understanding future concepts So while the odds of a chance selection of a single person this far above the mean is not all that unlikely, the odds of a sample this far above the mean are astronomical z = 2.19P = 0.0143 0.0143 3 = 0.0000029

86. Summary For multiple events there are two rules: “AND” (multiplication) and “OR” (addition) There are just a few special considerations: For the “And” rule, if the events are not independent, you don’t multiply, you use a table. For the “Or” rule, if the events are not mutually exclusive you have to subtract off their double count