Probability and Probability Distributions

Probability Concepts Probability: –We now assume the population parameters are known and calculate the chances of obtaining certain samples from this population. –This is the reverse of statistics and statistical measurements. –The ability to measure the probability of occurrence of a certain event or events is the basis for inference.

Definitions Experiment: –An act or process that leads to a single outcome that cannot be predicted with certainty. Event: –A collection of one or more simple events. Simple event - outcome of an experiment that cannot be decomposed into a simpler outcome.

Example of Simple Events Experiment: –Toss two coins and observe the up faces. Simple events: –Observe H1, H2; or –Observe H1, T2; or –Observe T1, H2; or –Observe T1, T2.

Example of Events Experiment: –Toss a die and observe the up face. Simple events: –Observe a 1; or- Observe a 2; or –Observe a 3; or- Observe a 4; or –Observe a 5; or- Observe a 6. –A event would be “Observe an even number” since it can be decomposed into the three simple events in the right column above.

Definitions Sample space of an experiment: –The collection of all of its simple events. Probability of a simple event (outcome): –The likelihood that the event will occur when the experiment is performed. –An important property of simple events is that with one performance of the experiment, one and only one of the simple events will occur.

Venn Diagram A graphical method for showing a sample space and its associated simple events. Example: –The experiment of “Toss a die and observe the up face”. –The associated Venn Diagram is: S

Probability A number that represents the chance that a particular outcome will occur if the experiment is conducted. Three types –A priori - each outcome equally likely. –Relative Frequency - proportion of past experiments where the outcome occurred. –Subjective - best estimate of an expert.

Simple Event Example Experiment: –Toss a coin and observe the up face. Venn Diagram: –Probability of obtaining a “Heads” on one toss of the coin equals 0.5; –Probability of obtaining a “Tails” on one toss of the coin equals 0.5. H T S

Event Example Experiment: –Toss a die and observe the up face. Venn Diagram: –Probability of “Obtaining an even number” (an event) equals the probability of obtaining a 2 plus the probability of obtaining a 4 plus the probability of obtaining a 6 (the sum of three simple events) S

Probability Notes For simple events: –All simple event probabilities must lie between 0 (0%) and 1 (100%) inclusive. (Simple events either happen with certainty, don’t happen at all, or somewhere in between.) –The probabilities of all simple events in the sample space must sum to 1 (100%). –The probability of an event is calculated by summing the probabilities of the simple events which compose that event.

Steps to Calculate Probabilities of Events Define the experiment. List the simple events. Assign probabilities to the simple events. Determine the collection of simple events contained in the event of interest. Sum the simple event probabilities to obtain the event probability.

Coin Toss Example Experiment: –Toss two coins and observe the up faces. Venn diagram: –P(H1, H2) = 1/4 or 0.25; –P(H1, T2) = 1/4 or 0.25; –P(T1, H2) = 1/4 or 0.25; –P(T1, T2) = 1/4 or H1, H2 H1, T2 T1, H2 T1, T2 S

Coin Toss Example, cont’d Event A: –Probability of observing exactly one head. –P(A) = P(H1, T2) + P(T1, H2) = –P(A) = 0.50 Event B: –Probability of observing at least one head. –P(B) = P(H1, H2) + P(H1, T2) + P(T1, H2) –P(B) = = 0.75

Venn Diagram T1, T2 H1, H2 H1, T2 T1, H2 S B A

Determining the Number of Simple Events Simple enumeration: –Four possibilities (X1, X2, X3, X4) and we need to choose two: –Simple Events (combinations): X1, X2-X2, X3 -X3, X4 X1, X3-X2, X4 X1, X4 –Exponentially complex as the number of possibilities increases.

Combinatorial Mathematics A way to calculate the total number of possible combinations for x samples from a population N: –N is the number of elements in the population. –x is the number of elements in each simple event.

Combinatorial Mathematics Example Four possibilities (X1, X2, X3, X4) and we need to choose two: Ten possibilities and we need to choose six:

Compound Events: Union Union: –All outcomes (events) that are either part of A or part of B or both. –Symbol: –Venn Diagram: S AB

Compound Events: Intersection Intersection: –All outcomes (events) that are part of both A and B. –Symbol: –Venn Diagram: S AB

Example Experiment: –Toss a die and observe the up face. Define the following events: –A: {Toss an even number} –B: {Toss a number less than or equal to 3} Venn Diagram: S 5 BA

Example, cont’d Union: –An even number or a number less than or equal to 3, or both. – ={1, 2, 3, 4, 6}. –P =P(1)+P(2)+P(3)+P(4)+P(6)=5/6 Intersection: –Both an even number and a number less than or equal to 3. – ={2} –P =P(2)=1/6

Additive Rule of Probability Additive Rule of Probability: –The probability of the union of events A and B is the sum of the probabilities of events A and B minus the probability of the intersection of events A and B. –Symbolically: –Subtract out the intersection because it was included twice.

Mutually Exclusive Events Events A and B are mutually exclusive if the intersection contains no simple events. –Venn Diagram: –Symbolically: S AB

Example Toss two fair coins: Define events: –A: {Observe at least one head} –B: {Observe exactly one head} –C: {Observe exactly two heads} So:

Venn Diagram T1, T2 H1, T2 T1, H2 S B B H1, H2 C A

Complimentary Events Compliment: –The compliment of any event A is the event that a does not occur, i.e.”not A”. –Symbolically: –The sum of the probabilities of complimentary events equals 1:

Using a Complimentary Event to Calculate Probability Toss two fair coins: –Let event A: {Observe at least one head}, i.e. A={H1, H2; H1, T2; T1, H2}. –The compliment of event A is: –Rewriting:

Conditional Probability Conditional probability: –The probability that event A occurs given that event B occurs. –Symbolically: –Venn Diagram: S 5 BA

Conditional Probability Formula Calculated as: –For die example:

Older Child Paradox A random family of two children, assuming all four gender combinations are equally likely: –P(FF)=P(FM)=P(MF)=P(MM)=0.25 –What is the conditional probability that FF will occur given that B occurred, where B is the event that at least one of the children is a girl? –What is the conditional probability that FF will occur given that B occurred, where B is the event that the older child is a girl?

Venn Diagrams M1M2 M1F2 F1F2 S B A F1M2 M1M2 M1F2 F1F2 S B A F1M2 At least one girl=1/3Oldest is a girl=1/2

Multiplicative Rules of Probability Multiplicative Rule: –or

Independence Independence: –Events A and B are said to be independent if the assumption that B has occurred does not alter the probability that A occurs.

Random Sampling Random sample - select a group of n units in such a way that each sample of size n has the same chance of being selected. Random number table - the numbers occur randomly and with equal probability no matter where you start or how you move.