1. EMIS 7300 SYSTEMS ANALYSIS METHODS Spring 2006 Dr. John Lipp Copyright © 2002 - 2005 Dr. John Lipp

2. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-2 Session 1 Outline Part 1: Decision Theory I. Part 2: Probability Theory. Part 3: Decision Theory II. Part 4: Utility Theory.

3. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-3 Today’s Session Topics Part 2: Probability Theory –Random Phenomena –Outcomes, Events, Sample spaces, and Venn Diagrams –Partitions –Probability Axioms –Joint Probability –Conditional Probability –Multiplication Rule –Total Probability Rule –Bayes’ Theorem –Statistical Independence

4. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-4 Random Phenomena When a phenomenon occurs that can have unpredictable results under “controlled” conditions, it is denoted as a random experiment or trial. The result is denoted as an event or outcome of the experiment (or trial). The terms experiment and trail are used interchangeably. However, event and outcome have specific meanings in the context of probability theory. –An outcome (elementary outcome) is the smallest unit or division that can result from a random experiment. –An event is a set of outcomes. –The event containing all possible outcomes, often denoted S, is called the sample space of the experiment. –All events are subsets of S.

5. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-5 Random Phenomena (cont.) Mathematically speaking, set theory describes outcomes and events. Set illustrations are called Venn diagrams. –Subset (“Events”): A  S, B  S –Set compliment (“Not A”): A c –Set union (“A or B”): A  B –Set intersection (“A and B”): A  B –DeMorgan’s Law: (A  B) c = A c  B c S AB S AAcAc S AB S AB S AB

6. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-6 Random Phenomena (cont.) Example: Rolling a die –Outcomes: 1,2,3,4,5, or 6 pips. –Events: A: Roll an even number B: Roll greater than three A and B (A  B) not A, or B (A c  B) 123456 Sample Space S 123456 123456 123456 123456

7. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-7 Random Phenomena (cont.) The “null” or “empty” event is denoted . Note S c = . If A and B are events and A  B =  then they are referred to as mutually exclusive (or disjoint) events. By definition, all outcomes are mutually exclusive. A set of events {A i : i = 1,…,M} are mutually exclusive (or pair-wise disjoint events) if A i  A j =  when i = j. –Pair-wise disjoint implies triplet-wise disjoint, etc. S AB

8. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-8 Random Phenomena (cont.) If the set of events {A i : i = 1,…,M} are mutually exclusive and then the {A i : i = 1,…,M} is called a partition of S. The set containing all (elementary) outcomes is a partition. Question: Can a partition be infinite in size?

9. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-9 Probability The probability of an outcome or event is the relative frequency of that outcome or event. That is, the ratio of the number of occurrences to the number of trials as the number of trails grows larger and larger. The probability of an outcome or event is written as P(E). In the simplest case, where all outcomes are equally likely, P(E) = 1/N, where N is the number of outcomes in S. Example: Over time, an equal number of heads and tails are expected to occur flipping a coin. That is, P(H) = P(T) = 1/2.

10. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-10 Probability Axioms 1.Probability of S is 1, that is, P(S) = 1. “The probability of some outcome is certain.” 2.Probability is bounded, 0  P(E)  1. 3.If A and B are disjoint events then –Note that  is disjoint from every other event and thus P(  ) = 0. “The probability that nothing happens is impossible.”

11. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-11 Probability Axioms (cont.) If the set of events {A i : i = 1,…,  } are mutually exclusive then The probability of an event A or its compliment A c happening is unity. A AcAc

12. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-12 What happens if two events A and B are not disjoint? Divide B with the partition {A, A c } Then and Probability Axioms (cont.) S A B S B-A = B  A c B  AB  A A

13. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-13 Joint Probability In practice experiments do not happen in isolation. A random phenomenon can encompass several co-occurring events. Example: Consider a drug test with two defined outcomes, Pass or Fail. The drug test is uninteresting until applied to a testee. The testee may be a drug user, or may be drug free. The events of each experiment in joint probability are assigned a letter and subscript. The letter specifies the experiment, and the subscript an event. Example: Denote the outcome of the drug test as A 1 = “pass” and A 2 = “fail”, and the testee’s status as B 1 = “drug free” and B 2 = “drug user.”

14. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-14 Joint Probability (cont.) SASA A 1 = “Pass” A 2 = “Fail”

15. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-15 Joint Probability (cont.) SBSB B 1 = “Drug Free” B 2 = “Drug User”

16. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-16 Joint Probability (cont.) S = S A  S B A 1, B 1 = “Pass, Drug Free” A 2, B 1 = “Fail, Drug Free” A 2, B 2 = “Fail, Drug User”A 1, B 2 = “Pass, Drug User”

17. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-17 Joint Probability (cont.) The joint probability of two events is the probability of the first event happening while simultaneously (“and”) a second event also happens. The joint probability for two events is written P(A  B), or (more commonly) as P(A, B). Example: The joint probabilities for the drug test and testee are P(A 1, B 1 )“Drug test passed by drug free” P(A 2, B 1 )“Drug test failed by drug free, aka, false alarm” P(A 1, B 2 )“Drug test passed by drug user, aka, miss” P(A 2, B 2 )“Drug test failed by drug user, aka, detection”

18. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-18 Conditional Probability Consider two experiments and their joint probability. The event from the first experiment is measured and therefore known. It stands to reason that the event probabilities of the remaining experiment have changed in light of the first experiment’s results. Let A be the measured (given) event, and B the unknown event. The probability of B given A is written as P(B|A) and is known as conditional probability. Example: If a legal battle over firing or non-hiring were to ensue over a drug test, what would be the probability of interest? It would be P(B 2 |A 2 ), that is, the probability of being a drug user if a drug test is failed.

19. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-19 Joint Probability (cont.) S = S A  S B A 1, B 1 = “Pass, Drug Free” A 2, B 2 = “Fail, Drug User”A 1, B 2 = “Pass, Drug User” A 2, B 1 = “Fail, Drug Free”

20. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-20 Multiplication Rule An important relationship exists between joint and conditional probability called the multiplication rule. The multiplication rule can be written both directions which, when P(A) > 0, is more usefully written as,

21. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-21 Conditional Probability (cont.) Example: Using the above to rewrite P(B 2 |A 2 ), P(A 2 |B 2 ) = probability of failing a drug test if a drug user (can be determined by controlled experiments) = 99% effective. P(B 2 ) = probability of being a drug user (can be hard to determine, will cover next) = 2% “general bad apples,” and P(A 2 ) = probability of drug test failure (can be obtained from the history of doing drug testing) = 2.96% failure rate.

22. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-22 Total Probability Rule If {A i : i = 1, …, M} is a partition then Example: From the drug test, P(A 2 ) was given as 2.96%. That number was computed since B 1 and B 2 form a partition, i.e., More realistic would be to know P(A 2 ) and from that compute P(B 2 ) and P(B 1 ) = 1 – P(B 2 ).

23. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-23 Total Probability Rule (cont.)

24. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-24 Bayes’ Theorem Combining a partition with the multiplication rule is known as Bayes’ Theorem (provided P(B) > 0), Example: Consider the case of passing a drug test. What is the probability of being drug free?

25. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-25 Statistical Independence Consider the outcome of flipping a coin vs. today’s price for Lockheed Martin stock. Both are random events. But do they have anything to do with each other? Not likely. These two events are independent from each other. In terms of probability, events are independent if and only if Note the impact of independence on conditional probability

26. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-26 Statistical Independence (cont.) Which of the following players has an advantage: –Player A: Tries to guess the suit of a card drawn randomly from a deck of cards. –Player B: Tries to guess the suit of a card drawn randomly from a deck of cards after being told its face value. Which of the following players has an advantage: –Player A: Tries to guess the suit of a card drawn randomly from a deck of cards. –Player B: Tries to guess the suit of the same card, but only if player A guesses unsuccessfully.

27. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-27 In Class Assignment In the Monte Haul Game show, a particular game was organized as follows: A prize was hidden behind one of three doors. The contestant picked one of the three doors (trying to find the prize, of course). One of the two remaining doors (not picked by the contestant) was opened (always revealing… nothing). The contestant was then asked if he/she wanted to change his/her mind on what door the prize is behind. Should he/she?

28. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-28 In Class Assignment (cont.) Hide Prize Door 2 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N Door 3 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N Door 1 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N Tree Diagram

29. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-29 In Class Assignment (cont.) Hide Prize Door 2 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N Door 3 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N Door 1 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N “Probability of winning if always switch”

30. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-30 In Class Assignment (cont.) Hide Prize Door 2 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N Door 3 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N Door 1 Pick Door Door 2 Switch? Y N Door 1 Switch? Y N Door 3 Switch? Y N “Probability of winning if never switch”

31. EMIS 7300 Copyright  2002 - 2006 Dr. John Lipp S1P2-31 Homework Mandatory (answers in the back of the book): 2-17 2-35 2-63 (part d correct answer = 24/39) 2-73 2-83 2-91 2-97 Optional: 2-123 (answer: 0.344%) 2-125 (answer: 200 kits  $5,500 average earnings / week)