Presentation is loading. Please wait.

Presentation is loading. Please wait.

Decision Making ECE457 Applied Artificial Intelligence Spring 2007 Lecture #10.

Published byBranden Patterson Modified over 8 years ago

Similar presentations

Presentation on theme: "Decision Making ECE457 Applied Artificial Intelligence Spring 2007 Lecture #10."— Presentation transcript:

1 Decision Making ECE457 Applied Artificial Intelligence Spring 2007 Lecture #10

2 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 2 Outline Maximum Expected Utility (MEU) Decision network Making decisions Russell & Norvig, chapter 16

3 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 3 Acting Under Uncertainty With no uncertainty, rational decision is to pick action with “best” outcome Two actions #1 leads to great outcome #2 leads to good outcome It’s only rational to pick #1 Assumes outcome is 100% certain With uncertainty, it’s a little harder Two actions #1 has 1% probability to lead to great outcome #2 has 90% probability to lead to good outcome What is the rational decision?

4 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 4 Acting Under Uncertainty Maximum Expected Utility (MEU) Pick action that leads to best outcome averaged over all possible outcomes of the action How do we compute the MEU? Easy once we know the probability of each outcome and their utility

5 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 5 Utility Value of a state or outcome Computed by utility function U(S) = utility of state S U(S)  [0,1] if normalized

6 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 6 Expected Utility Sum of utility of all possible outcomes times probability of that outcome Known evidence E about the world Action A has i possible outcomes, with probability P(Result i (A)|Do(A),E) Utility of each outcome is U(Result i (A)) Evaluation function of the state of the world given Result i (A) EU(A|E)=  i P(Result i (A)|Do(A),E) U(Result i (A))

7 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 7 Maximum Expected Utility List all possible actions A j For each action, list all possible outcomes Result i (A j ) Compute EU(A j |E) Pick action that maximises EU

8 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 8 Utility of Money Use money as measure of utility? Example A 1 = 100% chance of $1M A 2 = 50% change of $3M or nothing EU(A 2 ) = $1.5M > $1M = EU(A 1 ) Is that rational?

9 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 9 Utility of Money Utility/Money relationship is logarithmic, not linear Example EU(A 2 ) = 3.1 < 6 = EU(A 1 ) Insurance EU(paying) = –U(value of premium) EU(not paying) = U(value of premium) – U(value of house) * P(losing house)

10 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 10 Axioms Given three states A, B, C A  B The agent prefers A to B A ~ B The agent is indifferent between A and B A  B The agent prefers A to B or is indifferent between A and B [p 1, A; p 2, B; p 3, C] A can occur with probability p 1, B can occur with probability p 2, C can occur with probability p 3

11 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 11 Axioms Orderability (A  B)  (B  A)  (A ~ B) Transitivity (A  B)  (B  C)  (A  C) Continuity A  B  C   p [p, A; 1-p, C] ~ B Substituability A ~ B  [p, A; 1-p, C] ~ [p, B; 1-p, C] Monotonicity A  B  ( p  q  [p, A; 1-p, B]  [q, A; 1-q, B] ) Decomposability [p, A; 1-p, [q, B; 1-q, C]] ~ [p, A; (1-p)q, B; (1-p)(1-q), C]

12 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 12 Axioms Utility principle U(A) > U(B)  A  B U(A) = U(B)  A ~ B Maximum utility principle U([p 1, A 1 ; … ; p n, A n ]) =  i p i U(A i ) Given these axioms, MEU is rational!

13 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 13 Decision Network Our agent makes decisions given evidence Observed variables and conditional probability tables of hidden variables Similar to conditional probability Probability of variables given other variables Relationships represented graphically in Bayesian network Could we make a similar graph here?

14 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 14 Decision Network Sometimes called influence diagram Like a Bayesian Network for decision making Start with variables of problem Add decision variables that the agent controls Add utility variable that specify how good each state is

15 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 15 Decision Network Chance node (oval) Uncertain variable Like in Bayesian network Decision node (rectangle) Choice of action Parents: variables affecting decision, evidence Children: variables affected by decision Utility node (diamond) Utility function Parents: variables affecting utility Typically only one in network

16 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 16 Decision Network Example LuckyStudyPassExamWin Happiness P(L) = 0.75 LP(W) F0.01 T0.4 LSP(E) FF0.01 TF0.5 FT0.9 TT0.99 WEH FF0.2 TF0.6 FT0.8 TT0.99

17 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 17 Decision Network Example Bomber Patio SunnyHave $ U Join your friends Run into friends

18 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 18 Making a Rational Decision At a decision node Given a combination of values of evidence variables, and each possible action given this evidence Compute the EU of each action you can decide to do Decide to do the action with the maximum EU Policy: choice of action (not necessarily the best) for each possible combination of values of evidence variables

19 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 19 Policy Decision node D i Can take values in domain dom(D i ) Has set of parents P i that take values in domain dom(P i ) Policy  is a set of mappings  i of dom(P i ) to dom(D i )  i associates a decision to each state the parents of D i can be in  associates a series of decisions to each state the network can be in

20 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 20 Policy Bomber Patio SunnyHave $ Policy on going to Bomber patio  bp ($,S) = BP  bp (¬$,S) = BP  bp ($,¬S) = ¬BP  bp (¬$,¬S) = ¬BP

21 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 21 Value of a Policy Expected utility if decisions are taken according to the policy EU(  ) =  x P(x) U(x,  (x)) EU(  bp ) =  $,s P($,S) U($,S,  bp ($,S)) Optimal policy  * is the one with the highest expected utility EU(  * )  EU(  ) for all policies 

22 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 22 Computing the Optimal Policy Start from last decision node before utility For each combination of values of a node’s parents Compute the expected utility of each decision Set policy as decision that maximises utility Work backward to the first decision in the network

23 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 23 Computing the Optimal Policy Compute the optimal policy for JF For each combination of BP, RF and $, make a decision JF and compute U(JF,$) Set the policy as the max utility decision for each combination of BP and RF Bomber Patio SunnyHave $ U Join your friends Run into friends

24 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 24 Computing the Optimal Policy Compute the optimal policy for BP given  JF (BP,RF,$) For each combination of S and $, make a decision BP, which will affect RF and JF JF is decided by optimal policy So we can compute U(JF,$) Bomber Patio SunnyHave $ U Join your friends Run into friends

25 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 25 Decision Network Example Bob wants to buy a used car. Unfortunately, the car he’s considering has a 50% chance of being a lemon. Before buying, he can decide to take the car to a mechanic to have it inspected. The mechanic will report if the car is good or bad, but he can make mistakes, and the inspection is expensive. Bob prefers owning a good car to not owning a car, and prefers that to owning a lemon. Should Bob have the car inspected first or not?

26 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 26 Decision Network Example Lemon Inspect U BuyReport

27 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 27 Decision Network Example P(L) = 0.5 Lemon Inspect U Buy Report lbU FF-300 TF FT1000 TT-600 liP(G)P(¬G)P(N) FF001 TF001 FT0.90.10 TT0.20.80 Utility cost of inspection = -50

28 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 28 Decision Network Example Compute EU of Buy and Not Buy given all combinations of evidence Select action with MEU given each case Compute EU of Inspect and Not Inspect given all combinations of evidence and then select Buy/Not Buy action Decide on Inspect or Not Inspect, depending on MEU

29 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 29 Decision Network Example Compute the expected utility of buying and not buying the car given the evidence The evidence is whether or not Bob got the car inspected, and what the result of the inspection is EU(b|i,r) =  l P(l|b,i,r)U(b,i,l)

30 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 30 Decision Network Example EU(B|¬I,N) =  l P(l|B,¬I,N)U(B,¬I,l) EU(B|¬I,N) = P(L)U(B,L,¬I) + P(¬L)U(B,¬L,¬I) EU(B|¬I,N) = 0.5 * -600 + 0.5 * 1000 EU(B|¬I,N) = 200 EU(¬B|¬I,N) =  l P(l|¬B,¬I,N)U(¬B,¬I,l) EU(¬B|¬I,N) = P(L)U(¬B,L,¬I) + P(¬L)U(¬B,¬L,¬I) EU(¬B|¬I,N) = 0.5 * -300 + 0.5 * -300 EU(¬B|¬I,N) = -300 Rational decision, if Bob doesn’t get the car inspected, is to buy it

31 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 31 Decision Network Example EU(B|I,G) =  l P(l|B,I,G)U(B,I,l) We’re missing some information! From the network, we know P(L) and P(G|L), but not P(L|G) nor P(G) Compute P(G) using marginalization P(G) = P(G|L)P(L) + P(G|¬L)P(¬L) = 0.55 Compute P(L|G) using Bayes’ Theorem P(L|G) = P(G|L)P(L)/P(G) = 0.18 P(¬L|G) = P(G|¬L)P(¬L)/P(G) = 0.82

32 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 32 Decision Network Example EU(B|I,G) =  l P(l|B,I,G)U(B,I,l) EU(B|I,G) = P(L|G)U(B,L,I) + P(¬L|G)U(B,¬L,I) EU(B|I,G) = 0.18 * -650 + 0.82 * 950 EU(B|I,G) = 662 EU(¬B|I,G) =  l P(l|¬B,I,G)U(¬B,I,l) EU(¬B|I,G) = P(L|G)U(¬B,L,I) + P(¬L|G)U(¬B,¬L,I) EU(¬B|I,G) = 0.18 * -350 + 0.82 * -350 EU(¬B|I,G) = -350 Rational decision, if Bob gets the car inspected and the report says it’s good, is to buy it

33 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 33 Decision Network Example EU(B|I,¬G) =  l P(l|B,I,  G)U(B,I,l) EU(B|I,¬G) = P(L|  G)U(B,L,I) + P(¬L|  G)U(B,¬L,I) EU(B|I,¬G) = 0.89 * -650 + 0.11 * 950 EU(B|I,¬G) = -474 EU(¬B|I,¬G) =  l P(l|¬B,I,¬G)U(¬B,I,l) EU(¬B|I,¬G) = P(L|¬G)U(¬B,L,I) + P(¬L|¬G)U(¬B,¬L,I) EU(¬B|I,¬G) = 0.89 * -350 + 0.11 * -350 EU(¬B|I,¬G) = -350 Rational decision, if Bob gets the car inspected and the report says it’s not good, is to not buy it

34 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 34 Decision Network Example Should Bob get the car inspected? EU(i) =  l,r P(l,r|i)U(l,i,b) P(l,r|i) = P(r|l,i)P(l|i) = P(r|l,i)P(l) EU(i) =  l,r P(r|l,i)P(l)U(l,i,b) EU(¬I) = P(N|L,¬I)P(L)U(L,¬I,B) + P(N|¬L,¬I)P(¬L)U(¬L,¬I,B) EU(¬I) = 1 * 0.5 * -600 + 1 * 0.5 * 1000 EU(¬I) = 200

35 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 35 Decision Network Example EU(I) = P(G|L,I)P(L)U(L,I,B) + P(G|¬L,I)P(¬L)U(¬L,I,B) + P(¬G|L,I)P(L)U(L,I,¬B) + P(¬G|¬L,I)P(¬L)U(¬L,I,¬B) EU(I) = 0.2 * 0.5 * -650 + 0.9 * 0.5 * 950 + 0.8 * 0.5 * -350 + 0.1 * 0.5 * -350 EU(I) = 205

36 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 36 Decision Network Example EU(I) = 205 > EU(¬I) = 200 Therefore, Bob should get the car inspected

37 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 37 Value of Information Utility of decision without inspection is 200 Utility of decision with inspection is 205, utility of the decision minus the utility cost of the inspection Utility of decision is 255 At what point is the utility cost of the inspection too high? 255 – Utility Cost < 200 Value of the information gained from the inspection is 55

38 ECE457 Applied Artificial Intelligence R. Khoury (2007)Page 38 Value of Information Information has value if It causes a change in the decision The new decision has higher utility than the old one The value is Non-negative Zero for irrelevant facts Zero for information already known

Download ppt "Decision Making ECE457 Applied Artificial Intelligence Spring 2007 Lecture #10."

Similar presentations

Utility theory U: O-> R (utility maps from outcomes to a real number) represents preferences over outcomes ~ means indifference We need a way to talk about.

Utility theory U: O-> R (utility maps from outcomes to a real number) represents preferences over outcomes ~ means indifference We need a way to talk about.
Making Simple Decisions Chapter 16 Some material borrowed from Jean-Claude Latombe and Daphne Koller by way of Marie desJadines,

Making Simple Decisions Chapter 16 Some material borrowed from Jean-Claude Latombe and Daphne Koller by way of Marie desJadines,
Making Simple Decisions

Making Simple Decisions
Representations for KBS: Uncertainty & Decision Support

Representations for KBS: Uncertainty & Decision Support
Utility Theory.

Utility Theory.
THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 5220: Reasoning and Decision under Uncertainty L09: Graphical Models for Decision Problems Nevin.

THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 5220: Reasoning and Decision under Uncertainty L09: Graphical Models for Decision Problems Nevin.
Reasoning Under Uncertainty: Bayesian networks intro Jim Little Uncertainty 4 November 7, 2014 Textbook §6.3, 6.3.1, 6.5, 6.5.1, 6.5.2.

Reasoning Under Uncertainty: Bayesian networks intro Jim Little Uncertainty 4 November 7, 2014 Textbook §6.3, 6.3.1, 6.5, 6.5.1,
1 Utility Theory. 2 Option 1: bet that pays $5,000,000 if a coin flipped comes up tails you get $0 if the coin comes up heads. Option 2: get $2,000,000.

1 Utility Theory. 2 Option 1: bet that pays $5,000,000 if a coin flipped comes up tails you get $0 if the coin comes up heads. Option 2: get $2,000,000.
CPSC 502, Lecture 11Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 11 Oct, 18, 2011.

CPSC 502, Lecture 11Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 11 Oct, 18, 2011.
Decision Theory: Single Stage Decisions Computer Science cpsc322, Lecture 33 (Textbook Chpt 9.2) March, 30, 2009.

Decision Theory: Single Stage Decisions Computer Science cpsc322, Lecture 33 (Textbook Chpt 9.2) March, 30, 2009.
10/19. Survey Summary 18 filled the survey – Will make the full results available by email Most seem happy with the lectures Most say they spend 2- 6.

10/19. Survey Summary 18 filled the survey – Will make the full results available by Most seem happy with the lectures Most say they spend 2- 6.
1 Chapter 12 Probabilistic Reasoning and Bayesian Belief Networks.

1 Chapter 12 Probabilistic Reasoning and Bayesian Belief Networks.
Uncertainty and Consumer Behavior

Uncertainty and Consumer Behavior
Making Simple Decisions Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 16.

Making Simple Decisions Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 16.
Decision Making Under Uncertainty Russell and Norvig: ch 16, 17 CMSC421 – Fall 2005.

Decision Making Under Uncertainty Russell and Norvig: ch 16, 17 CMSC421 – Fall 2005.
1 Bayesian Reasoning Chapter 13 CMSC 471 Adapted from slides by Tim Finin and Marie desJardins.

1 Bayesian Reasoning Chapter 13 CMSC 471 Adapted from slides by Tim Finin and Marie desJardins.
Decision Making Under Uncertainty Russell and Norvig: ch 16 CMSC421 – Fall 2006.

Decision Making Under Uncertainty Russell and Norvig: ch 16 CMSC421 – Fall 2006.
5/25/2005EE562 EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005.

5/25/2005EE562 EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005.
Decision Making Under Uncertainty Russell and Norvig: ch 16, 17 CMSC421 – Fall 2003 material from Jean-Claude Latombe, and Daphne Koller.

Decision Making Under Uncertainty Russell and Norvig: ch 16, 17 CMSC421 – Fall 2003 material from Jean-Claude Latombe, and Daphne Koller.
CMSC 671 Fall 2003 Class #26 – Wednesday, November 26 Russell & Norvig 16.1 – 16.5 Some material borrowed from Jean-Claude Latombe and Daphne Koller by.

CMSC 671 Fall 2003 Class #26 – Wednesday, November 26 Russell & Norvig 16.1 – 16.5 Some material borrowed from Jean-Claude Latombe and Daphne Koller by.

Similar presentations

Ads by Google