ECE457 Applied Artificial Intelligence Spring 2008 Lecture #10

Slides:



Advertisements
Similar presentations
Making Simple Decisions Chapter 16 Some material borrowed from Jean-Claude Latombe and Daphne Koller by way of Marie desJadines,
Advertisements

Making Simple Decisions
Representations for KBS: Uncertainty & Decision Support
THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 5220: Reasoning and Decision under Uncertainty L09: Graphical Models for Decision Problems Nevin.
Causal and Bayesian Network (Chapter 2) Book: Bayesian Networks and Decision Graphs Author: Finn V. Jensen, Thomas D. Nielsen CSE 655 Probabilistic Reasoning.
CPSC 502, Lecture 11Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 11 Oct, 18, 2011.
Decision Theory: Single Stage Decisions Computer Science cpsc322, Lecture 33 (Textbook Chpt 9.2) March, 30, 2009.
1 Chapter 12 Probabilistic Reasoning and Bayesian Belief Networks.
Bayesian Networks. Motivation The conditional independence assumption made by naïve Bayes classifiers may seem to rigid, especially for classification.
Making Simple Decisions Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 16.
Decision Making Under Uncertainty Russell and Norvig: ch 16, 17 CMSC421 – Fall 2005.
1 Bayesian Reasoning Chapter 13 CMSC 471 Adapted from slides by Tim Finin and Marie desJardins.
Decision Making Under Uncertainty Russell and Norvig: ch 16 CMSC421 – Fall 2006.
Decision Making Under Uncertainty Russell and Norvig: ch 16, 17 CMSC421 – Fall 2003 material from Jean-Claude Latombe, and Daphne Koller.
Uncertainty Chapter 13.
CMSC 671 Fall 2003 Class #26 – Wednesday, November 26 Russell & Norvig 16.1 – 16.5 Some material borrowed from Jean-Claude Latombe and Daphne Koller by.
CSCI 121 Special Topics: Bayesian Network Lecture #1: Reasoning Under Uncertainty.
Reminder Midterm Mar 7 Project 2 deadline Mar 18 midnight in-class
CS 4100 Artificial Intelligence Prof. C. Hafner Class Notes March 13, 2012.
Bayesian networks. Motivation We saw that the full joint probability can be used to answer any question about the domain, but can become intractable as.
Probabilistic Reasoning ECE457 Applied Artificial Intelligence Spring 2007 Lecture #9.
Reasoning in Uncertain Situations
An Introduction to Artificial Intelligence Chapter 13 & : Uncertainty & Bayesian Networks Ramin Halavati
1 Reasoning Under Uncertainty Artificial Intelligence Chapter 9.
1 Chapter 12 Probabilistic Reasoning and Bayesian Belief Networks.
Decision Making Under Uncertainty CMSC 671 – Fall 2010 R&N, Chapters , , material from Lise Getoor, Jean-Claude Latombe, and.
Uncertainty ECE457 Applied Artificial Intelligence Spring 2007 Lecture #8.
Making Simple Decisions Utility Theory MultiAttribute Utility Functions Decision Networks The Value of Information Summary.
Chapter 16: Making Simple Decision March 23, 2004.
Decision Making Under Uncertainty CMSC 471 – Spring 2014 Class #12– Thursday, March 6 R&N, Chapters , material from Lise Getoor, Jean-Claude.
1 CSC 384 Lecture Slides (c) , C. Boutilier and P. Poupart CSC384: Lecture 25  Last time Decision trees and decision networks  Today wrap up.
Bayesian Inference Artificial Intelligence CMSC February 26, 2002.
Web-Mining Agents Agents and Rational Behavior Decision-Making under Uncertainty Simple Decisions Ralf Möller Universität zu Lübeck Institut für Informationssysteme.
Making Simple Decisions Chapter 16 Some material borrowed from Jean-Claude Latombe and Daphne Koller by way of Marie desJadines,
Chapter 16 March 25, Probability Theory: What an agent should believe based on the evidence Utility Theory: What the agent wants Decision Theory:
1 Automated Planning and Decision Making 2007 Automated Planning and Decision Making Prof. Ronen Brafman Various Subjects.
Decision Making ECE457 Applied Artificial Intelligence Spring 2007 Lecture #10.
Chapter 12. Probability Reasoning Fall 2013 Comp3710 Artificial Intelligence Computing Science Thompson Rivers University.
ECE457 Applied Artificial Intelligence Fall 2007 Lecture #8
CS 188: Artificial Intelligence Spring 2006
Utilities and Decision Theory
Nevin L. Zhang Room 3504, phone: ,
Ralf Möller Universität zu Lübeck Institut für Informationssysteme
Making Simple Decisions
ECE457 Applied Artificial Intelligence Fall 2007 Lecture #10
Chapter 10 (part 3): Using Uncertain Knowledge
Read R&N Ch Next lecture: Read R&N
Operations Management
Where are we in CS 440? Now leaving: sequential, deterministic reasoning Entering: probabilistic reasoning and machine learning.
Markov Decision Processes
CS 4/527: Artificial Intelligence
Decision Theory: Single Stage Decisions
MNG221- Management Science –
Read R&N Ch Next lecture: Read R&N
Rational Decisions and
Propagation Algorithm in Bayesian Networks
CS 188: Artificial Intelligence Fall 2008
CS 188: Artificial Intelligence Fall 2007
Utility Theory Decision Theory.
Decision Theory: Single Stage Decisions
13. Acting under Uncertainty Wolfram Burgard and Bernhard Nebel
Class #21 – Monday, November 10
Making Simple Decisions
Behavioral Finance Economics 437.
Read R&N Ch Next lecture: Read R&N
Utilities and Decision Theory
Utilities and Decision Theory
Decision Making.
Making Simple Decisions
ECE457 Applied Artificial Intelligence Spring 2008 Lecture #8
Presentation transcript:

ECE457 Applied Artificial Intelligence Spring 2008 Lecture #10 Decision Making ECE457 Applied Artificial Intelligence Spring 2008 Lecture #10

Outline Maximum Expected Utility (MEU) Decision network Making decisions Russell & Norvig, chapter 16 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 2

Acting Under Uncertainty With no uncertainty, rational decision is to pick action with “best” outcome Two actions #1 leads to great outcome #2 leads to good outcome It’s only rational to pick #1 Assumes outcome is 100% certain With uncertainty, it’s a little harder #1 has 1% probability to lead to great outcome #2 has 90% probability to lead to good outcome What is the rational decision? ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 3

Acting Under Uncertainty Maximum Expected Utility (MEU) Pick action that leads to best outcome averaged over all possible outcomes of the action How do we compute the MEU? Easy once we know the probability of each outcome and their utility ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 4

Utility Value of a state or outcome Computed by utility function U(S) = utility of state S U(S)  [0,1] if normalized ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 5

Expected Utility Sum of utility of each possible outcome times probability of that outcome Known evidence E about the world Action A has i possible outcomes, with probability P(Resulti(A)|Do(A),E) Utility of each outcome is U(Resulti(A)) Evaluation function of the state of the world given Resulti(A) EU(A|E)=i P(Resulti(A)|Do(A),E) U(Resulti(A)) ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 6

Maximum Expected Utility List all possible actions Aj For each action, list all possible outcomes Resulti(Aj) Compute EU(Aj|E) Pick action that maximises EU ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 7

Utility of Money Use money as measure of utility? Example A1 = 100% chance of $1M A2 = 50% chance of $3M or nothing EU(A2) = $1.5M > $1M = EU(A1) Is that rational? ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 8

Utility of Money Utility/Money relationship is logarithmic, not linear Example EU(A2) = .45 < .46 = EU(A1) Insurance EU(paying) = –U(value of premium) EU(not paying) = U(value of premium) – U(value of house) * P(losing house) ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 9

Axioms Given three states A, B, C A  B A ~ B A  B The agent prefers A to B A ~ B The agent is indifferent between A and B A  B The agent prefers A to B or is indifferent between A and B [p1, A; p2, B; p3, C] A can occur with probability p1, B can occur with probability p2, C can occur with probability p3 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 10

Axioms Orderability Transitivity Continuity Substituability (A  B)  (B  A)  (A ~ B) Transitivity (A  B)  (B  C)  (A  C) Continuity A  B  C  p [p, A; 1-p, C] ~ B Substituability A ~ B  [p, A; 1-p, C] ~ [p, B; 1-p, C] Monotonicity A  B  ( p  q  [p, A; 1-p, B]  [q, A; 1-q, B] ) Decomposability [p, A; 1-p, [q, B; 1-q, C]] ~ [p, A; (1-p)q, B; (1-p)(1-q), C] ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 11

Axioms Utility principle Maximum utility principle U(A) > U(B)  A  B U(A) = U(B)  A ~ B Maximum utility principle U([p1, A1; … ; pn, An]) = i piU(Ai) Given these axioms, MEU is rational! ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 12

Decision Network Our agent makes decisions given evidence Observed variables and conditional probability tables of hidden variables Similar to conditional probability Probability of variables given other variables Relationships represented graphically in Bayesian network Could we make a similar graph here? ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 13

Decision Network Sometimes called influence diagram Like a Bayesian Network for decision making Start with variables of problem Add decision variables that the agent controls Add utility variable that specify how good each state is ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 14

Decision Network Chance node (oval) Decision node (rectangle) Uncertain variable Like in Bayesian network Decision node (rectangle) Choice of action Parents: variables affecting decision, evidence Children: variables affected by decision Utility node (diamond) Utility function Parents: variables affecting utility Typically only one in network ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 15

Decision Network Example P(E) F 0.01 T 0.5 0.9 0.99 W E H F 0.2 T 0.6 0.8 0.99 Study Happiness PassExam Lucky L P(W) F 0.01 T 0.4 P(L) = 0.75 Win ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 16

Decision Network Example Run into friends Sunny Bomber Patio Join your friends Have $ U ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 17

Making a Rational Decision At a decision node Given a combination of values of evidence variables, and each possible action given this evidence Compute the EU of each action you can decide to do Decide to do the action with the maximum EU Policy: choice of action (not necessarily the best) for each possible combination of values of evidence variables ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 18

Policy Decision node Di Can take values in domain dom(Di) Has set of parents Pi that take values in domain dom(Pi) Policy  is a set of mappings i of dom(Pi) to dom(Di) i associates a decision to each state the parents of Di can be in  associates a series of decisions to each state the network can be in ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 19

Policy Have $ Bomber Patio Sunny Policy on going to Bomber patio bp($,S) = BP bp(¬$,S) = BP bp($,¬S) = ¬BP bp(¬$,¬S) = ¬BP ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 20

Value of a Policy Expected utility if decisions are taken according to the policy EU() = x P(x) U(x,(x)) EU(bp) = $,s P($,S) U($,S,bp($,S)) Optimal policy * is the one with the highest expected utility EU(*)  EU() for all policies  ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 21

Computing the Optimal Policy Start from last decision node before utility For each combination of values of a node’s parents Compute the expected utility of each decision Set policy as decision that maximises utility Work backward to the first decision in the network ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 22

Computing the Optimal Policy Run into friends Sunny Bomber Patio Have $ Join your friends U Compute the optimal policy for JF For each combination of BP, RF and $, make a decision JF and compute U(JF,$) Set the policy as the max utility decision for each combination of BP and RF ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 23

Computing the Optimal Policy Run into friends Sunny Bomber Patio Have $ Join your friends U Compute the optimal policy for BP given JF(BP,RF) For each combination of S and $, make a decision BP, which will affect RF and JF JF is decided by optimal policy So we can compute U(JF,$) ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 24

Decision Network Example Bob wants to buy a used car. Unfortunately, the car he’s considering has a 50% chance of being a lemon. Before buying, he can decide to take the car to a mechanic to have it inspected. The mechanic will report if the car is good or bad, but he can make mistakes, and the inspection is expensive. Bob prefers owning a good car to not owning a car, and prefers that to owning a lemon. Should Bob have the car inspected first or not? ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 25

Decision Network Example Report Buy Inspect Lemon U ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 26

Decision Network Example P(G) P(¬G) P(N) F 1 T 0.9 0.1 0.2 0.8 Utility cost of inspection = -50 l b i U F -300 T 1000 -600 -350 950 -650 Report Buy Inspect Lemon U P(L) = 0.5 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 27

Decision Network Example Compute EU of Buy and Not Buy given all combinations of evidence Select action with MEU given each case Compute EU of Inspect and Not Inspect given all combinations of evidence and then select Buy/Not Buy action Decide on Inspect or Not Inspect, depending on MEU ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 28

Decision Network Example Compute the expected utility of buying and not buying the car given the evidence The evidence is whether or not Bob got the car inspected, and what the result of the inspection is EU(b|i,r) = l P(l|b,i,r)U(b,i,l) ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 29

Decision Network Example EU(B|¬I,N) = l P(l|B,¬I,N)U(B,¬I,l) EU(B|¬I,N) = P(L)U(B,L,¬I) + P(¬L)U(B,¬L,¬I) EU(B|¬I,N) = 0.5 * -600 + 0.5 * 1000 EU(B|¬I,N) = 200 EU(¬B|¬I,N) = l P(l|¬B,¬I,N)U(¬B,¬I,l) EU(¬B|¬I,N) = P(L)U(¬B,L,¬I) + P(¬L)U(¬B,¬L,¬I) EU(¬B|¬I,N) = 0.5 * -300 + 0.5 * -300 EU(¬B|¬I,N) = -300 Rational decision, if Bob doesn’t get the car inspected, is to buy it ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 30

Decision Network Example EU(B|I,G) = l P(l|B,I,G)U(B,I,l) We’re missing some information! From the network, we know P(L) and P(G|L), but not P(L|G) nor P(G) Compute P(G) using marginalization P(G) = P(G|L)P(L) + P(G|¬L)P(¬L) = 0.55 Compute P(L|G) using Bayes’ Theorem P(L|G) = P(G|L)P(L)/P(G) = 0.18 P(¬L|G) = P(G|¬L)P(¬L)/P(G) = 0.82 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 31

Decision Network Example EU(B|I,G) = l P(l|B,I,G)U(B,I,l) EU(B|I,G) = P(L|G)U(B,L,I) + P(¬L|G)U(B,¬L,I) EU(B|I,G) = 0.18 * -650 + 0.82 * 950 EU(B|I,G) = 662 EU(¬B|I,G) = l P(l|¬B,I,G)U(¬B,I,l) EU(¬B|I,G) = P(L|G)U(¬B,L,I) + P(¬L|G)U(¬B,¬L,I) EU(¬B|I,G) = 0.18 * -350 + 0.82 * -350 EU(¬B|I,G) = -350 Rational decision, if Bob gets the car inspected and the report says it’s good, is to buy it ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 32

Decision Network Example EU(B|I,¬G) = l P(l|B,I,G)U(B,I,l) EU(B|I,¬G) = P(L|G)U(B,L,I) + P(¬L|G)U(B,¬L,I) EU(B|I,¬G) = 0.89 * -650 + 0.11 * 950 EU(B|I,¬G) = -474 EU(¬B|I,¬G) = l P(l|¬B,I,¬G)U(¬B,I,l) EU(¬B|I,¬G) = P(L|¬G)U(¬B,L,I) + P(¬L|¬G)U(¬B,¬L,I) EU(¬B|I,¬G) = 0.89 * -350 + 0.11 * -350 EU(¬B|I,¬G) = -350 Rational decision, if Bob gets the car inspected and the report says it’s not good, is to not buy it ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 33

Decision Network Example *B(I,R) EU ¬I N B 200 G 662 ¬G ¬B -350 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 34

Decision Network Example Should Bob get the car inspected? EU(i) = l,r P(l,r|i)U(l,i,b) P(l,r|i) = P(r|l,i)P(l|i) = P(r|l,i)P(l) EU(i) = l,r P(r|l,i)P(l)U(l,i,b) ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 35

Decision Network Example EU(i) = l,r P(r|l,i)P(l)U(l,i,b) EU(¬I) = P(N|L,¬I)P(L)U(L,¬I,B) + P(N|¬L,¬I)P(¬L)U(¬L,¬I,B) EU(¬I) = 1 * 0.5 * -600 + 1 * 0.5 * 1000 EU(¬I) = 200 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 36

Decision Network Example EU(i) = l,r P(r|l,i)P(l)U(l,i,b) EU(I) = P(G|L,I)P(L)U(L,I,B) + P(G|¬L,I)P(¬L)U(¬L,I,B) + P(¬G|L,I)P(L)U(L,I,¬B) + P(¬G|¬L,I)P(¬L)U(¬L,I,¬B) EU(I) = 0.2 * 0.5 * -650 + 0.9 * 0.5 * 950 + 0.8 * 0.5 * -350 + 0.1 * 0.5 * -350 EU(I) = 205 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 37

Decision Network Example EU(I) = 205 > EU(¬I) = 200 Therefore, Bob should get the car inspected * = { *B(I,R) , *I } *I EU I 205 I R *B(I,R) EU ¬I N B 200 G 662 ¬G ¬B -350 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 38

Value of Information Utility of decision without inspection is 200 Utility of decision with inspection is 205, utility of the decision minus the utility cost of the inspection Utility of decision is 255 At what point is the utility cost of the inspection too high? 255 – Utility Cost < 200 Value of the information gained from the inspection is 55 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 39

Value of Information Information has value if The value is It causes a change in the decision The new decision has higher utility than the old one The value is Non-negative Zero for irrelevant facts Zero for information already known ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 40

Categories of AI Humanly Rationally Think Act Logic and probabilistic reasoning Act Tree searching, iterative improvement ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 41

Exercise Accident (A) Pr W A U T S 0.4 F 0.7 L 0.3 0.6 0.1 1 0.9 U 0.9 U Wear Protection (Pr) Which Way (W) W P(A) S 0.6 L 0.3 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 42

Exercise Which way to go if you wear protection? EU( S | Pr ) = P(A|S) * U(Pr,S,A) + P(~A|S) * U(Pr,S,~A) EU( S | Pr ) = 0.6 * 0.4 + 0.4 * 0.7 EU( S | Pr ) = 0.52 EU( L | Pr ) = P(A|L) * U(Pr,L,A) + P(~A|L) * U(Pr,L,~A) EU( L | Pr ) = 0.3 * 0.3 + 0.7 * 0.6 EU( L | Pr ) = 0.51 Rational decision, if you wear protection, is to go the short way ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 43

Exercise Which way to go if you don’t wear protection? EU( S | ~Pr ) = P(A|S) * U(~Pr,S,A) + P(~A|S) * U(~Pr,S,~A) EU( S | ~Pr ) = 0.6 * 0.1 + 0.4 * 1 EU( S | ~Pr ) = 0.46 EU( L | ~Pr ) = P(A|L) * U(~Pr,L,A) + P(~A|L) * U(~Pr,L,~A) EU( L | ~Pr ) = 0.3 * 0 + 0.7 * 0.9 EU( L | ~Pr ) = 0.63 Rational decision, if you don’t wear protection, is to go the long way ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 44

Exercise Given the decision on which way to go, should we wear protection? EU( Pr ) = P(A|S) * U(Pr,S,A) + P(~A|S) * U(Pr,S,~A) EU( Pr ) = 0.6 * 0.4 + 0.4 * 0.7 EU( Pr ) = 0.52 EU( ~Pr ) = P(A|L) * U(~Pr,L,A) + P(~A|L) * U(~Pr,L,~A) EU( ~Pr ) = 0.3 * 0 + 0.7 * 0.9 EU( ~Pr ) = 0.63 Rational decision is to not wear protection And therefore go the long way ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 45