1. Decision Making

2. Outline Maximum Expected Utility (MEU) Decision network
Making decisions
Russell & Norvig, chapter 16

3. Acting Under Uncertainty
With no uncertainty, rational decision is to pick action with “best” outcome
Two actions
#1 leads to great outcome
#2 leads to good outcome
It’s only rational to pick #1
Assumes outcome is 100% certain
With uncertainty, it’s a little harder
#1 has 1% probability to lead to great outcome
#2 has 90% probability to lead to good outcome
What is the rational decision?

4. Acting Under Uncertainty
Maximum Expected Utility (MEU)
Pick action that leads to best outcome averaged over all possible outcomes of the action
How do we compute the MEU?
Easy once we know the probability of each outcome and their utility

5. Utility Value of a state or outcome Computed by utility function
U(S) = utility of state S
U(S)  [0,1] if normalized

6. Expected Utility
Sum of utility of each possible outcome times probability of that outcome
Known evidence E about the world
Action A has i possible outcomes, with probability P(Resulti(A)|Do(A),E)
Utility of each outcome is U(Resulti(A))
Evaluation function of the state of the world given Resulti(A)
EU(A|E)=i P(Resulti(A)|Do(A),E) U(Resulti(A))

7. Maximum Expected Utility
List all possible actions Aj
For each action, list all possible outcomes Resulti(Aj)
Compute EU(Aj|E)
Pick action that maximises EU

8. Utility of Money Use money as measure of utility? Example
A1 = 100% chance of $1M
A2 = 50% chance of $3M or nothing
EU(A2) = $1.5M > $1M = EU(A1)
Is that rational?

9. Utility of Money Utility/Money relationship is logarithmic, not linear
Example
EU(A2) = .45 < .46 = EU(A1)
Insurance
EU(paying) = –U(value of premium)
EU(not paying) = U(value of premium) – U(value of house) * P(losing house)

10. Decision Network Our agent makes decisions given evidence
Observed variables and conditional probability tables of hidden variables
Similar to conditional probability
Probability of variables given other variables
Relationships represented graphically in Bayesian network
Could we make a similar graph here?

11. Decision Network Sometimes called influence diagram
Like a Bayesian Network for decision making
Start with variables of problem
Add decision variables that the agent controls
Add utility variable that specify how good each state is

12. Decision Network Chance node (oval) Decision node (rectangle)
Uncertain variable
Like in Bayesian network
Decision node (rectangle)
Choice of action
Parents: variables affecting decision, evidence
Children: variables affected by decision
Utility node (diamond)
Utility function
Parents: variables affecting utility
Typically only one in network

13. Expected Utility

14. Simple Influence Diagram
U(-f)? U(+f)?

15. Decision Network Example
P(E) F
0.01 T
0.5
0.9
0.99 W E H F
0.2 T
0.6
0.8
0.99
Study
Happiness
PassExam
Lucky L
P(W) F
0.01 T
0.4
P(L) = 0.75
Win

16. U(+s)=0.65, U(-s) = 0.46 W E P(W,E|+s) H F 0.00475 0.2 T 0.00325 0.6
0.8
0.99
U(+s)=0.65, U(-s) = 0.46 W E
P(W,E|-s) H F
0.227
0.2 T
0.15
0.6
0.225
0.8
0.99

17. More Complex Influence Diagram

18. Information edge

19. Expected utility with Information

21. Value of perfect information
VPI(A | X) is the value of observing X before choosing an action at A
D = original influence diagram
DX → A = influence diagram with edge X → A
VPI(A|X) = MEU(DX → A )-MEU(D)
VPI=0, >0
برای مثال قبل
3.25-2

22. oil company is hoping to buy one of n indistinguishable
assume further that exactly one of the blocks contains oil worth C dollars, while
the others are worthless.
The asking price of each block is C/n dollars.
Now suppose that a seismologist offers the company the results of a survey of block
number 3. How much should the company be willing to pay for the information?

23. P U
1/n
C-C/n
(n-1)/n
C/(n-1)-C/n

25. Value of Information Utility of decision without inspection is 200
Utility of decision with inspection is 205, utility of the decision minus the utility cost of the inspection
Utility of decision is 255
At what point is the utility cost of the inspection too high?
255 – Utility Cost < 200
Value of the information gained from the inspection is 55

26. Value of Information Information has value if The value is
It causes a change in the decision
The new decision has higher utility than the old one
The value is
Non-negative
Zero for irrelevant facts
Zero for information already known