1. Nevin L. Zhang Room 3504, phone: 2358-7015,
THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 5220:  Reasoning and Decision under Uncertainty L09: Graphical Models for Decision Problems
Nevin L. Zhang Room 3504, phone: ,
Home page

2. L09: Graphical Models for Decision Problems
Page 2
L09: Graphical Models for Decision Problems
Introduction
Extending BN to Include a Single Decision
Fundamentals of Rational Decision Making
Decision Trees
Influence Diagrams
Solving influence Diagrams
Value of information

3. Probabilistic Reasoning and Decision
Method 1: Two-stage
In a BN, calculate posterior probabilities
Use the posteriors to make decisions
Method 2
Combine the two stages
Extend BN to include decisions
Better reveal structure of decision problem
Compute optimal decisions directly from model
Reasoning: Jensen & Nielsen, Sections , 10.2, 11.1

4. L09: Graphical Models for Decision Problems
Page 4
L09: Graphical Models for Decision Problems
Extending BN to Include a Single Decision
Fundamentals of Rational Decision Making
Decision Trees
Influence Diagrams
Solving influence Diagrams
Value of information

5. Poker
From Lecture 04
Extend the model so that I can calculate the probability that my hand is better than the opponent’s hand
MH: My Hand
BH: Best Hand

6. Fold or Call

7. Fold or Call
Information that I have: FC, SC, MH

8. Modeling One Action Start with a BN
Add the decision node and utility nodes
What information we have when making the decision
What chance and utility variables will the decision influence

9. Including More Decisions
Things become a bit more complicated.
Will see later.

10. L09: Graphical Models for Decision Problems
Extending BN to Include Decisions
Fundamentals of Rational Decision Making
Decision Trees
Influence Diagrams
Solving influence Diagrams
Value of information

11. Decision Theory Normative decision theory
How people should decide. (Rational agent)
Descriptive decision theory
How people actually decide.

12. Normative Decision Theory

13. Are you rational? Lottery A: [$1mill]
Lottery B: 0.5[$2mill] + 0.5[$0mill]
Which one do you choose?
Most people would choose A
U(1) > 0.5 U(2) U(0)
Most people are risk-averse, with concave utility function

14. Are your rational? Suppose that you are $2mill in debt
Lottery A: [$1mill]
Lottery B: 0.5[$2mill] + 0.5[$0mill]
Which one do you choose?
Probably B
U(1) < 0.5 U(2) U(0)
You are being risk-seeking, with convex utility function

15. Utilities without Money
Page 15
Utilities without Money

16. Utilities without Money
Page 16
Utilities without Money

17. Marks as Utilities

18. Other Considerations 2 is passing grade
If fail, can retake and hopefully get a better grade in transcript
In this case, 2 is the worst!

19. L10: Graphical Models for Decision Problems
Extending BN to Include Decisions
Fundamentals of Rational Decision Making
Decision Trees
Influence Diagrams
Solving influence Diagrams
Value of information

20. Decision Trees
Classical way to represent decision problems with multiple decisions
Explicitly show all possible sequences of decisions and observations.
Example: Oil Wildcatter
A wildcatter is a person who drills wildcat wells, which are oil wells drilled in areas not known to be oil fields.
Test on Seismic structure

21. Decision Tree for Oil Wildcatter

22. Decision Trees Decision nodes: Rectangles Chance nodes: ellipses
Utility values: at leaves, some times inside diamonds
To be read from root to leaves
Branches from a decision node: possible actions
Branches from a chance node: possible outcomes and probs
A decision node follows a chance node:
The chance node is observed before the decision is made
No-forgetting
Decision-maker remembers all the labels from root to a decision node
Game between decision maker and nature

23. Solution to a Decision Tree
Strategy: Which decision node to pick at each decision node

24. Solution to a Decision Tree
Optimal Strategy: The strategy with the highest expected utility

25. Solving Decision Trees

26. Example
77.59
77.59

28. L09: Graphical Models for Decision Problems
Extending BN to Include Decisions
Fundamentals of Rational Decision Making
Decision Trees
Influence Diagrams
Solving influence Diagrams
Value of information

29. Extending BN to Include one Decision
Start with a BN
Add the decision node and utility nodes
What information we have when making the decision
What chance and utility variables will the decision influence
To include multiple decision nodes,
Need to consider the interactions among the decisions

30. Including Multiple Decisions
Page 30
Including Multiple Decisions
Two more decisions
MFC: my first change
MSC: my second change

31. Representing the Decision Sequence
First representation
All nodes observed before a decision are parents of that decision.
Information arcs.
Assume that the decision maker doesn’t forget, then some links are redundant.

32. Representing the Decision Sequence
No-forgetting allows a more concise representation
Keep directed path going through all the decision node: Order of decision.
Arrows into a decision node only from those nodes observed immediately before that decision.
Implicit parents: parents of earlier decisions

33. Influence Diagram A DAG with three types of nodes
Chance nodes, decision nodes, and utility nodes
There is a directed path containing all the decision nodes.
The utility nodes have no children.
Each chance node is associated with the conditional distribution given its parents.
Each utility node is associated with a utility function, a real-valued function of its parents.

34. Page 34
Influence Diagram

35. Influence Diagram An influence diagram for the oil wildcatter problem
Page 35
Influence Diagram
An influence diagram for the oil wildcatter problem
Decision: T: test = {y, n}; D: drill={y, n}
Utility: C: cost of test ; V: Benefit of drilling
Chance:
O: Oil ={dry, wet, soaking}
R: seismic structure
{no-structure, open-structure, closed-structure, no-result}

36. L09: Graphical Models for Decision Problems
Extending BN to Include Decisions
Fundamentals of Rational Decision Making
Decision Trees
Influence Diagrams
Solving influence Diagrams
Value of information

37. Strategy (Policy) A policy specifies what to do for each decision
It is a function of observed variables
Different policies lead to different expected utility
Optimal policy: the Policy that yields the maximum expected utility.
How to find the optimal policy?

38. Finding Optimal Policy
First idea:
Convert to decision tree and solve it
How to convert influence diagram into decision tree
Draw tree
Root: the thing that happens first
Children of root: the thing that happens next …
Figure out numerical information

39. Order of events
Tree structure
Numerical info
Prob for branches from chance node
Utility for leaves

40. A Side Note Two decision trees for Oil Wildcatter First Second
directly from problem specification.
Asymmetric
Second
from influence diagram
Symmetric
Pro of ID: compact
Con of ID: cannot represent assymetry
Need to introduce artificial state R = no-result

41. Finding Optimal Policy
Page 41
Finding Optimal Policy
First idea:
Convert to decision tree and solve it
Exponential still!
Next:
Variable Elimination Algorithm for solving influence diagrams
Note
BN inference: All orderings give correct result, but might have different complexity
ID: Must use “strong elimination orderings”.

42. Temporal Order among Decisions and Observations
Notations
Decision nodes have a temporal order: D1, D2, …, Dn
T0: Set of chance nodes observed prior to any decision
Ti: Set of chance nodes observed after Di is taken and before Di+1 is taken
Oil Wildcatter
D1 = T; D2 = D
T0 = {}; T1 = {R}; T2={O}
Partial temporal order
T0, D1, T1, D2, T2, …., Dn, Tn
Oil Wildcatter: T, R, D, O

43. Temporal Order T0={}, T1={T}, T2={A, B, C} Partial temporal ordering
D1, T, D2. {A, B, C}
No ordering among A, B, C

44. Strong Elimination Ordering
Partial temporal order
T0, D1, T1, D2, T2, …., Dn, Tn
Strong elimination orders
First eliminate variables in Tn
Then eliminate Dn
Then eliminate variables in Tn-1
Then eliminate Dn-1
…..
Oil Wildcatter
Temporal order:
T, R, D, O
Strong elimination ordering
O, D, R, T

45. Strong Elimination Ordering
T0={}, T1={T}, T2={A, B, C}
Partial temporal ordering
D1, T, D2. {A, B, C}
No ordering among A, B, C
Strong elimination orderings
A, B, C, D2, T, D1
B, C, A, D2, T, D1
C, A, B, D2, T, D1 ….

46. Variable Elimination Two set of potentials (factors):
Eliminate decision and chance nodes one by one according to a strong elimination ordering.
When eliminate variable X

47. Variable Elimination on Oil Wildcatter
Strong Elimination Ordering: O, D, R, T

48. Variable Elimination on Oil Wildcatter
Eliminate: O

49. Page 49

50. Page 50

51. Potentials after Eliminating O

52. Potentials after Eliminating O

53. Eliminating D No probability potential involves D
Optimal decision for D

54. Potentials after Eliminating D

55. Eliminating R

56. Potentials after Eliminating R

57. Eliminating T Optimal decision for T
Results same as those by decision tree

58. L09: Graphical Models for Decision Problems
Page 58
L09: Graphical Models for Decision Problems
Extending BN to Include Decisions
Fundamentals of Rational Decision Making
Decision Trees
Influence Diagrams
Solving influence Diagrams
Value of information

59. Two types of Decisions Action decisions
Page 59
Two types of Decisions
Action decisions
Result in significant state change of variables of interest
Example:
D: Drill or not to drill
Test decisions
Look for more evidence
T: Test of Seismic structure

60. Two types of Decisions Typical scenario Need to make one decision
Page 60
Two types of Decisions
Typical scenario
Need to make one decision
Want to get more information before making the decision
Question
Is it worthwhile to perform a particular test?
Which test to choose if multiple tests are available?

61. Value of Information What is the value of a test?
Page 61
Value of Information
What is the value of a test?
Create two influence diagrams
Solve both
Compare their values
Example: Oil wildcatter
Is it worthwhile to perform the seismic test?
ID1: without the test
ID2: with the test

62. Value of Information Expected utility of ID2 U(ID2) = 22.55
Page 62
Value of Information
Expected utility of ID2
U(ID2) = 22.55
What is the expected utility of ID1?

63. Expected Utility of ID1 Temporal ordering: D, O
Elimination ordering: O, D
Eliminate O:

64. Expected Utility of ID1 Potentials after eliminating O Eliminate D
Page 64
Expected Utility of ID1
Potentials after eliminating O
Eliminate D
Expected utility of ID1
U(ID1) = 20

65. Value of Information Difference in expected utility
Page 65
Value of Information
Difference in expected utility
U(ID2) – U(ID1) = – 20 = 2.55
The expected value of the seismic test is 2.55
The test is worthwhile

66. Value of Information If there are multiple tests T1, T2, T3, …
Page 66
Value of Information
If there are multiple tests
T1, T2, T3, …
Compute the value of each test, pick the best one
If the value of the best is positive,
Pick the test among remain tests
Stop when value of the selected test is not positive