1. Making Simple Decisions Utility Theory MultiAttribute Utility Functions Decision Networks The Value of Information Summary

2. Beliefs and Uncertainty Utility Function Outcome Probabilities Expected Utility

4. Maximum Expected Utility EU(A|E) = Σ P(Result i (A) | E) U(Result i (A)) Principle of Maximum Expected Utility: Choose action A with highest EU(A|E)

5. Example Robot Turn Right Turn Left Hits wall (P = 0.1; U = 0) Finds target (P = 0.9; U = 10) Fall water (P = 0.3; U = 0) Finds target (P = 0.7; U = 10) Choose action “Turn Right”

6. Notation Utility Theory A > B  A is preferred to B A ~ B  indifferent between A and B A >~ B  A is preferred to or indifferent to B Lottery (or random variable) L = [p 1, S 1 ; p 2, S 2 ; …, p n, S n ] where p:probability and S: outcome

7. Utility Principle Principle U(A) > U(B)   A > B U(A) = U(B)   A ~ B

8. Utility Functions Television Game Show: Assume you already have won $1,000,000 Flip a coin: Tails (P = 0.5)  $3,000,000 Head (P = 0.5)  $0

9. Utility Functions EU(Accept) = 0.5 U(S k ) + 0.5 U(S k + 3M ) EU(Decline) = U(S k + 1M ) Assume: S k = 5 S k + 1M = 8 S k + 3M = 10

10. Utility Functions Then EU(Accept) = 0.5 x 5 + 0.5 x 10 = 7.5 EU(Decline) = 8 Result: Decline offer in view of assigned utilities

12. Risk-Averse Positive part: slope decreasing. Utility is less than expected monetary value $ U

13. Risk-Seeking Negative part: desperate region. $ U Linear curve: risk neutral $ U

14. Connection to AI Choices are as good as the preferences they are based on. If user embeds in our intelligent agents : contradictory preferences Results may be negative reasonable preferences Results may be positive

15. Assessing Utilities Best possible outcome: A max Worst possible outcome: A min Use normalized utilities: U(A max ) = 1 ; U(A min ) = 0

16. Making Simple Decisions Utility Theory MultiAttribute Utility Functions Decision Networks The Value of Information Summary

17. MultiAttribute Utility Functions Outcomes are characterized by more than one attribute: X 1, X 2, …, X n Example: Choosing right map successful trip Finding right equipment unsuccessful trip Acquiring food supplied

18. Simple Case: Dominance Assume higher values of attributes correspond to higher utilities. There are regions of clear “dominance”

20. Stochastic Dominance Plot probability distributions against negative costs. Example: S1: Build airport at site S1 S2: Build airport at site S2

23. Making Simple Decisions Utility Theory MultiAttribute Utility Functions Decision Networks The Value of Information Summary

24. Decision Networks It’s a mechanism to make rational decisions Also called influence diagram Combine Bayesian Networks with other nodes

25. Types of Nodes Chance Nodes. Represent random variables (like BBN) Decision Nodes Choice of action Utility Nodes Represent agent’s utility function

26. Decision Nodes Chance Nodes Utility Nodes

27. Making Simple Decisions Utility Theory MultiAttribute Utility Functions Decision Networks The Value of Information Summary

28. The Value of Information Important aspect of decision making: What questions to ask. Example: Oil company. Wishes to buy n blocks of ocean drilling rights.

29. The Value of Information Exactly one block has oil worth C dollars. The price of each block is C/n. A seismologist offers the results of a survey of block number 3. How much would you pay for the info?

30. The Value of Information With probability 1/n the survey will indicate there is oil in block 3. Buy it for C/n dollars to make a profit of C – C/n = (n-1) C / n With probability (n-1)/n the survey will show no oil. Buy different block. Expected profit is C/(n-1) – C/n = C/n(n-1) dollars.

31. Expected Profit The expected profit given the info is 1/n x (n-1)C / n + (n-1)/n x C / n(n-1) = C/n The info. is worth the price of the block itself.

32. The Value of Information Value of info: Expected improvement in utility compared with making a decision without that information.

33. Making Simple Decisions Utility Theory MultiAttribute Utility Functions Decision Networks The Value of Information Summary

34. Decision theory combines probability and utility theory. A rational agent chooses the action with maximum expected utility. Multiattribute utility theory deals with utilities that depend on several attributes Decision networks extend BBN with additional nodes To solve a problem we need to know the value of information.

35. Video Rover Curiosity explores Mars (decision making is crucial during navigation) https://www.youtube.com/watch?v=W6BdiKIWJhY