1. Rational Decisions and
Multiagent Systems
Rational Decisions and
Utility Theory
© Manfred Huber 2018

2. Rationality Rationality describes a decision making criterion
A rational agent always takes the action that leads to the “best” outcome for it
In multiagent scenarios this implies
Actions are not taken to harm other agents
Actions are taken in order to maximize the agent’s performance criterion
But: actions do not necessarily only consider aspects important to the individual agent
© Manfred Huber 2018

3. Rationality Rationality requires to define “best”
In multiagent systems what is “best” can be different for every agent
What is “best” depends on the state of the agent
What is “best” can be influenced by random effects and uncertainties
© Manfred Huber 2018

4. Decision and Utility Theory
Decision Theory deals with optimal decision making in single agent as well as multiagent systems
A rational decision agent takes the action that leads to the highest expected payoff
Utility Theory deals with the definition of “best”, “better”, and “payoff”
Utility defines the expected “payoff”
© Manfred Huber 2018

5. Utility Theory Utility:
quantifies the degree of preference across different alternatives
models the impact of uncertainty on preferences
represents a mapping from an agent’s state (or situation) to the degree of happiness (expected future payoff) for being in this state
© Manfred Huber 2018

6. Utility Theory The application of utility theory requires:
All preferences of an agent have to be expressed with a single utility function independent of the complexity of the task, environment, and action set
An agent’s decisions in the context of uncertainty are purely determined by the expected value of the utility
© Manfred Huber 2018

7. Rational Preferences
To make rational decisions we have to define rational preferences:
Outcomes oi
o1 is at least as desirable as o (the agent weakly prefers o1 to o2)
the agent is indifferent between o1 and o2 ( and )
the agent strictly prefers o1 to o ( and )
© Manfred Huber 2018

8. Rational Preferences
Preferences have to be expressible in the context of uncertainty:
Lottery:
specifies the probability of each possible outcome
[p1 : o1, p2 : o2, …, pn : on]
Lotteries represent uncertain outcomes
© Manfred Huber 2018

9. Rational Preferences
For preferences to be rational (more precisely for them to lead to rational decisions) they have to fulfill a number of axioms:
Completeness
Transitivity
Substitutability
Decomposability
Monotonicity
Continuity
© Manfred Huber 2018

10. Preferences - Completeness
All outcomes have to be comparable
A preference relationship has to be defined for every pair of outcomes
© Manfred Huber 2018

11. Preferences - Transitivity
Preferences have to be transitive
This implies that there can not be any circular preference relationships
© Manfred Huber 2018

12. Preferences - Substitutability
If an agent is indifferent between two outcomes then they can be substituted for each other without change in the preference
Outcomes that are indifferent have to be indifferent in all contexts
© Manfred Huber 2018

13. Preferences - Decomposability
An agent has to be indifferent between two choices that lead to the same outcome
The agent is indifferent between two lotteries that lead to the same outcome probabilities
This implies that the agent has no preference for the mechanism that leads to an outcome but only for the outcome itself
For example, a probabilistic outcome vs a probabilistic outcome where one choice includes a lottery
The agent is indifferent between one thing and gambling as long as it leads to the same outcome
© Manfred Huber 2018

14. Preferences - Monotonicity
An agent prefers a larger chance at a better outcome to a smaller chance
This also implies that the agent does not have any preference between deterministic and probabilistic outcomes
© Manfred Huber 2018

15. Preferences - Continuity
The preference relation of a lottery has to change continuously with the change of the outcome probabilities
© Manfred Huber 2018

16. From Rational Preferences to Utility
von Neumann and Morgenstern showed in 1944 that if preferences are rational (i.e. they obey the axioms), then there exists a scalar utility function that quantifies the preferences
Proof idea: select best and worst outcomes and assign 0 and 1, find probabilities at which lotteries of these two are indifferent to the other outcomes and set the utilities correspondingly
© Manfred Huber 2018

17. Utility Functions
A Utility function allows to quantify preferences for decision making
Rational decisions are simply the ones that lead to the largest value of the utility function
© Manfred Huber 2018

18. Utility Functions
There are an infinite number of utility functions for each set of rational preferences
E.g.: Linear offsets and scaling of the utility function preserves preferences
Utility can only be used to compare alternatives
The absolute value of the utility is arbitrary
The bounds on the values of the utility function are not necessary for rational decision making
Utilities can be arbitrary values (as long as they are finite)
© Manfred Huber 2018

19. From Russell and Norvig
Utility vs. Money
Money does not behave like a utility for humans
Humans preferences react to “risk” as well as money
For example, a probabilistic outcome vs a probabilistic outcome where one choice includes a lottery
The agent is indifferent between one thing and gambling as long as it leads to the same outcome
From Russell and Norvig
© Manfred Huber 2018

20. Utility vs. Money
That humans do not choose actions that maximize the expected amount of money implies either:
Human preferences are not rational (i.e. People are irrational)
Human preferences are not based solely on money (i.e. People are rational but money is not the utility function used)
Artificial Intelligence usually prefers the second option
For example, a probabilistic outcome vs a probabilistic outcome where one choice includes a lottery
The agent is indifferent between one thing and gambling as long as it leads to the same outcome
© Manfred Huber 2018