Rational Decisions and Multiagent Systems Rational Decisions and Utility Theory © Manfred Huber 2018

Rationality Rationality describes a decision making criterion A rational agent always takes the action that leads to the “best” outcome for it In multiagent scenarios this implies Actions are not taken to harm other agents Actions are taken in order to maximize the agent’s performance criterion But: actions do not necessarily only consider aspects important to the individual agent © Manfred Huber 2018

Rationality Rationality requires to define “best” In multiagent systems what is “best” can be different for every agent What is “best” depends on the state of the agent What is “best” can be influenced by random effects and uncertainties © Manfred Huber 2018

Decision and Utility Theory Decision Theory deals with optimal decision making in single agent as well as multiagent systems A rational decision agent takes the action that leads to the highest expected payoff Utility Theory deals with the definition of “best”, “better”, and “payoff” Utility defines the expected “payoff” © Manfred Huber 2018

Utility Theory Utility: quantifies the degree of preference across different alternatives models the impact of uncertainty on preferences represents a mapping from an agent’s state (or situation) to the degree of happiness (expected future payoff) for being in this state © Manfred Huber 2018

Utility Theory The application of utility theory requires: All preferences of an agent have to be expressed with a single utility function independent of the complexity of the task, environment, and action set An agent’s decisions in the context of uncertainty are purely determined by the expected value of the utility © Manfred Huber 2018

Rational Preferences To make rational decisions we have to define rational preferences: Outcomes oi o1 is at least as desirable as o2 (the agent weakly prefers o1 to o2) the agent is indifferent between o1 and o2 ( and ) the agent strictly prefers o1 to o2 ( and ) © Manfred Huber 2018

Rational Preferences Preferences have to be expressible in the context of uncertainty: Lottery: specifies the probability of each possible outcome [p1 : o1, p2 : o2, …, pn : on] Lotteries represent uncertain outcomes © Manfred Huber 2018

Rational Preferences For preferences to be rational (more precisely for them to lead to rational decisions) they have to fulfill a number of axioms: Completeness Transitivity Substitutability Decomposability Monotonicity Continuity © Manfred Huber 2018

Preferences - Completeness All outcomes have to be comparable A preference relationship has to be defined for every pair of outcomes © Manfred Huber 2018

Preferences - Transitivity Preferences have to be transitive This implies that there can not be any circular preference relationships © Manfred Huber 2018

Preferences - Substitutability If an agent is indifferent between two outcomes then they can be substituted for each other without change in the preference Outcomes that are indifferent have to be indifferent in all contexts © Manfred Huber 2018

Preferences - Decomposability An agent has to be indifferent between two choices that lead to the same outcome The agent is indifferent between two lotteries that lead to the same outcome probabilities This implies that the agent has no preference for the mechanism that leads to an outcome but only for the outcome itself For example, a probabilistic outcome vs a probabilistic outcome where one choice includes a lottery The agent is indifferent between one thing and gambling as long as it leads to the same outcome © Manfred Huber 2018

Preferences - Monotonicity An agent prefers a larger chance at a better outcome to a smaller chance This also implies that the agent does not have any preference between deterministic and probabilistic outcomes © Manfred Huber 2018

Preferences - Continuity The preference relation of a lottery has to change continuously with the change of the outcome probabilities © Manfred Huber 2018

From Rational Preferences to Utility von Neumann and Morgenstern showed in 1944 that if preferences are rational (i.e. they obey the axioms), then there exists a scalar utility function that quantifies the preferences Proof idea: select best and worst outcomes and assign 0 and 1, find probabilities at which lotteries of these two are indifferent to the other outcomes and set the utilities correspondingly © Manfred Huber 2018

Utility Functions A Utility function allows to quantify preferences for decision making Rational decisions are simply the ones that lead to the largest value of the utility function © Manfred Huber 2018

Utility Functions There are an infinite number of utility functions for each set of rational preferences E.g.: Linear offsets and scaling of the utility function preserves preferences Utility can only be used to compare alternatives The absolute value of the utility is arbitrary The bounds on the values of the utility function are not necessary for rational decision making Utilities can be arbitrary values (as long as they are finite) © Manfred Huber 2018

From Russell and Norvig Utility vs. Money Money does not behave like a utility for humans Humans preferences react to “risk” as well as money For example, a probabilistic outcome vs a probabilistic outcome where one choice includes a lottery The agent is indifferent between one thing and gambling as long as it leads to the same outcome From Russell and Norvig © Manfred Huber 2018

Utility vs. Money That humans do not choose actions that maximize the expected amount of money implies either: Human preferences are not rational (i.e. People are irrational) Human preferences are not based solely on money (i.e. People are rational but money is not the utility function used) Artificial Intelligence usually prefers the second option For example, a probabilistic outcome vs a probabilistic outcome where one choice includes a lottery The agent is indifferent between one thing and gambling as long as it leads to the same outcome © Manfred Huber 2018