Agent Technology for e-Commerce Appendix A: Introduction to Decision Theory Maria Fasli

Appendix A Agent Technology for e-Commerce 2 Decision theory Decision theory is the study of making decisions that have a significant impact Decision-making is distinguished into:  Decision-making under certainty  Decision-making under noncertainty  Decision-making under risk  Decision-making under uncertainty

Appendix A Agent Technology for e-Commerce 3 Probability theory  Most decisions have to be taken in the presence of uncertainty  Probability theory quantifies uncertainty regarding the occurrence of events or states of the world  Basic elements of probability theory:  Random variables describe aspects of the world whose state is initially unknown  Each random variable has a domain of values that it can take on (discrete, boolean, continuous)  An atomic event is a complete specification of the state of the world, i.e. an assignment of values to variables of which the world is composed

Appendix A Agent Technology for e-Commerce 4 Probability space  The sample space S={e 1,e 2,…,e n } which is a set of atomic events  The probability measure P which assigns a real number between 0 and 1 to the members of the sample space Axioms  All probabilities are between 0 and 1  The sum of probabilities for the atomic events of a probability space must sum up to 1  The certain event S (the sample space itself) has probability 1, and the impossible event which never occurs, probability 0

Appendix A Agent Technology for e-Commerce 5 Prior probability  In the absence of any other information, a random variable is assigned a degree of belief called unconditional or prior probability  P(X) denotes the vector consisting of the probabilities of all possible values that a random variable can take  If more than one variable is considered, then we have joint probability distributions  Lottery: a probability distribution over a set of outcomes L=[p 1,o 1 ;p 2,o 2 ;…;p n,o n ]

Appendix A Agent Technology for e-Commerce 6 Conditional probability  When we have information concerning previously unknown random variables then we use posterior or conditional probabilities: P(a|b) the probability of a given that we know b  Alternatively this can be written (the product rule): P(a  b)=P(a|b)P(b)  Independence P(a|b)=P(a) and P(b|a)=P(b) or P(a  b)=P(a)P(b)

Appendix A Agent Technology for e-Commerce 7 Bayes’ rule The product rule can be written as: P(a  b)=P(a|b)P(b) P(a  b)=P(b|a)P(a) By equating the right-hand sides: This is known as Bayes’ rule

Appendix A Agent Technology for e-Commerce 8 Making decisions Simple example: to take or not my umbrella on my way out The consequences of decisions can be expressed in terms of payoffs Payoff table Loss table

Appendix A Agent Technology for e-Commerce 9 An alternative representation of payoffs – tree diagram

Appendix A Agent Technology for e-Commerce 10 Admissibility  An action is said to dominate another, if for each possible state of the world the first action leads to at least as high a payoff (or at least as small a loss) as the second one, and there is at least one state of the world in which the first action leads to a higher payoff (or smaller loss) than the second one  If one action dominates another, then the latter should never be selected and it is called inadmissible Payoff table

Appendix A Agent Technology for e-Commerce 11 Non-probabilistic decision-making under uncertainty  The maximin rule  The maximax rule  The minimax loss Payoff table Loss table

Appendix A Agent Technology for e-Commerce 12 Probabilistic decision-making under uncertainty  The Expected Payoff (ER) rule dictates that the action with the highest expected payoff should be chosen  The Expected Loss (EL) rule dictates that the action with the smallest expected loss should be chosen If P(rain)=0.7 and P(not rain)=0.3 then: ER(carry umbrella) = 0.7(-£1)+0.3(-£1)=-£1 ER(not carry umbrella) = 0.7(-£50)+0.3(-£0)=-£35 EL(carry umbrella) = 0.7(£0)+0.3(£1)=£0.3 EL(not carry umbrella) = 0.7(£49)+0.3(£0)=£34.3

Appendix A Agent Technology for e-Commerce 13 Utilities  Usually the consequences of decisions are expressed in monetary terms  Additional factors such are reputation, time, etc. are also usually translated into money  Issue with the use of money to describe the consequences of actions:  If a fair coin comes up heads you win £1, otherwise you loose £0.75, would you take this bet?  If a fair coin comes up heads you win £1000, otherwise you loose £750, would you take this bet?  The value of a currency, differs from person to person

Appendix A Agent Technology for e-Commerce 14 Preferences  The concept of preference is used to indicate that we would like/desire/prefer one thing over another  o o’ indicates that o is (strictly) preferred to o’  o ~ o’ indicates that an agent is indifferent between o and o’  o o’ indicates that o is (weakly) preferred to o’  Given any o and o’, then o o’, or o’ o, or o ~ o’  Given any o, o’ and o’’, then if o o’ and o’ o’’, then o’ o’’  If o o’ o’’, then there is a p such that [p,o;1-p,o’’] ~ o’  If o ~ o’, then [p,o; 1-p,o’’] ~ [p,o’; 1-p, o’’]  If o o’, then (p  q  [p,o;1-p,o’]  [q,o;1-q,o’] )

Appendix A Agent Technology for e-Commerce 15 Utility functions  A utility function provides a convenient way of conveying information about preferences  If o o’, then u(o)>u(o’) and if o ~ o’ then u(o)=u(o’)  If an agent is indifferent between: (a) outcome o for certain and (b) taking a bet or lottery in which it receives o’ with probability p and o’’ with probability 1-p then u(o)=(p)u(o’)+(1-p)u(o’’)  Ordinal utilities  Cardinal utilities  Monotonic transformation

Appendix A Agent Technology for e-Commerce 16 Assessing a utility function How can an agent assess a utility function?  Suppose most and least preferable payoffs are R + and R - and u(R + )=1 and u(R - )=0  For any other payoff R, it should be: u(R + )  u(R)  u(R - ) or 1  u(R)  0

Appendix A Agent Technology for e-Commerce 17  To determine the value of u(R) consider:  L 1 : Receive R for certain  L 2 : Receive R + with probability p and R - with probability 1-p  Expected utilities:  EU(L 1 )=u(R)  EU(L 2 )=(p)u(R + )+(1-p)u(R - )=(p)(1)+(1-p)(0)=p  If u(R)>p, L 1 should be selected, whereas if u(R)<p, L 2 should be selected, and if u(R)=p then the agent is indifferent between the two lotteries

Appendix A Agent Technology for e-Commerce 18 Utility and money  The value, i.e. utility, of money may differ from person to person  Consider the lottery  L 1 : receive £0 for certain  L 2 : receive £100 with probability p and -£100 with (1-p)  Suppose an agent decides that for p=0.75 is indifferent between the two lotteries, i.e. p>0.75 prefers lottery L 2  The agent also assess u(-£50)=0.4 and u(£50)=0.9

Appendix A Agent Technology for e-Commerce 19

Appendix A Agent Technology for e-Commerce 20  If p is fixed, the amount of money that an agent would need to receive for certain in L 1 to make it indifferent between two lotteries can be determined. Consider:  L 1 : receive £x for certain  L 2 : receive £100 with probability 0.5 and -£100 with 0.5  Suppose x=-£30, then u(-£30)=0.5 and -£30 is considered to be the cash equivalent of the gamble involved in L 2  The amount of £30 is called the risk premium – the basis of insurance industry

Appendix A Agent Technology for e-Commerce 21 Utility function of risk-averse agent

Appendix A Agent Technology for e-Commerce 22 Utility function of a risk-prone agent

Appendix A Agent Technology for e-Commerce 23 Utility function of a risk-neutral agent

Appendix A Agent Technology for e-Commerce 24 Multi-attribute utility functions  The utility of an action may depend on a number of factors  Multi-dimensional or multi-attribute utility theory deals with expressing such utilities  Example: you are made a set of job offers, how do you decide? u(job-offer) = u(salary) + u(location) + u(pension package) + u(career opportunities) u(job-offer) = 0.4u(salary) + 0.1u(location) + 0.3u(pension package) + 0.2u(career opportunities) But if there are interdependencies between attributes, then additive utility functions do not suffice. Multi-linear expressions: u(x,y)=w x u(x)+w y u(y)+(1-w x -w y )u(x)u(y)