1. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Uncertainty
Chapter , +
Introduction & Basics
Probability Theory and Notation
Unconditional, Joint, Conditional Probabilities
Probability Distributions & Full Joint Prob. Dist. Table
Action-State Utility Matrix
Expected Utility & Optimal Decision Making
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

2. Autonomous Agents Agent Environment Model of World (being updated)
Prior Knowledge about the World
Sensors
Reasoning & Decisions Making
List of Possible Actions
Goals/Utility
Effectors
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

3. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
How an Agent Operates
Basic Cycle:
use sensors to sense the environment
update the world model
reason about the world (infer new facts)
update plan on how to reach goal
make decision on next action
use effectors to implement action
Basic cycle is repeated until goal is reached
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

4. Autonomous Agents Real World Driving Agent
Model of vehicle location & status road status
Prior Knowledge
physics of movement
rules of road
Sensors
camera
tachometer
engine status
temperature
Reasoning & Decisions Making
Actions
change speed
change steering
Goals
drive home
Effectors
accelerator
brakes
steering
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

5. The Agent’s World Model
World Model: internal representation of the external world that combines:
current inputs
prior states of world model
background knowledge
Necessarily, the world model is a simplification
e.g. in driving we cannot represent every detail
every pebble, leaf, snow flake on the road?
every dent and scratch in every vehicle in sight?
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

6. Uncertainty in the World Model
The agent can never be completely certain about the state of the external world since there is ambiguity and uncertainty.
Why?
sensors have limited precision
e.g. camera has only so many pixels to capture an image
sensors have limited accuracy
e.g. tachometer’s estimate of velocity is approximate
there are hidden variables that sensors can’t “see”
e.g. vehicle behind large truck or storm clouds approaching
the future is unknown, uncertain, i.e. cannot foresee all possible future events which may happen
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

7. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Rules and Uncertainty
Say we have a rule:
if toothache then problem is cavity
But not all patients have toothaches due to cavities
so we could set up rules like:
if toothache and not(gum disease) and not(filling) and ... then problem = cavity
This gets complicated, a better method would be:
if toothache then problem is cavity with 0.8 probability
or P(cavity|toothache) = 0.8
the probability of cavity is 0.8 given toothache is all that is known
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

8. Example of Uncertainty
Assume a camera and vision system is used to estimate the curvature of the road ahead
There's uncertainty about which way it curves
limited pixel resolution, noise in image
algorithm for “road detection” is not perfect
This uncertainty can be represented with a simple probability model:
P(road curves to left|E) = 0.6 P(road goes straight|E) = 0.3 P(road curves to right|E) = 0.1
where the probability of an event is a measure of agent’s belief in the event given the evidence E
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

9. Uncertainty in the World Model
True uncertainty: rules are probabilistic in nature
quantum mechanics
rolling dice, flipping a coin?
Laziness: too hard to determine exceptionless rules
takes too much work to determine all of the relevant factors
too hard to use the enormous rules that result
Theoretical ignorance: don't know all the rules
problem domain has no complete theory (medical diagnosis)
Practical ignorance: do know all the rules BUT
haven't collected all relevant information for a particular case
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

10. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Logics
Logics are characterized by what they commit to as "primitives".
Logic
What Exists in World
Knowledge States
Propositional
facts
true/false/unknown
First-Order
facts, objects, relations
Temporal
facts, objects, relations, times
Probability Theory
degree of belief 0..1
Fuzzy
degree of truth
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

11. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Probability Theory
Probability theory serves as a formal means
for representing and reasoning with uncertain knowledge
of manipulating degrees of belief in a proposition (event, conclusion, diagnosis, etc.)
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

12. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Probability Notation
Uses a representation that is similar to propositional logic but more expressive being able to represent probabilities
A proposition (event) is:
A=a which means variable A takes value a e.g. RoadCurvature=left
either true or false in the world
agent has a degree of belief in the proposition
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

13. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Probability Notation
Random Variables (RV):
are capitalized (usually) e.g. Sky, RoadCurvature, Temperature
refer to attributes of the world whose "status" is unknown
have one and only one value at a time
have a domain of values that are possible states of the world:
boolean: domain = <true, false>
Cavity=true abbreviated as cavity Cavity=false abbreviated as Øcavity
discrete: domain is countable (includes boolean) values are exhaustive and mutually exclusive
e.g. Sky domain = <clear, partly_cloudy, overcast> Sky=clear abbreviated as clear Sky¹clear also abbrv. as Øclear
continuous: domain is real numbers (beyond scope of CS540)
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

14. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Probability Notation
An agent’s uncertainty is represented by:
P(A=a) or simply P(a), this is:
the agent’s degree of belief that variable A takes on value a given no other information relating to A
a single probability called an unconditional or prior probability
Property:  P(ai) = P(a1) + P(a2) P(an) = 1 sum over all values in the domain of variable A is 1 because domain is exhaustive and mutually exclusive
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

15. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Example "Clue"
Goal to determine who killed Mr. John Boddy, where and with what weapon:
6 characters (Mr. Green, Miss Scarlet, Prof. Plum, …)
6 weapons (rope, lead pipe, candlestick, knife, …)
9 locations (conservatory, kitchen, billiard room, …)
Prior probabilities before any evidence is obtained
each character has 1/6 probability of being Who murdered e.g. P(Who=green) = P(green) = 1/6
each weapon has 1/6 probability of being What caused murder
each location has 1/9 probability of being Where murdered e.g. P(Where=kitchen) = P(kitchen) = 1/9
Total probability for each random variable is 1
Parker Bros. introduced in 1949
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

16. Source of Probabilities
Frequentists:
probabilities come from experiments
if 10 of 100 people tested have a cavity the P(cavity) = .1
probability means the fraction that would be observed in the limit of infinitely many samples
Objectivists:
probabilities are real aspects of the world
objects have a propensity to behave in certain ways
coin has a propensity to come up heads with a probability .5
Subjectivists:
probabilities characterize an agent's belief
have no external physical significance
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

17. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Atomic Events
An atomic event is a complete specification of the state of the world about which the agent is uncertain.
e.g. green Ù rope Ù kitchen or plum Ù knife Ù conservatory
Properties of atomic events:
they're mutually exclusive: at most one can be the case
set of all possible atomic events is exhaustive: at least one must be the case
any particular atomic event entails the truth/falsehood of every proposition
any proposition is logically equivalent to the disjunction of all atomic events that entail the truth of that proposition
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

18. Probability Distributions
Given A is a RV taking values in <a1, a2, … , an>
e.g. if A is Sky, then a is one of <clear, partly_cloudy, overcast>
P(a) represents a single probability where A=a
e.g. if A is Sky, then P(a) means any one of P(clear), P(partly_cloudy), P(overcast)
e.g. P(conservatory) = 1/9
P(A) represents a probability distribution
the set of values {P(a1), P(a2), …, P(an)}
If A takes n values, then P(A) is a set of n probabilities
e.g. if A is Sky, then P(Sky) is the set of probabilities: {P(clear), P(partly_cloudy), P(overcast)}
e.g. P(Who) = {1/6, 1/6, 1/6, 1/6, 1/6, 1/6}
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

19. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Joint Probabilities
Joint probabilities specify the probabilities for the conjunction of propositions.
P(a,b,…): joint probability of A=a Ù B=b Ù … e.g. P(green, rope, library) = P(green Ù rope Ù library) = P(Who=green Ù What=rope Ù Where=library)
P(A,B,…): joint probability distribution e.g. P(Who,Where) is a 6 * 9 table of probabilities P(green, library) P(green, conservatory) …
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

20. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Joint Probabilities
Full joint probability distribution:
e.g. P(Who,What,Where) is a 6 * 6 * 9 table of probabilities
completely specifies all of the possible probabilistic information by assigning probabilities to all possible combinations of values of all the random variables, which encode all of the relevant information about a problem
intractable representation since table grows exponentially in size, kn where n variables each have k possible values
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

21. Conditional Probability
Conditional (posterior) probabilities
formalize the process of accumulating evidence and updating probabilities based on new evidence
specify the belief in a proposition (event, conclusion, diagnosis, etc.) that is conditioned on a proposition (evidence, feature, symptom, etc.) being true
P(a|e): conditional probability of A=a given E=e evidence is all that is known true
P(a|e) = P(a Ù e) / P(e) = P(a, e) / P(e)
the conditional probability can viewed as the joint probability P(a, e) normalized by the prior probability P(e)
e.g. P(green|Øwhite) = P(green Ù Øwhite)/P(Øwhite) = 1/6/5/6 = 1/5
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

22. Conditional Probability
Conditional probabilities behave exactly like standard probabilities, for example:
0 <= P(a|e) <= 1
conditional probabilities are between 0 and 1 inclusive
P(a1|e) + P(a2|e) P(an|e) = 1
conditional probabilities sum to 1
where a1, …, an are all values in the domain of RV A
P(Øa|e) = 1 - P(a|e)
negation for conditional probabilities
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

23. Conditional Probability
P(conjunction of events|e)
P(a Ù b Ù c | e) or as P(a, b, c | e) is the agent’s belief in the sentence a Ù b Ù c conditioned on e being true
P(a| conjunction of evidences)
P(a | e Ù f Ù g) or as P(a | e, f, g) is the agent’s belief in the sentence a conditioned on e Ù f Ù g being true
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

24. Kolmogorov's Axioms of Probability
0 <= P(a) <= 1
probabilities are between 0 and 1 inclusive
P(true) = 1, P(false) = 0
probability of 1 for propositions believed to be absolutely true
probability of 0 for propositions believed to be absolutely false
P(a Ú b) = P(a) + P(b) - P(a Ù b)
probability of two states is their sum minus their “intersection”
Property: P(Øa) = 1 - P(a)
belief in Øa is one minus agent's belief in a
It has been shown that these are necessary if an agent is to behave rationally (see pp ).
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

25. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Actions and States
What is the optimal action to take given an agent's uncertain model of the world?
A rational agent will want to take the best action given information about the states.
For example
in what direction should a driving agent steer given:
probabilities for state of road curvature: {P(left), P(straight), P(right)}
and set of actions: {steer left, steer straight, steer right}
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

26. Action-State Utility Matrix
u(A, s) is the utility to an agent which would result from action A if the world were really in state s
utility is usually measured in units of “negative cost” for the agent
e.g. u(write $100 check, balance = $50) = -$10 fee
For a set of actions A and a set of states S this gives a utility matrix
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

27. Action-State Utility Matrix Example
Random variable C is for curvature of the road
domain is <l, s, r>
l = left, s = straight, r = right
Action A
domain is <SL, SS, SR, Halt>
SL = steer left, SS = steer straight, SR = steer right
STATE of C
ACTION A l s r SL
-1000 SS
-20 SR
Halt
-500
u(SL, l) = 0
u(SR, l) = -$20 go off the road
u(SL, r) = -$1000 cross into oncoming traffic!
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

28. Expected Utilities (EU)
The agent reasons “hypothetically” since it never really exactly knows the state of the world
How can the driving agent choose the best action given probabilities about the state of the world?
e.g. say P(l) = 0.2, P(s) = 0.7, P(r) = 0.1
The expected utility of an action is the sum of the utility over all possible states of the world
= sum over states of {utility(Action, state) * P(state)}
e.g. EU(SL) = u(SL,l) * P(l) + u(SL,s) * P(s) + u(SL,r) * P(r)
= * * * 0.1 =
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

29. Optimal Decision-Making
Optimal decision-making: choosing the best action
calculate the expected utility for each action
choose action with optimal expected utility (EU)
optimal EU is typically the minimum cost
This strategy is the optimal strategy for an agent that must make decisions in an uncertain world
assumes the probabilities are accurate
assumes the utilities are accurate
is a “greedy” strategy: only optimizes 1 step ahead
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

30. Optimal Decision-Making Example
Use action-state utility matrix from prior slide
State Probabilities are P(l) = 0.2, P(s) = 0.7, P(r) = 0.1
Calculate EU for all actions:
EU(SL) = * * * 0.1 = -800
EU(SS) = * * * 0.1 = -104
EU(SR) = * * * 0.1 = -18
EU(Halt) = * * * 0.1 = -500
Optimal utility action = steer right
note in this case this is the least likely state of the world!
but it is the one that has optimal expected utility i.e. it is the action that on average will minimize cost
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

31. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Summary
Random variable (RV): a variable (uppercase) that takes on values (lowercase) from a domain of mutually exclusive and exhaustive values
A=a: a proposition, world state, event, effect, etc.
abbreviate: P(A=true) to P(a)
abbreviate: P(A=false) to P(Øa)
abbreviate: P(A=value) to P(value)
abbreviate: P(A¹value) to P(Øvalue)
Atomic event: a complete specification of the state of the world about which the agent is uncertain
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

32. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Summary
P(a): a prior probability of RV A=a which is the degree of belief proposition a in absence of any other relevant information
P(a|e): conditional probability of RV A=a given E=e which is the degree of belief in proposition a when all that is known is evidence e
P(A): probability distribution, i.e. set of P(ai) for all i
Joint probabilities are for conjunctions of propositions
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.

33. ©2001-2004 James D. Skrentny from notes by C. Dyer, et. al.
Summary
Autonomous agents are involved in a cycle of:
sensing
estimating the state of the world
reasoning, planning
making decisions and taking actions
Probability allows the agent to represent uncertainty about the world
agent can assign probabilities to world states
agent can assign utilities to action-state pairs
Optimal decision-making is choosing the action with the optimal expected utility
2/24/2019
© James D. Skrentny from notes by C. Dyer, et. al.