1. Chapter 2
Intelligent Agent

2. Agents
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators
Human agent:
eyes, ears, and other organs for sensors; hands, legs, mouth, and other body parts for actuators
Robotic agent:
cameras and infrared range finders for sensors; various motors for actuators
any given instant can depend on the entire percept sequence observed to date, but not on anything it hasn't perceived.

3. Agents and environments
The agent function maps from percept histories to actions:
[f: P*  A]
The agent program runs on the physical architecture to produce f
agent = architecture + program
The agent function is an abstract mathematical description; the agent program is a specify implementation, running within some physical system.

4. Vacuum-cleaner world
Percepts: location and state of the environment, e.g., [A,Dirty], [A,Clean], [B,Dirty]
Actions: Left, Right, Suck, NoOp
Simple agent function:
if the current square is dirty, then suck; otherwise, move to the other square.

5. what makes an agent good or bad, intelligent or stupid?

6. Rational agents Rational agent: is one that does the right thing
Performance measure:
that evaluates any given sequence of environment states.
An objective criterion for success of an agent's behavior, e.g.,
Robot driver?
Chess-playing program?
Spam classifier?
Rational Agent: selects actions that is expected to maximize its performance measure,
given percept sequence
given agent’s built-in knowledge
sidepoint: how to maximize expected future performance, given only historical data

7. Rationality What is rational at any given time depends on four things:
The performance measure that defines the criterion of success.
The agent's prior knowledge of the environment.
The actions that the agent can perform.
The agent's percept sequence to date.
For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

8. Rational agents
Performance measure: An objective criterion for success of an agent's behavior, e.g.,
Robot driver?
Chess-playing program?
Spam classifier?
Rational Agent: selects actions that is expected to maximize its performance measure,
given percept sequence
given agent’s built-in knowledge
sidepoint: how to maximize expected future performance, given only historical data

9. Rational agents
Rationality is distinct from omniscience (all-knowing with infinite knowledge)
Agents can perform actions in order to modify future percepts so as to obtain useful information (information gathering, exploration)
An agent is autonomous if its behavior is determined by its own percepts & experience (with ability to learn and adapt) without depending solely on built-in knowledge99

10. PEAS: Task Environment
Before we design an intelligent agent, we must specify its “task environment”:
PEAS:
Performance measure
Environment
Actuators
Sensors

11. PEAS Example: Agent = robot driver in DARPA Challenge
Performance measure:
Time to complete course
Environment:
Roads, other traffic, obstacles
Actuators:
Steering wheel, accelerator, brake, signal, horn
Sensors:
Optical cameras, lasers, sonar, accelerometer, speedometer, GPS, odometer, engine sensors,

12. PEAS Example: Agent = Medical diagnosis system Performance measure:
Healthy patient, minimize costs, lawsuits
Environment:
Patient, hospital, staff
Actuators:
Screen display (questions, tests, diagnoses, treatments, referrals)
Sensors:
Keyboard (entry of symptoms, findings, patient's answers)

13. PEAS Percepts Actions Goals Environment Agent type
Medical diagnosis system
Symptoms, findings, patient's answers
Questions, tests, treatments
Healthy patients, minimize costs
Patient, hospital
Satellite image analysis system
Pixels of varying intensity, color
Print a categorization of scene
Correct categorization
Images from orbiting satellite
Part-picking robot
Pixels of varying intensity
Pick up parts and sort into bins
Place parts in correct bins
Conveyor belts with parts
Refinery controller
Temperature, pressure readings
Open, close valves; adjust temperature
Maximize purity, yield, safety
Refinery
Interactive English tutor
Typed words
Print exercises, suggestions, corrections
Maximize student's score on test
Set of students

14. Environment types Fully observable (vs. partially observable):
An agent's sensors give it access to the complete state of the environment at each point in time.
Fully observable environments are convenient because the agent need not maintain any internal state to keep track of the world.
E.g.: Playing soccer : Partial observable,
Perform high jump : Full observable.
Single agent (vs. multi-agent):
An agent operating by itself in an environment. Does the other agent interfere with my performance measure?
e,.g: crossword puzzle : Single agent
Playing chess : mult-iagent
(agent may be competitive or cooperation)

15. Environment types Deterministic (vs. stochastic):
The next state of the environment is completely determined by the current state and the action executed by the agent.
If the environment is deterministic except for the actions of other agents, then the environment is strategic
Deterministic environments can appear stochastic to an agent (e.g., when only partially observable)
E.g: shopping for IT book online: deterministic,
Stochastic: generally implies that uncertainty about outcomes is quantified in terms of probabilities
E.g: performing a high Jump :Stochastic

16. Episodic (vs. sequential):
An agent’s action is divided into atomic episodes. Decisions do not depend on previous decisions/actions.
Episodic : the next episode does not depend on the actions taken in previous episodes.
E.g.: an agent that has to spot defective parts on an assembly line bases each decision on the current part, regardless of previous decisions;
sequential the current decision could affect all future decisions e.g.: Chess and taxi driving are sequential

17. Environment types Static (vs. dynamic): Discrete (vs. continuous):
Static : The environment is unchanged while an agent is deliberating.
E.g.: crossword puzzle :static
Dynamic: agent need to keep looking at the world while it is deciding on an action and The environment is changed while an agent is deliberating. Dynamic environment continuously asking agent what want to do.
E.g.: Taxi driving : Dynamic
The environment is semidynamic if the environment itself does not change with the passage of time but the agent's performance score does.
Discrete (vs. continuous):
A discrete set of distinct, clearly defined percepts and actions.
How we represent or abstract or model the world
Dynamic
Need to worry about time
Need to observe while deliberating

18. task environm.
observable
deterministic/
stochastic
episodic/
sequential
static/
dynamic
discrete/
continuous
agents
crossword
puzzle
fully
determ.
static
discrete
single
chess with
clock
strategic
semi
multi
poker
taxi
driving
partial
medical
diagnosis
image
analysis
episodic
partpicking
robot
refinery
controller
interact.
tutor

19. What is the environment for the DARPA Challenge?
Agent = robotic vehicle
Environment = 130-mile route through desert
Observable?
Deterministic?
Episodic?
Static?
Discrete?
Agents?

20. Agent functions and programs
An agent is completely specified by the agent function mapping percept sequences to actions
One agent function (or a small equivalence class) is rational
Difference between agent program and agent function:
Agent program: takes the current percept as input because nothing more is available from the environment;
Agent function, which takes the entire percept history.
Aim: find a way to implement the rational agent function concisely

21. Examples of how the agent function can be implemented
Table-driven agent
Simple reflex agent
Reflex agent with internal state
Agent with explicit goals
Utility-based agent
More
sophisticated

22. Table-lookup agent Drawbacks: Agent programs type: Huge table
Take a long time to build the table
No autonomy
Even with learning, need a long time to learn the table entries
Agent programs type:
Simple reflex agents
Model-based reflex agents
Goal-based agents
Utility-based agents

23. Table-driven agent

24. Simple reflex agents

25. Simple reflex agent…
Table lookup of condition-action pairs defining all possible condition-action rules necessary to interact in an environment
e.g. if car-in-front-is-breaking then initiate breaking
select precepts based on the current percept
ignoring the rest of the precept history
The rules are like the form “if … then …”
efficient but have narrow range of applicability Because knowledge sometimes cannot be stated explicitly
Work only
if the environment is fully observable
Problems
Table is still too big to generate and to store (e.g. taxi)
Takes long time to build the table
No knowledge of non-perceptual parts of the current state
Not adaptive to changes in the environment; requires entire table to be updated if changes occur
Looping: Can’t make actions conditional

26. A Simple Reflex Agent in Nature
percepts
(size, motion)
RULES:
(1) If small moving object,
then activate SNAP
(2) If large moving object,
then activate AVOID and inhibit SNAP
ELSE (not moving) then NOOP
needed for
completeness
Action: SNAP or AVOID or NOOP

27. Model-based reflex agents

28. Model-Based Reflex Agents
Most effective way to keep track of the part of the world it can’t see now.
Maintain some internal state that depends on percept history and thereby reflects at least some of the unobserved aspects of the current state (e.g. using some type of variable).
For the world that is partially observable
the agent has to keep track of an internal state
That depends on the percept history
Reflecting some of the unobserved aspects
E.g., driving a car and changing lane
Requiring two types of knowledge
How the world evolves independently of the agent
How the agent’s actions affect the world

29. Goal-based agents

30. Current state of the environment is always not enough
The goal is another issue to achieve
Judgment of rationality / correctness
Actions chosen  goals, based on
the current state
the current percept
Conclusion
Goal-based agents are less efficient
but more flexible
Agent  Different goals  different tasks
Search and planning
two other sub-fields in AI
to find out the action sequences to achieve its goal

31. Utility-based agents

32. Goals alone are not enough
to generate high-quality behavior
E.g. meals in Canteen, good or not ?
Many action sequences  the goals
some are better and some worse
If goal means success,
then utility means the degree of success (how successful it is)
State A has higher utility
If state A is more preferred than others
Utility is therefore a function
that maps a state onto a real number
the degree of success
Utility has several advantages:
When there are conflicting goals,
Only some of the goals but not all can be achieved
utility describes the appropriate trade-off
When there are several goals
None of them are achieved certainly
utility provides a way for the decision-making

33. Learning Agents Four conceptual components
After an agent is programmed, can it work immediately?
No, it still need teaching
In AI,
Once an agent is done
We teach it by giving it a set of examples
Test it by using another set of examples
We then say the agent learns
A learning agent
Four conceptual components
Learning element
Making improvement
Performance element
Selecting external actions
Critic
Tells the Learning element how well the agent is doing with respect to fixed performance standard.
(Feedback from user or examples, good or not?)
Problem generator
Suggest actions that will lead to new and informative experiences.

34. Learning agents

35. Agent types Five basic types in order of increasing generality:
Table Driven agent
Simple reflex agents
Model-based reflex agents
Goal-based agents
Problem-solving agents
Utility-based agents
Can distinguish between different goals
Learning agents

36. Problem-Solving Agents
Intelligent agents can solve problems by searching a state-space
State-space Model
the agent’s model of the world
usually a set of discrete states
e.g., in driving, the states in the model could be towns/cities
Goal State(s)
a goal is defined as a desirable state for an agent
there may be many states which satisfy the goal test
e.g., drive to a town with a ski-resort
or just one state which satisfies the goal
e.g., drive to Mammoth
Operators (actions, successor function)
operators are legal actions which the agent can take to move from one state to another

37. Initial Simplifying Assumptions
Environment is static
no changes in environment while problem is being solved
Environment is observable
Environment and actions are discrete
(typically assumed, but we will see some exceptions)
Environment is deterministic

38. Example: Traveling in Romania
On holiday in Romania; currently in Arad
Flight leaves tomorrow from Bucharest
Formulate goal:
be in Bucharest
Formulate problem:
states: various cities
actions/operators: drive between cities
Find solution
By searching through states to find a goal
sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharest
Execute states that lead to a solution

39. Example: Traveling in Romania

40. State-Space Problem Formulation
A problem is defined by four items:
initial state e.g., "at Arad“
actions or successor function
S(x) = set of action–state pairs e.g., S(Arad) = {<Arad  Zerind, Zerind>, … }
3. goal test (or set of goal states)
e.g., x = "at Bucharest”, Checkmate(x)
4. path cost (additive)
e.g., sum of distances, number of actions executed, etc.
c(x,a,y) is the step cost, assumed to be ≥ 0
A solution is a sequence of actions leading from the initial state to a goal state

41. Example: Formulating the Navigation Problem
Set of States
individual cities
e.g., Irvine, SF, Las Vegas, Reno, Boise, Phoenix, Denver
Operators
freeway routes from one city to another
e.g., Irvine to SF via 5, SF to Seattle, etc
Start State
current city where we are, Irvine
Goal States
set of cities we would like to be in
e.g., cities which are closer than Irvine
Solution
a specific goal city, e.g., Boise
a sequence of operators which get us there,
e.g., Irvine to SF via 5, SF to Reno via 80, etc

42. Abstraction Definition of Abstraction:
Process of removing irrelevant detail to create an abstract representation: ``high-level”, ignores irrelevant details
Navigation Example: how do we define states and operators?
First step is to abstract “the big picture”
i.e., solve a map problem
nodes = cities, links = freeways/roads (a high-level description)
this description is an abstraction of the real problem
Can later worry about details like freeway onramps, refueling, etc
Abstraction is critical for automated problem solving
must create an approximate, simplified, model of the world for the computer to deal with: real-world is too detailed to model exactly
good abstractions retain all important details

43. The State-Space Graph Graphs: Search graphs: Problem formulation:
nodes, arcs, directed arcs, paths
Search graphs:
States are nodes
operators are directed arcs
solution is a path from start S to goal G
Problem formulation:
Give an abstract description of states, operators, initial state and goal state.
Problem solving:
Generate a part of the search space that contains a solution

44. The Traveling Salesperson Problem
Find the shortest tour that visits all cities without visiting any city twice and return to starting point.
State: sequence of cities visited
S0 = A C D A E F B
G = a complete tour

45. Example: 8-queens problem

46. State-Space problem formulation
states? -any arrangement of n<=8 queens
-or arrangements of n<=8 queens in leftmost n
columns, 1 per column, such that no queen
attacks any other.
initial state? no queens on the board
actions? -add queen to any empty square
-or add queen to leftmost empty square such that it is not attacked by other queens.
goal test? 8 queens on the board, none attacked.
path cost? 1 per move

47. Example: Robot Assembly
States
Initial state
Actions
Goal test
Path Cost

48. Example: Robot Assembly
States: configuration of robot (angles, positions) and object parts
Initial state: any configuration of robot and object parts
Actions: continuous motion of robot joints
Goal test: object assembled?
Path Cost: time-taken or number of actions

49. Learning a spam email classifier
States
Initial state
Actions
Goal test
Path Cost

50. Learning a spam email classifier
States: settings of the parameters in our model
Initial state: random parameter settings
Actions: moving in parameter space
Goal test: optimal accuracy on the training data
Path Cost: time taken to find optimal parameters
(Note: this is an optimization problem – many machine learning problems can be cast as optimization)