1. Intelligent Agents Russell and Norvig: 2
CISC4/681 Introduction to Artificial Intelligence

2. Agents
Agent – perceives the environment through sensors and acts on it through actuators
Percept – agent’s perceptual input (the basis for its actions)
Percept Sequence – complete history of what has been perceived.
What is an agent? Anything that can perceive its environment through sensors and act on it through actuators.
Generally an agent’s actions may depend on the entire percept sequence observed to date.
CISC4/681 Introduction to Artificial Intelligence

3. Agent Function
Agent Function – maps a give percept sequence into an action; describes what the agent does.
Externally – Table of actions
Internally – Agent Program
Agent function is an abstract mathematical description; the agent program is a concrete implementation running on the agent architecture.
CISC4/681 Introduction to Artificial Intelligence

4. Vacuum Cleaner World Percepts: which square (A or B); dirt?
Actions: move right, move left, suck, do nothing
Agent function: maps percept sequence into actions
Agent program: function’s implementation
How should the program act?
Illustrate these ideas… Very simply Vacuum cleaner world. The vacuum can perceive which location it is in (A or B), and whether or not there is dirt there.
Simple agent function: if the current square is dirty, then suck, otherwise move to the other square. But, lots of other choices exist for how it should act. What is the right way?
CISC4/681 Introduction to Artificial Intelligence

5. Agent Characterization
Meant to be a tool for analyzing systems – not characterizing them as agent versus non-agent
Lots of things can be characterized as agents (artifacts) that act on the world
AI operates where
Artifacts have significant computational resources
Task Environments require nontrivial decision making
AI operates at the most interesting end of the spectrum…
CISC4/681 Introduction to Artificial Intelligence

6. Vacuum Cleaner World Percepts: which square (A or B); dirt?
Actions: move right, move left, suck, do nothing
Agent function: maps percept sequence into actions
Agent program: function’s implementation
How should the program act?
So – let’s go back to this agent concept.
What we need to think about is how this program should act.
CISC4/681 Introduction to Artificial Intelligence

7. Rational Agent – does the right thing
What does that mean? One that behaves as well as possible given the Environment in which it acts. How should success be measured? On consequences.
Performance measure
Embodies criterion for success
Amount of dirt cleaned?
Cleaned floors?
Generally defined in terms of desired effect on environment (not on actions of agent)
Defining measure not always easy!
We want to design are agents to be “rational” and what that means is that they do the right thing.
Agents actions cause the environment to go through different states – are they desirable? The notion of what it means to be desired is captured in a performance measure.
E.g. not a good idea for performance measure to measure the amount of dirt cleaned in an hour – rational agent would clean, dump dirt, clean it up again, etc…
For instance, is the performance measured on average cleanliness of the floor? Is it better to be moderately clean all the time – or sparkling clean a very small amount of the time (with a lot of time between cleaning)?
CISC4/681 Introduction to Artificial Intelligence

8. Rationality Depends on:
Performance measure that defines criterion for success.
Agent’s prior knowledge of the environment.
Actions the agent can perform.
Agent’s percept sequence to date.
For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.
So, if the agent moves from room to room and cleans it if it is dirty, and otherwise moves. In an environment where the rooms stay clean – this agent would be rational UNDER THESE CIRCUMSTANCES (even though it would keep oscillating between rooms). Note, if there were an electricity penalty in the performance measure, then this same agent would not be rational.
CISC4/681 Introduction to Artificial Intelligence

9. Rationality
For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.
Notice the rationality is dependent on EXPECTED maximization.
Agent might need to learn how the environment changes, what action sequences to put together, etc…
It takes perfection to maximize actual performance – this requires omnisciece.
Take crossing street example….
Rationality not the same as perfection.
CISC4/681 Introduction to Artificial Intelligence

10. Rationality
For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.
Notice that an agent may be rational because the designer thought of everything, or it may have learned it itself (more autonomous)
So, agent may need to learn not to cross the street without first getting the perceptual information to know whether or not a car is coming. Information gathering, exploration.
CISC4/681 Introduction to Artificial Intelligence

11. Task Environment
The “problems” for which rational agents are the “solutions”
PEAS Description of Task Environment
Performance Measure
Environment
Actuators (actions)
Sensors (what can be perceived)
In designing an agent, the first step must always be to specify the task environment as fully as possible.
The book provides a number of examples of PEAS descriptions of various problems (taxi driver, medical diagnosis system, refinery controller, interactive English tutor).
Let’s take a look at some of the properties of the task environment (they help us determine an appropriate agent design).
CISC4/681 Introduction to Artificial Intelligence

12. Properties of Task Environments (affect appropriate agent design)
Fully observable vs partially observable
Fully observable gives access to complete state of the environment
Complete state means aspects relevant to action choice
global vs local dirt sensor
CISC4/681 Introduction to Artificial Intelligence

13. Properties of Task Environments (affect appropriate agent design)
Single Agent vs Multi-agent
Single Agent – crossword puzzle
Multi-agent – chess, taxi driving? (are other drivers best described as maximizing a performance element?)
Multi-agent means other agents may be competitive or cooperative and may require communication
Multi-agent may need communication
CISC4/681 Introduction to Artificial Intelligence

14. Properties of Task Environments (affect appropriate agent design)
Deterministic vs Stochastic
Deterministic – next state completely determined by current state and action
Uncertainty may arise because of defective actions or partially observable state (i.e., agent might not see everything that affects the outcome of an action).
CISC4/681 Introduction to Artificial Intelligence

15. Properties of Task Environments (affect appropriate agent design)
Episodic vs Sequential
Episodic the agent’s experience divided into atomic episodes
Next episode not dependent on actions taken in previous episode. E.g., assembly line
Sequential – current action may affect future actions. E.g., playing chess, taxi
short-term actions have long-term effects
must think ahead in choosing an action
CISC4/681 Introduction to Artificial Intelligence

16. Properties of Task Environments (affect appropriate agent design)
Static vs Dynamic
does environment change while agent is deliberating?
Static – crossword puzzle
Dynamic – taxi driver
CISC4/681 Introduction to Artificial Intelligence

17. Properties of Task Environments (affect appropriate agent design)
Discrete vs Continuous.
Can refer to
the state of the environment (chess has finite number of discrete states)
the way time is handled (taxi driving continuous – speed and location of taxi sweep through range of continuous values)
percepts and actions (taxi driving continuous – steering angles)
CISC4/681 Introduction to Artificial Intelligence

18. Properties of Task Environments (affect appropriate agent design)
Known vs Unknown
This does not refer to the environment itself, but rather the agent’s knowledge of it and how it changes.
If unknown, the agent may need to learn
CISC4/681 Introduction to Artificial Intelligence

19. Properties of Task Environments (affect appropriate agent design)
Easy: Fully observable, Deterministic, Episodic, Static, Discrete, Single agent.
Hard: Partially observable, Sochastic, Sequential, Dynamic, Continuous, Multi-Agent
CISC4/681 Introduction to Artificial Intelligence

20. Environment types
Chess with Chess without Taxi driving
a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No
The environment type largely determines the agent design
The real world is (of course) partially observable, stochastic, sequential, dynamic, continuous, multi-agent

21. Agent Programs
Need to develop agents – programs that take the current percept as input from the sensors and return an action to the actuators.
Could just have a big table look-up (shown in the book) – where map each sequence of percepts into an action. Problem is that table is generally very large! So, can we do it with less code??
CISC4/681 Introduction to Artificial Intelligence

22. Possible Agent Program
So, this is a table of percept sequences – and the table can get extremely large… probably not what we mean by an intelligent program!
CISC4/681 Introduction to Artificial Intelligence

23. Agent Programs
Need to develop agents – programs that take the current percept as input from the sensors and return an action to the actuators.
The key challenge for AI is to find out how to write programs that, to the extent possible, produce rational behavior from a small amount of code.
Could just have a big table look-up (shown in the book) – where map each sequence of percepts into an action. Problem is that table is generally very large! So, can we do it with less code??
CISC4/681 Introduction to Artificial Intelligence

24. Simple Reflective Agent
Selects actions on the basis of the current percept ignoring the rest of the percept history. These agents work with a set of condition-action rules where the condition test whether or not some condition holds in the current percept information – if it does, then the action is done.
CISC4/681 Introduction to Artificial Intelligence

25. Simple Reflexive Agent
Handles simplest kind of world
Agent embodies a set of condition-action rules
If percept then action
Agent simply takes in a percept, determines which action could be applied, and does that action.
NOTE:
Action dependent on current percept only
Only works in fully observable environment
Action dependent on current percept only – this is quite a big condition to put on things!
CISC4/681 Introduction to Artificial Intelligence

26. Simply Reflexive Vacuum Agent
Implements the agent function (described in earlier table)
CISC4/681 Introduction to Artificial Intelligence

27. Model-Based Reflex Agent
To handle partially observable, keep track of the part of the world not seen now. For this we keep an internal state – it needs two kinds of information to work – how does the world evolve independently of the agent, and information about how the agent’s actions affects the world.
CISC4/681 Introduction to Artificial Intelligence

28. Model-Based Reflex Agent
Upon getting a percept
Update the state (given the current state, the action you just did, and the observations)
Choose a rule to apply (whose conditions match the state)
Schedule the action associated with the chosen rule
CISC4/681 Introduction to Artificial Intelligence

29. Goal Based Agent CISC4/681 Introduction to Artificial Intelligence
Knowing the current state is not always enough - in addition often need some sort of goal information that describes situations that are desirable. Sometimes action selection is simple, but often need to plan further into the future to reach the desired goal. This is the kind of environment we will spend more time on… search and planning.
The reasoning here if fundamentally different because it involves reasoning about the future and what will “make the agent happy”.
CISC4/681 Introduction to Artificial Intelligence

30. Utility-Based Agent CISC4/681 Introduction to Artificial Intelligence
Even goals are not enough for every situation. Utility functions can help when: (1) there are conflicting goals (e.g., speed and safety which need to be traded off against each other), and (2) when there are several goals to go for and the agent must reason about the likelihood of success that can be weighed against the importance of the goals.
CISC4/681 Introduction to Artificial Intelligence

31. Learning agents Finally, learning agents…
These agents look quite different and consist of four parts:

32. Learning Agent Components
Learning Element – responsible for making improvements (on what ever aspect is being learned…)
Performance Element – responsible for selecting external actions. In previous parts, this was the entire agent!
Critic – gives feedback on how agent is going and determines how performance element should be modified to do better in the future
Problem Generator – suggests actions for new and informative experiences.
Note the need for the problem generator – if the performance element had its way, it would keep doing actions that are the best and would never learn that there might be a better way to do things. The problem generator allows the agent to explore a little.
CISC4/681 Introduction to Artificial Intelligence

33. Summary Chapter 2
Agents interact with environments through actuators and sensors
The agent function describes what the agent does in all circumstances.
The performance measure evaluates the environment sequence.
A perfectly rational agent maximizes expected performance.
Agent programs implement (some) agent functions.
CISC4/681 Introduction to Artificial Intelligence

34. Summary (cont) PEAS descriptions define task environments.
Environments are categorized along several dimensions:
Observable? Deterministic? Episodic? Static? Discrete? Single-agent?
Several basic agent architectures exist:
Reflex, reflex with state, goal-based, utility-based
CISC4/681 Introduction to Artificial Intelligence