Intelligent Agents Russell and Norvig: 2 CISC4/681 Introduction to Artificial Intelligence

Agents Agent – perceives the environment through sensors and acts on it through actuators Percept – agent’s perceptual input (the basis for its actions) Percept Sequence – complete history of what has been perceived. What is an agent? Anything that can perceive its environment through sensors and act on it through actuators. Generally an agent’s actions may depend on the entire percept sequence observed to date. CISC4/681 Introduction to Artificial Intelligence

Agent Function Agent Function – maps a give percept sequence into an action; describes what the agent does. Externally – Table of actions Internally – Agent Program Agent function is an abstract mathematical description; the agent program is a concrete implementation running on the agent architecture. CISC4/681 Introduction to Artificial Intelligence

Vacuum Cleaner World Percepts: which square (A or B); dirt? Actions: move right, move left, suck, do nothing Agent function: maps percept sequence into actions Agent program: function’s implementation How should the program act? Illustrate these ideas… Very simply Vacuum cleaner world. The vacuum can perceive which location it is in (A or B), and whether or not there is dirt there. Simple agent function: if the current square is dirty, then suck, otherwise move to the other square. But, lots of other choices exist for how it should act. What is the right way? CISC4/681 Introduction to Artificial Intelligence

Agent Characterization Meant to be a tool for analyzing systems – not characterizing them as agent versus non-agent Lots of things can be characterized as agents (artifacts) that act on the world AI operates where Artifacts have significant computational resources Task Environments require nontrivial decision making AI operates at the most interesting end of the spectrum… CISC4/681 Introduction to Artificial Intelligence

Vacuum Cleaner World Percepts: which square (A or B); dirt? Actions: move right, move left, suck, do nothing Agent function: maps percept sequence into actions Agent program: function’s implementation How should the program act? So – let’s go back to this agent concept. What we need to think about is how this program should act. CISC4/681 Introduction to Artificial Intelligence

Rational Agent – does the right thing What does that mean? One that behaves as well as possible given the Environment in which it acts. How should success be measured? On consequences. Performance measure Embodies criterion for success Amount of dirt cleaned? Cleaned floors? Generally defined in terms of desired effect on environment (not on actions of agent) Defining measure not always easy! We want to design are agents to be “rational” and what that means is that they do the right thing. Agents actions cause the environment to go through different states – are they desirable? The notion of what it means to be desired is captured in a performance measure. E.g. not a good idea for performance measure to measure the amount of dirt cleaned in an hour – rational agent would clean, dump dirt, clean it up again, etc… For instance, is the performance measured on average cleanliness of the floor? Is it better to be moderately clean all the time – or sparkling clean a very small amount of the time (with a lot of time between cleaning)? CISC4/681 Introduction to Artificial Intelligence

Rationality Depends on: Performance measure that defines criterion for success. Agent’s prior knowledge of the environment. Actions the agent can perform. Agent’s percept sequence to date. For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has. So, if the agent moves from room to room and cleans it if it is dirty, and otherwise moves. In an environment where the rooms stay clean – this agent would be rational UNDER THESE CIRCUMSTANCES (even though it would keep oscillating between rooms). Note, if there were an electricity penalty in the performance measure, then this same agent would not be rational. CISC4/681 Introduction to Artificial Intelligence

Rationality For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has. Notice the rationality is dependent on EXPECTED maximization. Agent might need to learn how the environment changes, what action sequences to put together, etc… It takes perfection to maximize actual performance – this requires omnisciece. Take crossing street example…. Rationality not the same as perfection. CISC4/681 Introduction to Artificial Intelligence

Rationality For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has. Notice that an agent may be rational because the designer thought of everything, or it may have learned it itself (more autonomous) So, agent may need to learn not to cross the street without first getting the perceptual information to know whether or not a car is coming. Information gathering, exploration. CISC4/681 Introduction to Artificial Intelligence

Task Environment The “problems” for which rational agents are the “solutions” PEAS Description of Task Environment Performance Measure Environment Actuators (actions) Sensors (what can be perceived) In designing an agent, the first step must always be to specify the task environment as fully as possible. The book provides a number of examples of PEAS descriptions of various problems (taxi driver, medical diagnosis system, refinery controller, interactive English tutor). Let’s take a look at some of the properties of the task environment (they help us determine an appropriate agent design). CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Fully observable vs partially observable Fully observable gives access to complete state of the environment Complete state means aspects relevant to action choice global vs local dirt sensor CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Single Agent vs Multi-agent Single Agent – crossword puzzle Multi-agent – chess, taxi driving? (are other drivers best described as maximizing a performance element?) Multi-agent means other agents may be competitive or cooperative and may require communication Multi-agent may need communication CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Deterministic vs Stochastic Deterministic – next state completely determined by current state and action Uncertainty may arise because of defective actions or partially observable state (i.e., agent might not see everything that affects the outcome of an action). CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Episodic vs Sequential Episodic the agent’s experience divided into atomic episodes Next episode not dependent on actions taken in previous episode. E.g., assembly line Sequential – current action may affect future actions. E.g., playing chess, taxi short-term actions have long-term effects must think ahead in choosing an action CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Static vs Dynamic does environment change while agent is deliberating? Static – crossword puzzle Dynamic – taxi driver CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Discrete vs Continuous. Can refer to the state of the environment (chess has finite number of discrete states) the way time is handled (taxi driving continuous – speed and location of taxi sweep through range of continuous values) percepts and actions (taxi driving continuous – steering angles) CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Known vs Unknown This does not refer to the environment itself, but rather the agent’s knowledge of it and how it changes. If unknown, the agent may need to learn CISC4/681 Introduction to Artificial Intelligence

Properties of Task Environments (affect appropriate agent design) Easy: Fully observable, Deterministic, Episodic, Static, Discrete, Single agent. Hard: Partially observable, Sochastic, Sequential, Dynamic, Continuous, Multi-Agent CISC4/681 Introduction to Artificial Intelligence

Environment types Chess with Chess without Taxi driving a clock a clock Fully observable Yes Yes No Deterministic Strategic Strategic No Episodic No No No Static Semi Yes No Discrete Yes Yes No Single agent No No No The environment type largely determines the agent design The real world is (of course) partially observable, stochastic, sequential, dynamic, continuous, multi-agent

Agent Programs Need to develop agents – programs that take the current percept as input from the sensors and return an action to the actuators. Could just have a big table look-up (shown in the book) – where map each sequence of percepts into an action. Problem is that table is generally very large! So, can we do it with less code?? CISC4/681 Introduction to Artificial Intelligence

Possible Agent Program So, this is a table of percept sequences – and the table can get extremely large… probably not what we mean by an intelligent program! CISC4/681 Introduction to Artificial Intelligence

Agent Programs Need to develop agents – programs that take the current percept as input from the sensors and return an action to the actuators. The key challenge for AI is to find out how to write programs that, to the extent possible, produce rational behavior from a small amount of code. Could just have a big table look-up (shown in the book) – where map each sequence of percepts into an action. Problem is that table is generally very large! So, can we do it with less code?? CISC4/681 Introduction to Artificial Intelligence

Simple Reflective Agent Selects actions on the basis of the current percept ignoring the rest of the percept history. These agents work with a set of condition-action rules where the condition test whether or not some condition holds in the current percept information – if it does, then the action is done. CISC4/681 Introduction to Artificial Intelligence

Simple Reflexive Agent Handles simplest kind of world Agent embodies a set of condition-action rules If percept then action Agent simply takes in a percept, determines which action could be applied, and does that action. NOTE: Action dependent on current percept only Only works in fully observable environment Action dependent on current percept only – this is quite a big condition to put on things! CISC4/681 Introduction to Artificial Intelligence

Simply Reflexive Vacuum Agent Implements the agent function (described in earlier table) CISC4/681 Introduction to Artificial Intelligence

Model-Based Reflex Agent To handle partially observable, keep track of the part of the world not seen now. For this we keep an internal state – it needs two kinds of information to work – how does the world evolve independently of the agent, and information about how the agent’s actions affects the world. CISC4/681 Introduction to Artificial Intelligence

Model-Based Reflex Agent Upon getting a percept Update the state (given the current state, the action you just did, and the observations) Choose a rule to apply (whose conditions match the state) Schedule the action associated with the chosen rule CISC4/681 Introduction to Artificial Intelligence

Goal Based Agent CISC4/681 Introduction to Artificial Intelligence Knowing the current state is not always enough - in addition often need some sort of goal information that describes situations that are desirable. Sometimes action selection is simple, but often need to plan further into the future to reach the desired goal. This is the kind of environment we will spend more time on… search and planning. The reasoning here if fundamentally different because it involves reasoning about the future and what will “make the agent happy”. CISC4/681 Introduction to Artificial Intelligence

Utility-Based Agent CISC4/681 Introduction to Artificial Intelligence Even goals are not enough for every situation. Utility functions can help when: (1) there are conflicting goals (e.g., speed and safety which need to be traded off against each other), and (2) when there are several goals to go for and the agent must reason about the likelihood of success that can be weighed against the importance of the goals. CISC4/681 Introduction to Artificial Intelligence

Learning agents Finally, learning agents… These agents look quite different and consist of four parts:

Learning Agent Components Learning Element – responsible for making improvements (on what ever aspect is being learned…) Performance Element – responsible for selecting external actions. In previous parts, this was the entire agent! Critic – gives feedback on how agent is going and determines how performance element should be modified to do better in the future Problem Generator – suggests actions for new and informative experiences. Note the need for the problem generator – if the performance element had its way, it would keep doing actions that are the best and would never learn that there might be a better way to do things. The problem generator allows the agent to explore a little. CISC4/681 Introduction to Artificial Intelligence

Summary Chapter 2 Agents interact with environments through actuators and sensors The agent function describes what the agent does in all circumstances. The performance measure evaluates the environment sequence. A perfectly rational agent maximizes expected performance. Agent programs implement (some) agent functions. CISC4/681 Introduction to Artificial Intelligence

Summary (cont) PEAS descriptions define task environments. Environments are categorized along several dimensions: Observable? Deterministic? Episodic? Static? Discrete? Single-agent? Several basic agent architectures exist: Reflex, reflex with state, goal-based, utility-based CISC4/681 Introduction to Artificial Intelligence