1. Chapter 2 INTELLIGENT AGENTS Agents and Environments The concept of rationality The nature of environment The structure of Agents Types of Agents Learning Agent

2. shiwani gupta2 Agents An agent is an entity that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors/actuators to achieve goals An ideal agent is one that always takes the action that is expected to maximize its performance measure, given the percept sequence it has seen so far. A Rational Agent is one that does an action that will cause it to be most successful Human agent: eyes, ears and other organs for sensors; hands, legs, mouth and other body parts for effectors Robotic agent: cameras and infrared range finders for sensors; various motors for effectors A software agent has encoded bit strings as its percepts and actions

3. shiwani gupta3 Rational Agent what is rational at any given time depends on four things: The performance measure that defines degree of success. Everything that agent has perceived so far - percept sequence. What the agent knows about the environment. The actions that the agent can perform. This leads to a definition of an ideal rational agent: For each possible percept sequence, an ideal rational agent should do whatever action is expected to maximize its performance measure, on the basis of the evidence provided by the percept sequence and whatever built-in knowledge the agent has. Doing actions in order to obtain useful information Perfect rationality might be unachievable due to computational limitations. Thus design best program given available resources.

4. shiwani gupta4 Agent The agent function maps from percept sequence to actions: [f: P*  A ] The agent program runs on the physical architecture to produce f agent = architecture + program

5. shiwani gupta5 A Simple intelligent agent architecture

6. shiwani gupta6 Agent contd… Consider a very simple agent: the square-root function on a calculator. The percept sequence for this agent is a sequence of keystrokes representing a number, and the action is to display a number on the display screen. The ideal mapping is that when the percept is a positive number x, the right action is to display a positive number z such that z² « x, accurate to, say, 15 decimal places. agent program: a function that implements the agent mapping from percepts to actions. We assume this program will run on some sort of computing device, which we will call the architecture. Obviously, the program we choose has to be one that the architecture will accept and run. The architecture might be a plain computer, or it might include special-purpose H/W for certain tasks, such as processing camera images or filtering audio input. In general, the architecture makes the percepts from the sensors available to the program, runs the program, and feeds the program's action choices to the effectors as they are generated.

7. shiwani gupta7 Software Agents Software agents (or software robots or softbots) exist in rich, unlimited domains. Imagine a softbot designed to fly a flight simulator for a 747. The simulator is a very detailed, complex environment, and the software agent must choose from a wide variety of actions in real time. Or Imagine a softbot designed to scan online news sources and show the interesting items to its customers. The appropriate design of the agent program depends on the percepts, actions, goals, and environment.

8. shiwani gupta8

9. 9 Example of Intelligent Agent

10. shiwani gupta10 Performance measure (the criteria that determines how successful an agent is) How happy the agent is with its performance – Some unable to answer – Others delude themselves (sour grapes) Thus objective performance measure eg. Vacuum cleaning agent – amount of dirt cleaned in single 8 hour shift – amount of electricity consumed and noise generated Thus map percept sequence to actions

11. shiwani gupta11 Autonomous agent One which is based completely on built in knowledge and doesn’t pay attention to its percepts

12. shiwani gupta12 Agent program types Four basic types in order of increasing generality Simple reflex agents Reflex agents respond immediately to percepts Model-based reflex agents Reflexive with state information Goal-based agents Goal-based agents act so that they will achieve their goal(s) Utility-based agents Utility-based agents try to maximize their own "happiness."

13. shiwani gupta13 Simple reflex agents (agents that keep track of the world)

14. shiwani gupta14 a condition-action rule written as if car-in-front-is-braking then initiate-braking learned responses (as for driving) innate reflexes (such as blinking when something approaches the eye). Rectangles to denote the current internal state of the agent's decision process Ovals to represent the background information used in the process. The simple reflex agent will work only if the correct decision can be made on the basis of the current percept. Simple Reflex Agent

15. shiwani gupta15 Example: Vacuum cleaner world 2 locations: square A, square B Agent perceives location and contents (dirty/not dirty) Actions: left, right, suck, no_op

16. shiwani gupta16 A vacuum cleaner agent What’s the ‘right’ way to fill out the table? ‘Right’ way makes agent good/intelligent

17. shiwani gupta17 Is the vacuum cleaner agent rational? Rational under the following assumptions: –Performance measure: 1 point for each clean square over ‘lifetime’ of 1000 steps –Clean squares stay clean, sucking cleans squares –Left and Right don’t take agent outside environment –Available actions: Left, Right, Suck, NoOp –Agent knows where it is and whether that location contains dirt But notice that under different assumptions, this vacuum cleaner agent would not be rational –Performance measure penalty for unnecessary movement –If clean squares become dirty –If environment is unknown, contains more than A and B

18. Performance measure Performance measures of a vacuum-cleaner agent: amount of dirt cleaned up, amount of time taken, amount of electricity consumed, level of noise generated, etc. Performance measures self-driving car: time to reach destination (minimize), safety, predictability of behavior for other agents, reliability, etc. Performance measure of game-playing agent: win/loss percentage (maximize), robustness, unpredictability (to “confuse” opponent), etc.

19. shiwani gupta19 Model-based reflex agents

20. shiwani gupta20 Model Based Reflex agents Also termed reflex agents with state or Context driven agents Need model of world for how actions affect the world Thus, even for the simple braking rule, our driver will have to maintain some sort of internal state in order to choose an action (Representation and Reasoning). Updating this internal state : First, we need some information about how the world evolves independently of the agent. Second, we need some information about how the agent's own actions affect the world

21. shiwani gupta21 Goal-based agents

22. shiwani gupta22 Goal based agents Knowing about the current state of the environment is not always enough to decide what to do. For example, at a road junction The agent needs some sort of goal information, which describes situations that are desirable—for example, being at the passenger's destination. Search (Problem Solving Agent) and planning are the subfields of AI Although the goal-based agent appears less efficient, it is far more flexible. If it starts to rain… For the reflex agent, on the other hand, we would have to rewrite a large number of condition-action rules. The goal-based agent is also more flexible with respect to reaching different destinations. Simply by specifying a new destination, we can get the goal-based agent to come up with a new behavior. The reflex agent's rules will only work for a single destination; they must all be replaced to go somewhere new.

23. shiwani gupta23 Utility-based agents

24. shiwani gupta24 Utility based agents Goals alone are not really enough to generate high-quality behavior. For example, there are many action sequences that will get the taxi to its destination, thereby achieving the goal, but some are quicker, safer, more reliable, or cheaper than others. Goals just provide a crude distinction between "happy" and "unhappy" states, whereas a more general performance measure should allow a comparison of different world states (or sequences of states). The customary terminology is to say that if one world state is preferred to another, then it has higher utility for the agent. Utility is therefore a function that maps a state onto a real number (Decision Making), which describes the associated degree of happiness in two kinds of cases where goals have trouble. First, when there are conflicting goals, only some of which can be achieved (for example, speed and safety), the utility function specifies the appropriate trade-off. Second, when there are several goals that the agent can aim for, none of which can be achieved with certainty, utility provides a way in which the likelihood of success can be weighed up against the importance of the goals.

25. shiwani gupta25 Learning agents

26. shiwani gupta26 Environment types Accessible/ Fully observable (vs. inaccessible/ partially observable): – If an agent's sensory apparatus gives it access to the complete state of the environment, then we say that the environment is accessible to that agent. – An accessible environment is convenient because the agent need not maintain any internal state to keep track of the world. – An environment might be partially observable because of noisy and inaccurate sensors. – Chess is fully observable – taxi driving is partially observable – Vacuum agent with only local dirt sensor is partially observable

27. shiwani gupta27 Environment types Episodic (vs. sequential): – In an episodic environment, the agent's experience is divided into "episodes." Each episode consists of the agent perceiving and then acting. – The quality of its action depends just on the episode itself, because subsequent episodes do not depend on what actions occur in previous episodes. – Episodic environments are much simpler because the agent does not need to think ahead. – In sequential environment, current decision could affect all future decisions – part-picking robot is Episodic – chess is Sequential

28. shiwani gupta28 Environment types Static (vs. dynamic): – If the environment can change while an agent is deliberating, then we say the environment is dynamic for that agent; otherwise it is static. – Static environments are easy to deal with because the agent need not keep looking at the world while it is deciding on an action, nor need it worry about the passage of time. – Dynamic environments are continuously asking the agent what it wants to do – If the environment does not change with the passage of time but the agent's performance score does, then we say the environment is semidynamic. – Cross word puzzles are static – Chess is semidynamic – taxi driving is dynamic

29. shiwani gupta29 Environment types Discrete (vs. continuous): – If there are a limited number of distinct, clearly defined percepts and actions we say that the environment is discrete. – Chess has finite no. of distinct states and discrete set of percepts and actions – taxi driving is cont state and cont time prob

30. shiwani gupta30 Environment types Deterministic (vs. stochastic): – If the next state of the environment is completely determined by the current state and the actions selected by the agents, then we say the environment is deterministic. – In principle, an agent need not worry about uncertainty in an accessible, deterministic environment. – If the environment is inaccessible, however, then it may appear to be nondeterministic. – If environment is deterministic except for actions of other agents, we say environment is strategic – Chess is deterministic – taxi driving is stochastic

31. shiwani gupta31 Environment types Single agent (vs. multiagent): An agent operating by itself in an environment. – An agent solving a crossword puzzle by itself – A agent playing chess in a 2 agent environment Multiagent – Competitive: Chess – Cooperative: Taxi Driving Some environments are more demanding than others. Environments that are inaccessible, nondeterministic, nonepisodic, dynamic, and continuous are the most challenging.

32. shiwani gupta32

33. shiwani gupta33 Environment summary Solitaire: observable, deterministic, sequential, static, discrete, single-agent Backgammon: observable, deterministic, sequential, semi- static, discrete, multi-agent Internet shopping: partially observable, partially deterministic, sequential, semi-static, discrete, single-agent (except auctions) Taxi driving (“the real world”): partially observable, not deterministic, sequential, dynamic, continuous, multi-agent

34. shiwani gupta34 Task Environment Design of rational agent require definition of task environment Defined by Performance measure Environment Actuators Sensors PEAS is first step in designing an agent Do PEAS for automated taxi driver and internet shopping agent

35. shiwani gupta35 PEAS for an automated taxi driver Performance – safe, fast, legal, comfortable, max profit Environment – Road, other cars, pedestrians, customers Actuators – Steering, accelerator, brake, signal, horn, display Sensors – Camera, laser, sonar, speedometer, GPS, odometer, engine light, keyboard

36. shiwani gupta36 PEAS for internet shopping agent Performance – Price, quality, efficiency, delivery time Environment – www sites, vendors, shippers Actuators – Order, follow URL, fill in form, pay Sensors – HTML pages, text graphics, scripts

37. shiwani gupta37 Question Bank Describe environment simulator programs with performance measure that can be used as test beds for agent programs. Consider vacuum cleaner problem and explain – How is it rational? Which behavior will be irrational. – Give success function. Explain performance measure.  Give PEAS description for ROBOT Soccer player. Characterize its environment. [MU May 16]