1. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides1 Week 2: ARCHITECTURES FOR INTELLIGENT AGENTS Based on the slides from Wooldridge’s: An Introduction to MultiAgent Systems http://www.csc.liv.ac.uk/~mjw/pubs/imas

2. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 2 What is an Agent? The main point about agents is they are autonomous: capable of acting independently, exhibiting control over their internal state Thus: an agent is a computer system capable of autonomous action in some environment in order to meet its design objectives SYSTEM ENVIRONMENT input output Note that this definition focuses on 1 agent! This is the AI view – Other agents are implicit parts of the environment

3. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 3 Agent Architectures A key problem facing an agent is deciding which of its actions to perform to best satisfy its design objectives Agent architectures are software architectures for decision-making systems that are embedded in some environment

4. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 4 What is an Agent? Trivial (non-interesting/not intelligent) agents:  Thermostat – in a real environment  UNIX daemon (e.g., biff) – in a software environment An intelligent agent is a computer system capable of flexible autonomous action in some environment By flexible, we mean:  reactive  pro-active  social

5. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 5 Environments Accessible vs. inaccessible – if the agent can obtain complete, accurate, up-to-date information about the state of the environment Deterministic vs. non-deterministic – if any action has a single guarranteed effect, no uncertainty about the state resulting from an action Static vs. dynamic – if an environment can be assumed to remain unchanged except by the effects of the actions performed by the agent Discrete vs. continuous – if there is a fixed, finite number of actions and percepts in the environment, Details

6. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 6 Environments – Accessible vs. inaccessible An accessible environment is one in which the agent can obtain complete, accurate, up-to-date information about the environment’s state Most moderately complex environments (including, for example, the everyday physical world and the Internet) are inaccessible The more accessible an environment is, the simpler it is to build agents to operate in it

7. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 7 Environments – Deterministic vs. non-deterministic A deterministic environment is one in which any action has a single guaranteed effect — there is no uncertainty about the state that will result from performing an action The physical world can to all intents and purposes be regarded as non-deterministic Non-deterministic environments present greater problems for the agent designer

8. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 8 Environments - Episodic vs. non-episodic In an episodic environment, the performance of an agent is dependent on a number of discrete episodes, with no link between the performance of an agent in different scenarios Episodic environments are simpler from the agent developer’s perspective because the agent can decide what action to perform based only on the current episode — it need not reason about the interactions between this and future episodes

9. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 9 Environments - Static vs. dynamic A static environment is one that can be assumed to remain unchanged except by the performance of actions by the agent A dynamic environment is one that has other processes operating on it, and which hence changes in ways beyond the agent’s control Other processes can interfere with the agent’s actions (as in concurrent systems theory) The physical world is a highly dynamic environment

10. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 10 Environments – Discrete vs. continuous An environment is discrete if there are a fixed, finite number of actions and percepts in it Russell and Norvig give a chess game as an example of a discrete environment, and taxi driving as an example of a continuous one Continuous environments have a certain level of mismatch with computer systems Discrete environments could in principle be handled by a kind of “lookup table” Back

11. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 11 Reactivity If a program’s environment is guaranteed to be fixed, the program need never worry about its own success or failure – program just executes blindly  Example of fixed environment: compiler The real world is not like that: things change, actions may fail, information is incomplete. Many (most?) interesting environments are dynamic, non-deterministic, inaccessible Software is hard to build for dynamic domains: program must take into account possibility of failure – ask itself whether it is worth executing! A reactive system is one that maintains an ongoing interaction with its environment, and responds to changes that occur in it (in time for the response to be useful)

12. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 12 Proactiveness Reacting to an environment is easy (e.g., stimulus  response rules) But we generally want agents to do things for us Hence goal directed behavior Pro-activeness = generating and attempting to achieve goals; not driven solely by events; taking the initiative

13. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 13 Balancing Reactive and Goal-Oriented Behavior We want our agents to systematically work towards long-term goals We want our agents to be reactive, to recognize opportunities, responding to changing conditions in an appropriate (timely) fashion These two considerations can be at odds with one another Designing an agent that can balance the two remains an open research problem

14. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 14 Social Ability The real world is a multi-agent environment: we cannot go around attempting to achieve goals without taking others into account Some goals can only be achieved with the cooperation of others Similarly for many computer environments: witness the Internet Social ability in agents is the ability to interact with other agents (and possibly humans) via some kind of agent-communication language, and perhaps cooperate with others

15. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 15 Other Properties Other properties, sometimes discussed in the context of agency: mobility: the ability of an agent to move around an electronic network veracity: an agent will not knowingly communicate false information benevolence: agents do not have conflicting goals, and that every agent will therefore always try to do what is asked of it rationality: agent will act in order to achieve its goals, and will not act in such a way as to prevent its goals being achieved — at least insofar as its beliefs permit learning/adaption: agents improve performance over time

16. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 16 Agents and Objects Are agents just objects by another name? Object:  encapsulates some state  communicates via message passing  has methods, corresponding to operations that may be performed on this state

17. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 17 Agents and Objects Main differences:  agents are autonomous: agents embody stronger notion of autonomy than objects, and in particular, they decide for themselves whether or not to perform an action on request from another agent  agents are smart: capable of flexible (reactive, pro-active, social) behavior, and the standard object model has nothing to say about such types of behavior  agents are active: a multi-agent system is inherently multi-threaded, in that each agent is assumed to have at least one thread of active control

18. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 18 Objects do it for free… agents do it because they want to agents do it for money

19. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 19 Agents and Expert Systems Aren’t agents just expert systems by another name? Expert systems typically disembodied ‘expertise’ about some (abstract) domain of discourse (e.g., blood diseases) Example: MYCIN knows about blood diseases in humans  It has a wealth of knowledge about blood diseases, in the form of rules  A doctor can obtain expert advice about blood diseases by giving MYCIN facts, answering questions, and posing queries

20. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 20 Agents and Expert Systems Main differences:  agents situated in an environment: MYCIN is not aware of the world — only information obtained is by asking the user questions  agents act: MYCIN does not operate on patients Some real-time (typically process control) expert systems are agents

21. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 21 Intelligent Agents and AI Aren’t agents just the AI project? Isn’t building an agent what AI is all about? AI aims to build systems that can (ultimately) understand natural language, recognize and understand scenes, use common sense, think creatively, etc. — all of which are very hard So, don’t we need to solve all of AI to build an agent…?

22. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 22 Intelligent Agents and AI When building an agent, we simply want a system that can choose the right action to perform, typically in a limited domain We do not have to solve all the problems of AI to build a useful agent: a little intelligence goes a long way! Oren Etzioni, speaking about the commercial experience of NETBOT, Inc: “We made our agents dumber and dumber and dumber…until finally they made money.”

23. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 23 Agents as Intentional Systems When explaining human activity, it is often useful to make statements such as the following: Janine took her umbrella because she believed it was going to rain. Michael worked hard because he wanted to possess a PhD. These statements make use of a folk psychology, by which human behavior is predicted and explained through the attribution of attitudes, such as believing and wanting (as in the above examples), hoping, fearing, and so on The attitudes employed in such folk psychological descriptions are called the intentional notions

24. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 24 Agents as Intentional Systems The philosopher Daniel Dennett coined the term intentional system to describe entities ‘whose behavior can be predicted by the method of attributing belief, desires and rational acumen’ Dennett identifies different ‘grades’ of intentional system: ‘A first-order intentional system has beliefs and desires (etc.) but no beliefs and desires about beliefs and desires. …A second-order intentional system is more sophisticated; it has beliefs and desires (and no doubt other intentional states) about beliefs and desires (and other intentional states) — both those of others and its own’

25. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 25 Agents as Intentional Systems Is it legitimate or useful to attribute beliefs, desires, and so on, to computer systems?

26. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 26 Agents as Intentional Systems McCarthy argued that there are occasions when the intentional stance is appropriate: ‘To ascribe beliefs, free will, intentions, consciousness, abilities, or wants to a machine is legitimate when such an ascription expresses the same information about the machine that it expresses about a person. It is useful when the ascription helps us understand the structure of the machine, its past or future behavior, or how to repair or improve it. It is perhaps never logically required even for humans, but expressing reasonably briefly what is actually known about the state of the machine in a particular situation may require mental qualities or qualities isomorphic to them. Theories of belief, knowledge and wanting can be constructed for machines in a simpler setting than for humans, and later applied to humans. Ascription of mental qualities is most straightforward for machines of known structure such as thermostats and computer operating systems, but is most useful when applied to entities whose structure is incompletely known’.

27. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 27 Agents as Intentional Systems What objects can be described by the intentional stance? As it turns out, more or less anything can... consider a light switch: But most adults would find such a description absurd! Why is this? ‘It is perfectly coherent to treat a light switch as a (very cooperative) agent with the capability of transmitting current at will, who invariably transmits current when it believes that we want it transmitted and not otherwise; flicking the switch is simply our way of communicating our desires’. (Yoav Shoham)

28. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 28 Agents as Intentional Systems The answer seems to be that while the intentional stance description is consistent,... it does not buy us anything, since we essentially understand the mechanism sufficiently to have a simpler, mechanistic description of its behavior. (Yoav Shoham) Put crudely, the more we know about a system, the less we need to rely on animistic, intentional explanations of its behavior But with very complex systems, a mechanistic, explanation of its behavior may not be practicable As computer systems become ever more complex, we need more powerful abstractions and metaphors to explain their operation — low level explanations become impractical. The intentional stance is such an abstraction

29. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 29 Agents as Intentional Systems The intentional notions are thus abstraction tools, which provide us with a convenient and familiar way of describing, explaining, and predicting the behavior of complex systems Remember: most important developments in computing are based on new abstractions:  procedural abstraction  abstract data types  objects Agents, and agents as intentional systems, represent a further, and increasingly powerful abstraction So agent theorists start from the (strong) view of agents as intentional systems: one whose simplest consistent description requires the intentional stance

30. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 30 Agents as Intentional Systems This intentional stance is an abstraction tool — a convenient way of talking about complex systems, which allows us to predict and explain their behavior without having to understand how the mechanism actually works Now, much of computer science is concerned with looking for abstraction mechanisms (witness procedural abstraction, ADTs, objects,…) So why not use the intentional stance as an abstraction tool in computing — to explain, understand, and, crucially, program computer systems? This is an important argument in favor of agents

31. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 31 Agents as Intentional Systems Other 3 points in favor of this idea: Characterizing Agents:  It provides us with a familiar, non-technical way of understanding & explaining agents Nested Representations:  It gives us the potential to specify systems that include representations of other systems  It is widely accepted that such nested representations are essential for agents that must cooperate with other agents

32. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 32 Agents as Intentional Systems Post-Declarative Systems:  This view of agents leads to a kind of post-declarative programming: In procedural programming, we say exactly what a system should do In declarative programming, we state something that we want to achieve, give the system general info about the relationships between objects, and let a built-in control mechanism (e.g., goal-directed theorem proving) figure out what to do With agents, we give a very abstract specification of the system, and let the control mechanism figure out what to do, knowing that it will act in accordance with some built-in theory of agency (e.g., the well-known Cohen-Levesque model of intention)

33. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 33 Abstract Architecture for Agents Assume the environment may be in any of a finite set E of discrete, instantaneous states: Agents are assumed to have a repertoire of possible actions available to them, which transform the state of the environment: A run, r, of an agent in an environment is a sequence of interleaved environment states and actions:

34. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 34 Abstract Architecture for Agents Let:  R be the set of all such possible finite sequences (over E and Ac )  R Ac be the subset of these that end with an action  R E be the subset of these that end with an environment state

35. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 35 State Transformer Functions A state transformer function represents behavior of the environment: Note that environments are…  history dependent  non-deterministic If  ( r )= , then there are no possible successor states to r. In this case, we say that the system has ended its run Formally, we say an environment Env is a triple Env =  E, e 0,  where: E is a set of environment states, e 0  E is the initial state, and  is a state transformer function

36. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 36 Agents Agent is a function which maps runs to actions: An agent makes a decision about what action to perform based on the history of the system that it has witnessed to date. Let AG be the set of all agents

37. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 37 Systems A system is a pair containing an agent and an environment Any system will have associated with it a set of possible runs; we denote the set of runs of agent Ag in environment Env by R (Ag, Env) (We assume R (Ag, Env) contains only terminated runs)

38. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 38 Systems Formally, a sequence represents a run of an agent Ag in environment Env =  E, e 0,  if: 1. e 0 is the initial state of Env 2.  0 = Ag(e 0 ) ; and 3. For u > 0,

39. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 39 Purely Reactive Agents Some agents decide what to do without reference to their history — they base their decision making entirely on the present, with no reference at all to the past We call such agents purely reactive: A thermostat is a purely reactive agent

40. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 40 Perception Now introduce perception system: Environment Agent seeaction

41. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 41 So far – agents were functions. But how to construct them?? Let’s refine our abstract model by breaking it into sub-systems  agent architecture What data structures and internal operations on these data structures, the internal flow of control. First – let’s consider an agent with two components:  perception and action

42. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 42 Perception The see function is the agent’s ability to observe its environment, whereas the action function represents the agent’s decision making process Output of the see function is a percept: see : E  Per which maps environment states to percepts, and action is now a function action : Per*  A which maps sequences of percepts to actions Note: in this way we can represent agents, whose decision making is influenced by the history of percepts

43. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 43 Agents with State We now consider agents that maintain state: Environment Agent seeaction next state

44. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 44 Agents with State These agents have some internal data structure, which is typically used to record information about the environment state and history. Let I be the set of all internal states of the agent. The perception function see for a state-based agent is unchanged: see : E  Per The action-selection function action is now defined as a mapping action : I  Ac from internal states to actions. An additional function next is introduced, which maps an internal state and percept to an internal state: next : I  Per  I

45. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 45 Agent Control Loop 1. Agent starts in some initial internal state i 0 2. Observes its environment state e, and generates a percept see ( e ) 3. Internal state of the agent is then updated via next function, becoming next ( i 0, see ( e )) 4. The action selected by the agent is action ( next ( i 0, see ( e ))) 5. Goto 2

46. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 46 Tasks for Agents We build agents in order to carry out tasks for us The task must be specified by us… But we want to tell agents what to do without telling them how to do it

47. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 47 Utility Functions over States One possibility: associate utilities with individual states — the task of the agent is then to bring about states that maximize utility A task specification is a function u : E   which associates a real number with every environment state

48. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 48 Utility Functions over States But what is the value of a run…  minimum utility of state on run?  maximum utility of state on run?  sum of utilities of states on run?  average? Disadvantage: difficult to specify a long term view when assigning utilities to individual states (One possibility: a discount for states later on.)

49. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 49 Utilities over Runs Another possibility: assigns a utility not to individual states, but to runs themselves: u : R   Such an approach takes an inherently long term view Other variations: incorporate probabilities of different states emerging Difficulties with utility-based approaches:  where do the numbers come from?  we don’t think in terms of utilities!  hard to formulate tasks in these terms

50. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 50 Utility in the Tileworld Simulated two dimensional grid environment on which there are agents, tiles, obstacles, and holes An agent can move in four directions, up, down, left, or right, and if it is located next to a tile, it can push it Holes have to be filled up with tiles by the agent. An agent scores points by filling holes with tiles, with the aim being to fill as many holes as possible TILEWORLD changes with the random appearance and disappearance of holes Utility function defined as follows:

51. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 51 The Tile-World Environment

52. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 52 Expected Utility & Optimal Agents Write P ( r | Ag, Env ) to denote probability that run r occurs when agent Ag is placed in environment Env Note: Then optimal agent Ag opt in an environment Env is the one that maximizes expected utility:

53. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 53 Bounded Optimal Agents Some agents cannot be implemented on some computers (A function Ag : R E  Ac may need more than available memory to implement) Write AG m to denote the agents that can be implemented on machine (computer) m : We can replace equation (1) with the following, which defines the bounded optimal agent Ag opt :

54. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 54 Agent Architectures An agent is a computer system capable of flexible autonomous action… Issues one needs to address in order to build agent-based systems… Three types of agent architecture:  symbolic/logical  reactive  hybrid

55. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 55 Agent Architectures We want to build agents, that enjoy the properties of autonomy, reactiveness, pro-activeness, and social ability that we talked about earlier This is the area of agent architectures Maes defines an agent architecture as: ‘[A] particular methodology for building [agents]. It specifies how… the agent can be decomposed into the construction of a set of component modules and how these modules should be made to interact. The total set of modules and their interactions has to provide an answer to the question of how the sensor data and the current internal state of the agent determine the actions… and future internal state of the agent. An architecture encompasses techniques and algorithms that support this methodology.’

56. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 56 Agent Architectures Kaelbling considers an agent architecture to be: ‘[A] specific collection of software (or hardware) modules, typically designated by boxes with arrows indicating the data and control flow among the modules. A more abstract view of an architecture is as a general methodology for designing particular modular decompositions for particular tasks.’

57. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 57 Predicate Task Specifications A special case of assigning utilities to histories is to assign 0 (false) or 1 (true) to a run If a run is assigned 1, then the agent succeeds on that run, otherwise it fails Call these predicate task specifications Denote predicate task specification by . Thus  : R  {0, 1}.

58. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 58 Task Environments A task environment is a pair  Env,  where Env is an environment,  : R  {0, 1} is a predicate over runs. Let TE be the set of all task environments. A task environment specifies:  the properties of the system the agent will inhabit  the criteria by which an agent will be judged to have either failed or succeeded

59. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 59 Task Environments Write R  ( Ag, Env ) to denote set of all runs of the agent Ag in environment Env that satisfy , i.e. are successful: We then say that an agent Ag succeeds in task environment  Env,  if Pessimistic! How to give an optimistic definition of success for an agent?

60. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 60 The Probability of Success Let P ( r | Ag, Env ) denote probability that run r occurs if agent Ag is placed in environment Env Then the probability P (  | Ag, Env ) that  is satisfied by Ag in Env would then simply be:

61. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 61 Achievement & Maintenance Tasks Two most common types of tasks are achievement tasks and maintenance tasks: 1. Achievement tasks are those of the form “achieve state of affairs  ” 2. Maintenance tasks are those of the form “maintain state of affairs  ”

62. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 62 Achievement & Maintenance Tasks An achievement task is specified by a set G of “good” or “goal” states: G  E The agent succeeds if it is guaranteed to bring about at least one of these states (we do not care which one — they are all considered equally good). A maintenance goal is specified by a set B of “bad” states: B  E The agent succeeds in a particular environment if it manages to avoid all states in B — if it never performs actions which result in any state in B occurring

63. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 63 Agent Architectures Originally (1956-1985), pretty much all agents designed within AI were symbolic reasoning agents Its purest expression proposes that agents use explicit logical reasoning in order to decide what to do Problems with symbolic reasoning led to a reaction against this — the so-called reactive agents movement, 1985–present From 1990-present, a number of alternatives proposed: hybrid architectures, which attempt to combine the best of reasoning and reactive architectures

64. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 64 Symbolic Reasoning Agents The classical approach to building agents is to view them as a particular type of knowledge-based system, and bring all the associated (discredited?!) methodologies of such systems to bear This paradigm is known as symbolic AI We define a deliberative agent or agent architecture to be one that:  contains an explicitly represented, symbolic model of the world  makes decisions (for example about what actions to perform) via symbolic reasoning

65. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 65 Robotic Agent with Symbolic Representation of the Environment Knowledge base (beliefs): Dist (me, b)= 3 Block (b); Interp: pixels Plan: Stop! Action: Brake!

66. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 66 Symbolic Reasoning Agents If we aim to build an agent in this way, there are two key problems to be solved: 1. The transduction problem: that of translating the real world into an accurate, adequate symbolic description, in time for that description to be useful… vision, speech understanding, learning 2. The representation/reasoning problem: that of how to symbolically represent information about complex real-world entities and processes, and how to get agents to reason with this information in time for the results to be useful… knowledge representation, automated reasoning, automatic planning

67. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 67 Symbolic Reasoning Agents Most researchers accept that neither problem is anywhere near solved Underlying problem lies with the complexity of symbol manipulation algorithms in general: many (most) search-based symbol manipulation algorithms of interest are highly intractable Because of these problems, some researchers have looked to alternative techniques for building agents; we look at these later

68. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 68 Deductive Reasoning Agents How can an agent decide what to do using theorem proving? Basic idea is to use logic to encode a theory stating the best action to perform in any given situation Let:   be this theory (typically a set of rules)   be a logical database that describes the current state of the world  Ac be the set of actions the agent can perform   |   mean that  can be proved from  using 

69. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 69 Deductive Reasoning Agents /* try to find an action explicitly prescribed */ for each a  Ac do if  |  Do ( a ) then return a end-if end-for /* try to find an action not excluded */ for each a  Ac do if  |   Do ( a ) then return a end-if end-for return null /* no action found */

70. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 70 Deductive Reasoning Agents An example: The Vacuum World Goal is for the robot to clear up all dirt

71. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 71 Deductive Reasoning Agents Use 3 domain predicates to solve problem: In(x, y) agent is at (x, y) Dirt(x, y) there is dirt at (x, y) Facing(d) the agent is facing direction d Possible actions: Ac = {turn, forward, suck} P.S. turn means “turn right”

72. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 72 Deductive Reasoning Agents Rules  for determining what to do: In (x,y) /\ Dirt (x,y)  Do (suck) …and so on! Using these rules (+ other obvious ones), starting at (0, 0) the robot will clear up dirt

73. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 73 Deductive Reasoning Agents Problems:  How to convert video camera input to Dirt(0, 1) ?  decision making assumes a static environment: calculative rationality (decision-making that suggests an action that is optimal when the decision-making process began)  decision making using first-order logic is undecidable! Even where we use propositional logic, decision making in the worst case means solving co-NP- complete problems (PS: co-NP-complete = bad news!) Typical solutions:  weaken the logic  use symbolic, non-logical representations  shift the emphasis of reasoning from run time to design time We will look at some examples of these approaches co-NP is the complexity class that contains the complements of decision problems in the complexity class NP.decision problemsNP

74. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 74 More Problems… The “logical approach” that was presented implies adding and removing things from a database -- that’s not pure logic Representing and reasoning about dynamic real world environments is extremely hard  Representing & Reasoning about temporal information  Representing Procedural Knowledge in traditional logic is unintuitive, cumbersome

75. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 75 Summary – logic-based approaches to building agents Decision making is viewed as deduction The agent’s decision making strategy is encoded as a logical theory The process of selecting an action is reduced to a proof Advantages: Elegant, clean semantics Disadvantages:  Can not operate effectively since theorem proving is proven to be complex  Calculative rationality is good only for static domains

76. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 76 Example: theorem proving applied to planning systems Planning systems find a sequence of actions that transforms an initial state into a goal state IG a1 a17 a142

77. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 77 Planning Planning involves issues of both Search and Knowledge Representation Sample planning systems:  Robot Planning (STRIPS)  Planning of biological experiments (MOLGEN)  Planning of speech acts For purposes of exposition, we use a simple domain – The Blocks World

78. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 78 The Blocks World The Blocks World (today) consists of equal sized blocks on a table A robot arm can manipulate the blocks using the actions:  UNSTACK(a, b)  STACK(a, b)  PICKUP(a)  PUTDOWN(a)

79. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 79 The Blocks World We also use predicates to describe the world:  ON(A,B)  ONTABLE(B)  ONTABLE(C)  CLEAR(A)  CLEAR(C)  ARMEMPTY A BC In general: ON(a,b) HOLDING(a) ONTABLE(a) ARMEMPTY CLEAR(a)

80. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 80 Logical Formulas to Describe Facts Always True of the World And of course we can write general logical truths relating the predicates: [  x HOLDING(x) ]  ¬ ARMEMPTY  x [ ONTABLE(x)  ¬  y [ON(x,y)] ]  x [ ¬  y [ON(y, x)]  CLEAR(x) ] So…how do we use theorem-proving techniques to construct plans?

81. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 81 Green’s Method Add state variables to the predicates, and use a function DO that maps actions and states into new states DO: A x S  S Example: DO(UNSTACK(x, y), S) is a new state

82. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 82 UNSTACK So to characterize the action UNSTACK we could write: [ CLEAR(x, s)  ON(x, y, s) ]  [HOLDING(x, DO(UNSTACK(x,y),s))  CLEAR(y, DO(UNSTACK(x,y),s))] We can prove that if S0 is ON(A,B,S0)  ONTABLE(B,S0)  CLEAR(A, S0) then HOLDING(A,DO(UNSTACK(A,B),S0))  CLEAR(B,DO(UNSTACK(A,B),S0)) S1 A B

83. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 83 More Proving The proof could proceed further; if we characterize PUTDOWN: HOLDING(x,s)  ONTABLE(x,DO(PUTDOWN(x),s)) Then we could prove: ONTABLE(A, DO(PUTDOWN(A), DO(UNSTACK(A,B), S0))) The nested actions in this constructive proof give you the plan: 1. UNSTACK(A,B); 2. PUTDOWN(A) S1 S2

84. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 84 More Proving So if we have in our database: ON(A,B,S0)  ONTABLE(B,S0)  CLEAR(A,S0) and our goal is  s (ONTABLE(A, s)) we could use theorem proving to find the plan But could I prove: ONTABLE(B, DO(PUTDOWN(A), DO(UNSTACK(A,B), S0))) A B S1 S2 ?

85. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 85 The Frame Problem How do you determine what changes and what doesn’t change when an action is performed? One solution: “Frame axioms” that specify how predicates can remain unchanged after an action Example: 1. ONTABLE(z, s)  ONTABLE(z,DO(UNSTACK(x,y),s)) 2. [ON(m, n, s)  DIFF(m, x)]  ON(m,n,DO(UNSTACK(x,y),s))

86. slightly adapted from Jeff Rosenshein's version of Wooldridge's slides 86 Frame Axioms Problem: Unless we go to a higher-order logic, Green’s method forces us to write many frame axioms Example: COLOR(x, c, s)  COLOR(x,c,DO(UNSTACK(y,z),s)) We want to avoid this…other approaches are needed…