1. 3-1 Chapter 3 DEDUCTIVE REASONING AGENTS

2. 3-2 Agent Architectures An agent is a computer system capable of flexible autonomous action… Issues one needs to address in order to build agent-based systems… Three types of agent architecture:  symbolic/logical (theorem proving, prolog-like)  reactive (no model of world)  hybrid (mixture of the two)

3. 3-3 Agent Architectures Kaelbling considers an agent architecture to be: ‘[A] specific collection of software (or hardware) modules, typically designated by boxes with arrows indicating the data and control flow among the modules. A more abstract view of an architecture is as a general methodology for designing particular modular decompositions for particular tasks.’

4. 3-4 Agent Architectures Originally (1956-1985), pretty much all agents designed within AI were symbolic reasoning agents Its purest expression proposes that agents use explicit logical reasoning in order to decide what to do Problems with symbolic reasoning led to a reaction against this — the so-called reactive agents movement, 1985–present. What were these problems? From 1990-present, a number of alternatives proposed: hybrid architectures, which attempt to combine the best of reasoning and reactive architectures

5. 3-5 Symbolic Reasoning Agents The classical approach to building agents is to view them as a particular type of knowledge-based system, and bring all the associated (discredited?!) methodologies of such systems to bear This paradigm is known as symbolic AI (similar to Prolog) We define a deliberative agent or agent architecture to be one that:  contains an explicitly represented, symbolic model of the world  makes decisions (for example about what actions to perform) via symbolic reasoning

6. 3-6 Symbolic Reasoning Agents If we aim to build an agent in this way, there are two key problems to be solved: 1. The transduction problem: that of translating the real world into an accurate, adequate symbolic description, in time for that description to be useful…vision, speech understanding, learning 2. The representation/reasoning problem: that of how to symbolically represent information about complex real-world entities and processes, and how to get agents to reason with this information in time for the results to be useful…knowledge representation, automated reasoning, automatic planning

7. 3-7 Example – strips planner A linear planner is one in which the order in which subproblems are solved is linearly related to the order in which the actions of the plan are executed. Divide and conquer: to create a plan to achieve a conjunction of goals, create a plan to achieve one goal, and then create a plan to achieve the rest of the goals. To achieve a list of goals: choose one of them to achieve. If it is not already achieved  choose an action that makes the goal true  achieve the preconditions of the action carry out the action achieve the rest of the goals.

8. 3-8 Look at Strips Planner handout

9. 3-9 Undoing Achieved Goals Example: consider trying to achieve achieve_all([carrying(fred,gift), agent_at(fred,lab2)], init, S, 18, N). Example: consider trying to achieve achieve_all([agent_at(fred,lab2),agent_at(fred,lab1)], init, S, 18, N). The STRIPS algorithm, as presented, is unsound. Achieving one subgoal may undo already achieved subgoals.

10. 3-10 I have made all “adjacencies” two way… S = do(move(fred, lab2, lab1), do(unlock(fred, door2), do(move(fred, usu, lab2), do(unlock(fred, door1), do(move(fred, mail, usu), do(pickup(fred, key1, mail), do(pickup(fred, money, mail), do(pickup(fred, key2, mail), do(move(fred, bank, mail), do(move(fred, lounge, bank), do(move(fred, downtown, lounge), do(move(fred, usu, downtown), do(move(fred, downtown, usu), do(move(fred, usu, downtown), do(move(fred, downtown, usu), do(move(fred, usu, downtown), do(move(fred, lab2, usu), init))))))))))))))))) N = 1

11. 3-11 Goal: get to lab, but must have key and money to get in. pre-conditions and post-conditions, threats. You would have to keep a list of conditions that you cannot alter. Then “threats” would tell you if you would undo conditions you’ve already achieved. You can have conflicting goals.

12. 3-12 Symbolic Reasoning Agents Most researchers accept that neither problem (transduction or reasoning) is anywhere near solved Underlying problem lies with the complexity of symbol manipulation algorithms in general: many (most) search-based symbol manipulation algorithms of interest are highly intractable Because of these problems, some researchers have looked to alternative techniques for building agents; Attractive, however, as the rules are executed (an executable specification) directly to produce the agents behavior

13. 3-13 Deductive Reasoning Agents Basic idea is to use logic to encode a theory stating the best action to perform in any given situation Let:   be this theory (typically a set of rules)   be a logical database that describes the current state of the world  Ac be the set of actions the agent can perform   ├   mean that  can be proved from  using 

14. 3-14 Deductive Reasoning Agents /* first try to find an action explicitly prescribed Brute force. Examine all choices to see where they lead*/ for each a  Ac do if  ├  Do ( a ) then /* action is to be done */ return a end-if end-for /* if that doesn’t work, try to find an action not excluded */ for each a  Ac do if  (not) ├   Do ( a ) then return a end-if end-for return null /* no action found */

15. 3-15 Planning Systems (in general) Planning systems find a sequence of actions that transforms an initial state into a goal state IG a1 a17 a142

16. 3-16 We see the un-directed (aimless?) branching behavior as the strips planner kept repeating what it had already done – going between two locations cyclically – because it could.

17. 3-17 The Blocks World The Blocks World (version 0) consists of equal sized blocks on a table A robot arm can manipulate the blocks using the actions:  UNSTACK(a, b)  STACK(a, b)  PICKUP(a)  PUTDOWN(a)

18. 3-18 The Blocks World We also use predicates to describe the world:  ON(A,B)  ONTABLE(B)  ONTABLE(C)  CLEAR(A)  CLEAR(C)  ARMEMPTY A BC

19. 3-19 Logical Formulas to Describe Facts Always True of the World And of course we can write general logical truths relating the predicates: [  x HOLDING(x) ]  ¬ ARMEMPTY  x [ ONTABLE(x)  ¬  y [ON(x,y)] ]  x [ ¬  y [ON(y, x)]  CLEAR(x) ] So…how do we use theorem-proving techniques to construct plans?

20. 3-20 Green’s Method Basically this states that we note the changes to a state produced by the application of a rule. Since Green's method is based on resolution, the following strong properties can be proved: the method is sound in that it produces only correct plans; and  it is complete in that it is guaranteed to produce a correct plan whenever one exists.  There is no restriction about the type of plan involved: we use it in creation of action blocks and conditional programs Add state variables to the predicates, and use a function DO that maps actions and states into new states DO: A x S  S Example: DO(UNSTACK(x, y), S) is a new state A series of actions result in a state such as DO(PICKUP(y),DO( PUTDOWN(x),DO(UNSTACK(x, y),init)))

21. 3-21 UNSTACK So to characterize the action UNSTACK we could write: [ CLEAR(x, s)  ON(x, y, s) ]  [HOLDING(x, DO(UNSTACK(x,y),s))  CLEAR(y, DO(UNSTACK(x,y),s))] We can prove what the new state is in terms of the old state. A B

22. 3-22 COMPONENTS Of Planning 1 choose the best rule based upon heuristics 2 apply this rule to create a new state 3 detect when a solution is found 4 detect dead ends so that they can be avoided 5 detect when a nearly solved state occurs & use special methods to make it a solved state

23. 3-23 Methods used find the differences between the current states and the goal states and then choosing the rules that reduce these differences (means end analysis). Example If we wish to travel by car to visit a friend the first thing to do is to fill up the car with fuel. If we do not have a car then we need to acquire one. The largest difference must be tackled first. To travel by car requires fuel, but if the vehicle is not there acquire the vehicle. If you can’t acquire the car, there is no reason to worry about fuel.

24. 3-24 Problems with Green’s method The frame problem  In the above we know that B is still on the table. This needs to be encoded into frame axioms that describe components of the state not affected by the operator. The qualification problem  If we resolve the frame problem the resulting description may still be inadequate. Do we need to encode that a block cannot be placed on top of itself? The ramification problem  After unstacking block A, previously, how do we know that A is no longer at its initial location? Not only is it hard to specify exactly what does not happen ( frame problem) it is hard to specify exactly what does happen.

25. 3-25 The Frame Problem Basically each operator has three lists of predicates associated with it, the first is a list of things that become true called ACHIEVES and the second are things that become false called DELETE. The third list is a set of PRECONDITIONS that must be true before the operator can be applied. Anything not in these lists is assumed to be unaffected by the operation.

26. 3-26 Planning Agents Since the early 1970s, the AI planning community has been closely concerned with the design of artificial agents Planning is essentially automatic programming: the design of a course of action that will achieve some desired goal Within the symbolic AI community, it has long been assumed that some form of AI planning system will be a central component of any artificial agent But in the mid 1980s, Chapman established some theoretical results which indicate that AI planners will ultimately turn out to be unusable in any time-constrained system

27. 3-27 Attempts at specifying reasoning

28. 3-28 AGENT0 AGENT0 is implemented as an extension to LISP Each agent in AGENT0 has 4 components:  a set of capabilities (things the agent can do)  a set of initial beliefs  a set of initial commitments (things the agent will do)  a set of commitment rules The key component, which determines how the agent acts, is the commitment rule set

29. 3-29 AGENT0 Each commitment rule contains  a message condition (request)  a mental condition (precondition)  an action On each ‘agent cycle’…  If the rule fires, then the agent becomes committed to the action (the action gets added to the agent’s commitment set)

30. 3-30 AGENT0 – early prototype language Actions may be  private: an internally executed computation, or  communicative: sending messages Messages are constrained to be one of three types:  “requests” to commit to action  “unrequests” to refrain from actions  “informs” which pass on information

31. 3-31 AGENT0

32. 3-32 AGENT0 A commitment rule: COMMIT( /* request from agent1 asking Do */ ( agent1, REQUEST, DO(time, action) ), ;;; msg condition ( B, Beliefs [now, Friend agent1] AND CAN(self, action) AND NOT [time, CMT(self, anyaction)] ), ;;; mental condition self, DO(time, action) )

33. 3-33 AGENT0 This rule may be paraphrased as follows: if I receive a message from agent which requests me to do action at time, and I believe that:  agent is currently a friend  I can do the action  At time, I am not committed to doing any other action then commit to doing action at time

34. 3-34 Concurrent METATEM Concurrent METATEM is a multi-agent language in which each agent is programmed by giving it a temporal logic specification of the behavior it should exhibit These specifications are executed directly in order to generate the behavior of the agent. Engine which runs the agent is distinct from the rules. Easy to modify. Temporal logic is classical logic augmented by modal operators for describing how the truth of propositions changes over time

35. 3-35 Temporal Logic propositions qualified in terms of time first studied by Aristotle Amir Pnueli (turing award 1996), Zohar Manna We know how to deal with true or false I am hungry – varies over time.  hungry now  always be hungry  I will be hungry until I eat something

36. 3-36 Used in Formal Verification In systems work  request will eventually be granted  resource never granted to two requestors simultaneously Reasons about a timeline In Branching logic (future worlds), reasons about multiple time lines.

37. 3-37 Notation varies between sources

38. 3-38 Concurrent METATEM For example...  important(agents) means “it is now, and will always be true that agents are important”  important(ConcurrentMetateM) means “sometime in the future, ConcurrentMetateM will be important”  important(Prolog) means “sometime in the past it was true that Prolog was important” (  friends(us)) U apologize(you) means “we are not friends until you apologize”  apologize(you) means “tomorrow (in the next state), you apologize”.

39. 3-39 Concurrent METATEM MetateM is a framework for directly executing temporal logic specifications The root of the MetateM concept is Gabbay’s separation theorem: Any arbitrary temporal logic formula can be rewritten in a logically equivalent past  future form. This past  future form can be used as execution rules A MetateM program is a set of such rules Execution proceeds by a process of continually matching rules against a “history”, and firing those rules whose antecedents are satisfied The future-time consequents become commitments which must subsequently be satisfied

40. 3-40 Concurrent METATEM An example MetateM program: the resource controller… First rule ensure that an ‘ask’ is eventually followed by a ‘give’ Second rule ensures that only one ‘give’ is ever performed at any one time There are algorithms for executing MetateM programs that appear to give reasonable performance Rules are independent Variables are universally quantified

41. 3-41 Concurrent METATEM For example, a ‘stack’ object’s interface: stack(pop, push)[popped, stackfull] {pop, push} = inputs {popped, stackfull} = outputs

42. 3-42 Concurrent METATEM To illustrate the language Concurrent MetateM in more detail, here are some example programs… Snow White has some sweets (resources), which she will give to the Dwarves (resource consumers) Here is Snow White, written in Concurrent MetateM: What does this mean? (inputs) [output] x,y are dwarfs With no temporal qualifier, we mean NOW

43. 3-43 She will always eventually give to a dwarf that asks She will only give to one dwarf at a time

44. 3-44 Concurrent METATEM The dwarf ‘eager’ asks for a sweet initially, and then whenever he has just received one, asks again Some dwarves are even less polite: ‘greedy’ just asks every time

45. 3-45 Concurrent METATEM Try at seats… Pavlov will ask once, but only asks again (eventually) if he gets something. Shadow will ask immediately after his friend Pavlov asks. Fortunately, some have better manners; ‘courteous’ only asks when ‘eager’ and ‘greedy’ have eaten ‘shy’ will only ask for a sweet when no-one else has just asked

46. 3-46 Concurrent METATEM Try at seats… Fortunately, some have better manners; ‘courteous’ only asks when ‘eager’ and ‘greedy’ have eaten ‘shy’ will only ask for a sweet when no-one else has just asked