1. Koen HindriksMulti-Agent Systems Environments, Rules, Architectures Koen Hindriks Delft University of Technology, The Netherlands

2. Koen Hindriks Multi-Agent Systems 2011 2 Koen HindriksMulti-Agent Systems 2 Question Q: Which three goals does agent stackBuilder have? Hint: Inspect the knowledge base and think about its combination with goals in G OAL … A: third goal is clear(table). Knowledge implies goals... But not achievement goal! i.e. not: a-goal(clear(table)). init module{ knowledge{ block(X) :- on(X, Y). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } goals{ on(a, b), on(b, c), on(c, table). on(d, e), on(e, f), on(f, table). } actionspec{ move(X,Y) { pre{ clear(X),clear(Y),on(X,Z),not(on(X,Y)) } post{ not(on(X,Z)), on(X,Y) } } main module{ program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X, table). } Q: Why useful? A: Y=table which is always clear.

3. Koen Hindriks Multi-Agent Systems 2011 3 Koen HindriksMulti-Agent Systems 3 Macro Definitions instead of writing mental state conditions: you may use macros in rules: mental state condition is macro’s definition program{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table). } program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table). }

4. Koen Hindriks Multi-Agent Systems 2011 4 Koen HindriksMulti-Agent Systems 4 Macro Definitions Use to increase readability Macro definitions must be placed inside the program section, before any of the rules All variables in macro should occur in defining mental state condition. Other vars in condition are just placeholders. program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table). }

5. Koen Hindriks Multi-Agent Systems 2011 5 Koen HindriksMulti-Agent Systems 5 Macro Definitions 1.When is a block marked as obstructing? Informally explain the macro definition. 2.In which cases is the concept of an obstructing block useful? knowledge{ … above(X,Y) :- on(X,Y). above(X,Y) :- on(X,Z), above(Z,Y). tower([X]) :- on(X,table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } program{ #define inPosition(X) goal-a( tower([X|T]) ). #define obstructingBlock(X) a-goal( on(Y,Z) ), bel( above(X,Z); above(X,Y) ). … } EXERCISE

6. Koen Hindriks Multi-Agent Systems 2011 6 Koen HindriksMulti-Agent Systems 6 Rule Order in the Blocks World main: stackBuilder { knowledge{ block(X) :- on(X, Y). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } beliefs{ … } goals{ on(a,table), on(b,a), on(c,b), on(d,table), on(e,table), on(f,e), on(g,f), on(h,g), on(i,h). } program[order=random]{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X, table). } actionspec{ move(X,Y) { pre{ clear(X),clear(Y),on(X,Z),not(on(X,Y)) } post{ not(on(X,Z)), on(X,Y) } } Agent performs action: move(a,table) Agent performs action: move(g,table) Agent performs action: move(h,table) Agent performs action: move(b,a) Agent performs action: move(c,b) Agent performs action: move(d,table) Agent performs action: move(e,table) Agent performs action: move(f,e) Agent performs action: move(g,f) Agent performs action: move(h,g) Agent performs action: move(i,h) Agent performs action: move(a,table) Agent performs action: move(b,a) Agent performs action: move(c,table) Agent performs action: move(c,b) Agent performs action: move(g,table) Agent performs action: move(d,table) Agent performs action: move(h,table) Agent performs action: move(e,table) Agent performs action: move(f,e) Agent performs action: move(g,f) Agent performs action: move(h,g) Agent performs action: move(i,h) Trace 1: 11 actions performed Trace 2: 12 actions performed

7. Koen Hindriks Multi-Agent Systems 2011 7 Koen HindriksMulti-Agent Systems 7 Rule Order in the Blocks World main: stackBuilder { knowledge{ block(X) :- on(X, Y). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } beliefs{ … } goals{ on(a,table), on(b,a), on(c,b), on(d,table), on(e,table), on(f,e), on(g,f), on(h,g), on(i,h). } module main{ program[order=random]{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X, table). } actionspec{ move(X,Y) { pre{ clear(X),clear(Y),on(X,Z),not(on(X,Y)) } post{ not(on(X,Z)), on(X,Y) } } EXERCISE: Q: What is worst-case behavior? A:7 moves to table + 6 constructive = 13. Initial, arbitrary state. goal state

8. Koen Hindriks Multi-Agent Systems 2011 8 Koen HindriksMulti-Agent Systems 8 Rule Order in Main Module By default rules are evaluated in linear order Can be used to exploit “logical” order of rules. Additional advantage: –Possibly shorter mental state conditions in rules. –If Rule 2 is evaluated, you know that mental state condition of Rule 1 does not hold. program[order=linear]{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y).  RULE 1 if misplaced(X) then move(X, table).  RULE 2 }

9. Koen Hindriks Multi-Agent Systems 2011 9 Koen HindriksMulti-Agent Systems 9 Outline Environments and Perception Environments and Durative Actions Architectures Multiple actions

10. Koen Hindriks Multi-Agent Systems 2011 10 Koen HindriksMulti-Agent Systems 10 G OAL Architecture Agent Environment Real or simulated world of events action percept message Beliefs Goals Action Rules Process events

11. Koen Hindriks Multi-Agent Systems 2011 11 Koen HindriksMulti-Agent Systems DEMO: TOWERWORLD

12. Koen Hindriks Multi-Agent Systems 2011 12 Koen HindriksMulti-Agent Systems SENSING AND PERCEPTION

13. Koen Hindriks Multi-Agent Systems 2011 13 Koen HindriksMulti-Agent Systems 13 Sensing Agents need sensors to: –explore the environment when they have incomplete information (e.g. Wumpus World) –keep track of changes in the environment that are not caused by itself G OAL agents sense the environment through a perceptual interface defined between the agent and the environment. –Environment generates perceptions

14. Koen Hindriks Multi-Agent Systems 2011 14 Koen HindriksMulti-Agent Systems 14 Sensing in the Tower World Percepts are received by an agent in it’s percept base. The reserved keyword percept is wrapped around the percept content, e.g. percept(block(a)). Not automatically inserted into beliefs!

15. Koen Hindriks Multi-Agent Systems 2011 15 Koen HindriksMulti-Agent Systems 15 Processing Percepts The percept base is refreshed, i.e. emptied, every reasoning cycle of the agent. Agent has to decide what to do when it perceives something, i.e. receives a percept. Use percepts to update agent’s mental state: –Ignore the percept –Update the beliefs of the agent –Adopt a new goal

16. Koen Hindriks Multi-Agent Systems 2011 16 Koen HindriksMulti-Agent Systems 16 Updating Agent’s Mental State One way to update beliefs with percepts: First, delete everything agent believes. Example: remove all block and on facts. Second, insert new information about current state provided from percepts into belief base. Example: insert block and on facts for every percept(block(…)) and percept(on(…)). Assumes that environment is fully observable with respect to block and on facts. Downside: not very efficient…

17. Koen Hindriks Multi-Agent Systems 2011 17 Koen HindriksMulti-Agent Systems 17 Percept Update Pattern A typical pattern for updating is: Rule 1 If the agent –perceives block X is on top of block Y, and –does not believe that X is on top of Y Then insert on(X,Y) into the belief base. Rule 2 If the agent –believes that X is on top of Y, and –does not perceive block X is on top of block Y Then remove on(X,Y) from the belief base. Assumes full observability with respect to on facts.

18. Koen Hindriks Multi-Agent Systems 2011 18 Koen HindriksMulti-Agent Systems 18 Percepts and Event Module Percepts are processed in G OAL by means of event rules, i.e. rules in the event module. Event module is executed every time that agent receives new percepts. event module{ program{ <… rules …> }

19. Koen Hindriks Multi-Agent Systems 2011 19 Koen HindriksMulti-Agent Systems 19 Implementing Pattern Rule 1 Rule 1 If the agent –perceives block X is on top of block Y, and –does not believe that X is on top of Y Then insert on(X,Y) into the belief base. Note: percept base is inspected using the bel operator, e.g. bel(percept(on(X,Y))). event module { program{ % assumes full observability. if bel(percept(on(X,Y)), not(on(X,Y))) then insert(on(X,Y)). … } INCORRECT!

20. Koen Hindriks Multi-Agent Systems 2011 20 Koen HindriksMulti-Agent Systems 20 Implementing Pattern Rule 1 Rule 1 If the agent perceives block X is on top of block Y, and does not believe that X is on top of Y, then insert on(X,Y) into the belief base. We want to apply this rule for all percept instances that match it! event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). … } Content Percept Base percept(on(a,table)) percept(on(b,table)) percept(on(c,table)) percept(on(d,table)) …

21. Koen Hindriks Multi-Agent Systems 2011 21 Koen HindriksMulti-Agent Systems 21 Implementing Pattern Rule 2 Rule 2 If the agent –believes that X is on top of Y, and –does not perceive block X is on top of block Y Then remove on(X,Y) from the belief base. 1.We want that all rules are applied! By default the event module applies ALL rules in linear order. 2.Note that none of these rules fires if nothing changed. event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). forall bel(on(X,Y), not(percept(on(X,Y)))) do delete(on(X,Y)). }

22. Koen Hindriks Multi-Agent Systems 2011 22 Koen HindriksMulti-Agent Systems 22 Event Module for Tower World Tower World “informs” about: blocks present; whether gripper is holding a block or not; a block being on top of another or on the table. event module { program{ % assumes full observability. forall bel( block(X), not(percept(block(X))) ) do delete( block(X) ). forall bel( percept(block(X)), not(block(X)) ) do insert( block(X) ). forall bel( holding(X), not(percept(holding(X))) ) do delete( holding(X) ). forall bel( percept(holding(X)), not(holding(X)) ) do insert( holding(X) ). forall bel( on(X,Y), not(percept(on(X,Y))) ) do delete( on(X,Y) ). forall bel( percept(on(X,Y)), not(on(X,Y)) ) do insert( on(X,Y) ). }

23. Koen Hindriks Multi-Agent Systems 2011 23 Koen HindriksMulti-Agent Systems 23 Initially… Agent Knows Nothing In most environments an agent initially has no information about the state of the environment, e.g. Tower World, Wumpus World, … Represented by an empty belief base: There is no need to include a belief base in this case in a G OAL agent. It is ok to simply have no belief base section. beliefs{ }

24. Koen Hindriks Multi-Agent Systems 2011 24 Koen HindriksMulti-Agent Systems 24 Built-in Actions G OAL provides a number of built-in actions: insert( ) example: insert(on(X,Y), not(on(X,Z))) meaning: add on(X,Y) and delete on(X,Z) delete( ) example: delete(on(X,Y), not(on(X,Z))) meaning: delete on(X,Y) and add on(X,Z) adopt( ) example: adopt(on(a,b), on(b,c)) meaning: add a new goal to goal base (if not already implied by a goal) drop( ) example: drop(on(b,a), not(on(c,table))) meaning: remove all goals that imply from the goal base Note All actions need to be closed when they are executed.

25. Koen Hindriks Multi-Agent Systems 2011 25 Koen HindriksMulti-Agent Systems 25 Drop Action drop(on(b,a), not(on(c,table))) Is goal in goal base dropped? Check: does goal imply on(b,a), not(on(c,table)) ? A: Yes, so goal is removed by drop action. knowledge{ block(X) :- on(X, Y). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]):- on(X,Y),tower([Y|T]). } goals{ on(a,table), on(b,a), on(c,b), on(d,table), on(e,table), on(f,e), on(g,f), on(h,g), on(i,h). }

26. Koen Hindriks Multi-Agent Systems 2011 26 Koen HindriksMulti-Agent Systems 26 Summarizing Two types of rules: –if then. is applied at most once (if multiple instances chooses randomly) –forall do. is applied once for each instantiation of parameters that satisfy condition. Main module by default: –checks rules in linear order –applies first applicable rule (also checks action precondition!) Event module by default: –Checks rules in linear order –Applies all applicable rules (rules may enable/disable each other!) Program section modifiers: [order=random], [order=linear], [order=linearall], [order=randomall] Built-in actions: insert, delete, adopt, drop.

27. Koen Hindriks Multi-Agent Systems 2011 27 Koen HindriksMulti-Agent Systems ACTIONS IN ENVIRONMENTS

28. Koen Hindriks Multi-Agent Systems 2011 28 Koen HindriksMulti-Agent Systems 28 Instantaneous versus Durative Instantaneous actions Actions in the Blocks World environment are instantaneous, i.e. they do not take time. Wumpus World actions are of this type as well. Durative actions Actions in the Tower World environment take time. When a G OAL agent sends an action to such an environment, the action will not be completed immediately.

29. Koen Hindriks Multi-Agent Systems 2011 29 Koen HindriksMulti-Agent Systems 29 Durative Actions Deciding on a next action typically gets more complicated when actions take time! Agents may decide to perform an action before finishing another. Three cases: 1.Sending an action B while another action A is still ongoing overrides and terminates previous action A… Example: actions in Tower World, goto action in UT2004. 2.Action B that is sent while another action A is still ongoing is performed in parallel… both actions are performed. Example: shooting while running in UT2004. 3.Action B that is sent while another action A is still ongoing is simply ignored… nothing new happens. Less frequent.

30. Koen Hindriks Multi-Agent Systems 2011 30 Koen HindriksMulti-Agent Systems 30 Durative Actions and Sensing While durative actions are performed an agent may receive percepts. Useful to monitor progress of action. Tower World Example: gripper may no longer be holding block because user removed it. UT2004 Example: Other bot is perceived while moving.

31. Koen Hindriks Multi-Agent Systems 2011 31 Koen HindriksMulti-Agent Systems 31 Specifying Durative Actions Tower World has two actions: pickup and putdown block both take time. How should we specify these actions?  delayed effect problem. Solution: –do not specify postcondition, –make sure action effects are handled by event rules.

32. Koen Hindriks Multi-Agent Systems 2011 32 Koen HindriksMulti-Agent Systems 32 Specifying Durative Actions Tower World action specification: Postcondition may also be “empty”: post { } Better practice is to indicate that you have not forgotten to specify it by using post { true }. actionspec{ pickup(X) { pre{ clear(X), not(holding(Y)) } post{ true } } putdown(X,Y) { pre { holding(X), clear(Y) } post { true } } nil { pre { true } post { true } }

33. Koen HindriksMulti-Agent Systems Interpreter Architectures

34. Koen Hindriks Multi-Agent Systems 2011 34 Koen HindriksMulti-Agent Systems 34 Interpreters: G OAL Process events (= apply event rules) Select action (= apply action rules) Perform action (= send to environment) Update mental state (= apply action specs + commitment strategy) Also called reasoning or deliberation cycle. G OAL ’s cycle is a classic sense-plan-act cycle. Other languages may have more elaborate cycles based on e.g. nr of types of rules in the language.

35. Koen Hindriks Multi-Agent Systems 2011 35 Koen HindriksMulti-Agent Systems 35 Interpreter: 2APL Plan base

36. Koen Hindriks Multi-Agent Systems 2011 36 Koen HindriksMulti-Agent Systems 36 Interpreter: Jason

37. Koen Hindriks Multi-Agent Systems 2011 37 Koen HindriksMulti-Agent Systems MULTIPLE ACTIONS

38. Koen Hindriks Multi-Agent Systems 2011 38 Koen HindriksMulti-Agent Systems 38 Multiple Actions in Rule Using the + construct multiple actions can be combined and selected in a single rule. The actions are executed in same reasoning cycle Combined actions are executed in order (left-to-right) Example: rule to handle failure of forward action. program{ if bel( percept(bump), position(Xcurrent, Ycurrent), behind(Xcurrent, Ycurrent, X, Y) ) then delete(position(Xcurrent, Ycurrent)) + insert( position(X,Y), wall(Xcurrent, Ycurrent) ). ….. } program{ if then + + …. … }

39. Koen Hindriks Multi-Agent Systems 2011 39 Koen HindriksMulti-Agent Systems 39 Changing Agent’s Mind: Goals Changing One’s Mind Pattern: If a “better” goal G2 can be set, then replace a current goal G1 with that new goal G2. Example in Tower World Agent: program{ … if goal( holding(X) ), bel(not(clear(X)), constructiveMove(Z,Y) then drop( holding(X) ) + adopt( holding(Z) ). … } program{ … if goal( G1 ), then drop( G1 ) + adopt( G2 ). … }

40. Koen Hindriks Multi-Agent Systems 2011 40 Koen HindriksMulti-Agent Systems 40 Multiple Environment Actions How to turn 180 ◦ and face opposite direction in Wumpus World? Available actions: turn(left), turn(right), forward, … Solution: program{ … if then turn(left) + turn(left). … }

41. Koen Hindriks Multi-Agent Systems 2011 41 Koen HindriksMulti-Agent Systems 41 Multiple Environment Actions Warning: It may not be useful to combine environment actions. Environment actions may cancel each other out What happens with the following rule? Nothing… if gripper is already at “home position” See also slides on Durative Actions. program{ … if goal( holding(X) ) then pickup(X) + nil. … }

42. Koen Hindriks Multi-Agent Systems 2011 42 Koen HindriksMulti-Agent Systems WRITING GOAL AGENTS

43. Koen Hindriks Multi-Agent Systems 2011 43 Koen HindriksMulti-Agent Systems 43 General Guidelines –Design representation of environment –Code belief base –Identify goals –Code goal base –Identify percepts & events –Write event rules –Identify actions –Write action specifications –Determine action selection strategy –Write action rules a b c d e

44. Koen Hindriks Multi-Agent Systems 2011 44 Koen HindriksMulti-Agent Systems TOWERWORLD EXAMPLE

45. Koen Hindriks Multi-Agent Systems 2011 45 Koen HindriksMulti-Agent Systems 45 Tower World: Design Issue: Blocks are not moved instantaneously. We need to keep track of what we are doing! Q: What is specific problem in Tower World? A: We need to track which tower we’re building Idea: Agent sets target goal to move block. Choice based on block agent (wants) to hold. if a-goal( holding(X) ) then pickup(X). % put a block you're holding down, ideally where you want it, but otherwise put it on the table. if bel( holding(X) ) then { if constructiveMove(X,Y) then putdown(X, Y). if true then putdown(X, table). }

46. Koen Hindriks Multi-Agent Systems 2011 46 Koen HindriksMulti-Agent Systems 46 Tower World: Set Target Only adopt new goal if there is no target yet. Prefer making constructive moves. Note: Goal automatically removed when achieved… % Think before you act... if not(goal( holding(X) )), not(bel( holding(Y) )) then adoptgoal. module adoptgoal{ % order=linear: adopt at most one goal to hold a block at any one time. % the gripper is limited and cannot hold more than one block. program{ % a block X is an obstructing block if it prevents moving a block Y in position % either because there is a block above the target block, % or because a block above Y prevents moving it. #define obstructingBlock(X) a-goal( on(Y, Z) ), bel( above(X, Z); above(X, Y) ). % prefer making constructive moves. if constructiveMove(X, Y) then adopt( holding(X) ). % if no constructive move can be made aim for moving obstructing block. if obstructingBlock(X) then adopt( holding(X) ). }

47. Koen Hindriks Multi-Agent Systems 2011 47 Koen HindriksMulti-Agent Systems 47 Tower World: Change Management Dynamic environment: agent is not alone, also user. What if… block configuration is changed by user? Issue: is target (sub)goal still feasible? Solution: Check for all reasons to drop target. % configuration has changed, check whether there is a reason to drop the target. % check for reasons to drop or adopt a goal (goal management). if goal( holding(X) ) then { % first reason: cannot pick up block X because it's not clear. if bel( not(clear(X)) ) then drop( holding(X) ). % second reason: cannot pick up block X because now holding other block! if bel( holding(_) ) then drop( holding(X) ). % third reason: block X is already in position, don't touch it. if inPosition( X ) then drop( holding(X) ). % fourth reason: we can do better by moving another block constructively. listall L <- constructiveMove(Y,Z) do { if bel(not(L=[]), not(member([X,_],L))) then drop( holding(X) ). }

48. Koen Hindriks Multi-Agent Systems 2011 48 Koen HindriksMulti-Agent Systems 48 Organisation Tutorial this week: –Exercises GOAL Programming Guide –Assignment 3: BW4T agent Next week: modules Let us know any problems you have at: goal@mmi.tudelft.nl