Koen HindriksMulti-Agent Systems Environments and Sensing Multi-Agent Systems Course Koen Hindriks Delft University of Technology, The Netherlands

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 2 Outline for Today Previous lecture: core G OAL language: Knowledge, beliefs, goals, actions, rules Today: Agent Program Structure Macros Environments, sensing, and acting BW4T & Agent design guidelines

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems AGENT PROGRAM STRUCTURE

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 4 An Agent is a Set of Modules Built-in modules: init module: –Define global knowledge –Define initial beliefs & goals –Process “send once” percepts –Specify environment actions main module –Action selection strategy event module –Process percepts –Process messages –Goal management User-defined modules. init module{ knowledge{ … } beliefs{ %% INITIAL BELIEFS ONLY IN INIT MODULE %% } goals{ … } program{ %% PROCESS “SEND ONCE” PERCEPTS HERE %% } actionspec{ %% SPECIFY ENVIRONMENT ACTIONS HERE %% } main module{ % OPTIONAL knowledge section % NO beliefs section HERE! % OPTIONAL goal section (not advised in ‘main’) program{ %% ENVIRONMENT ACTION SELECTION HERE %% } event module{ program{ %% PROCESS PERCEPTS HERE %% %% PROCESS MESSAGES HERE %% %% PERFORM GOAL MANAGEMENT HERE %% }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 5 G OAL Agent Cycle Process events (= apply event rules) Select action (= apply action rules) Perform action (= send to environment) Update mental state (= apply action specs + commitment strategy) Also called reasoning or deliberation cycle. G OAL ’s cycle is a classic sense-plan-act cycle. event module main module

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems MACROS

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 7 Macro Definitions instead of writing mental state conditions: you may use macros in rules: mental state condition is macro’s definition program{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table). } program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table). }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 8 Macro Definitions Use to increase readability Macro definitions must be placed inside the program section, before any of the rules All variables in macro should occur in defining mental state condition. Other vars in condition are just placeholders. program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table). }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 9 Macro Definitions 1.When is a block marked as obstructing? Informally explain the macro definition. 2.In which cases is the concept of an obstructing block useful? knowledge{ … above(X,Y) :- on(X,Y). above(X,Y) :- on(X,Z), above(Z,Y). tower([X]) :- on(X,table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } program{ #define inPosition(X) goal-a( tower([X|T]) ). #define obstructingBlock(X) a-goal( on(Y,Z) ), bel( above(X,Z); above(X,Y) ). … } EXERCISE

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 10 SAY SOMETHING ABOUT ORDER MODIFIER, random, randomall, linear, linearall

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 11 Outline Environments and Perception Built-in Actions and Selecting Multiple Actions Environments and Durative Actions

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 12 This Lecture Percepts Action events actionsgoals plans beliefs environment agent

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems DEMO: TOWERWORLD

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 14 TowerWorld Environment The TowerWorld is: –Multi-agent: user can also change the world –Fully observable: the agent can see “everything” –Dynamic: environment can change if agent does nothing  We need sensing / percepts.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems SENSING AND PERCEPTION

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 16 Sensing Agents need sensors to: –explore the environment when they have incomplete information (e.g. Wumpus World) –keep track of changes in the environment that are not caused by itself G OAL agents sense the environment through a perceptual interface defined between the agent and the environment. –Environment generates perceptions

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 17 Sensing in the Tower World Percepts are received by an agent in it’s percept base. The reserved keyword percept is wrapped around the percept content, e.g. percept(block(a)). Not automatically inserted into beliefs!

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 18 Inspecting the Percept Base Percept base is inspected using the bel operator, e.g. bel( percept(on(X,Y)) ). Often useful to check for difference with current beliefs: 1.Agent does not believe something it perceives: bel( percept(on(X,Y)), not(on(X,Y)) ) 2.Agent believes something it does not perceive: bel( on(X,Y), not(percept(on(X,Y))) ) Why not: bel( not(percept(on(X,Y))), on(X,Y) ) ? Evaluate with only: percept(on(a,b)) and on(a,c) EXERCISE

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 19 Processing Percepts The percept base is refreshed, i.e. emptied and then filled again, every cycle of the agent. Agent has to decide what to do when it perceives something, i.e. receives a percept. Use percepts to update agent’s mental state: –Ignore the percept –Update the beliefs of the agent –Adopt a new goal

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 20 Built-in Actions G OAL provides a number of built-in actions: insert( ) example: insert(on(X,Y), not(on(X,Z))) meaning: add on(X,Y) and delete on(X,Z) delete( ) example: delete(on(X,Y), not(on(X,Z))) meaning: delete on(X,Y) and add on(X,Z) adopt( ) example: adopt(on(a,b), on(b,c)) meaning: add new goal (if not already implied by a goal nor believed) drop( ) example: drop(on(b,a), not(on(c,table))) meaning: remove all goals that imply from the goal base Note All actions need to be closed when they are executed.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 21 Built-in Action Insert Example 1: Example 2: Belief base affected, sometimes also goal base affected. Example 2 preferred: Use insert only for inserting. beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,b). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } insert(on(a,b), not(on(a,table))) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). on(d,table). } goals{ on(a,b), on(b,table). } insert(on(d,table))

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 22 Built-in Action Delete Example 1: Example 2: Only belief base affected, if used as in Example 2 (preferred). In that case, goal base always remains unchanged beliefs{ on(a,b). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } delete(on(a,b), not(on(a,table))) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } delete(on(a,table))

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 23 Multiple Actions in Rule Using the + operator multiple actions can be combined and performed in a single rule. The actions are executed in same reasoning cycle Combined actions are executed in order (left-to-right) Example: rule to update moving state of BW4T robot. Why not: insert ( state(NewState) ) + delete ( state(OldState) )? If OldState = NewState information about state would be deleted! program{ if bel(state(OldState), percept(state(NewState)) ) then delete( state(OldState) ) + insert( state(NewState) ). ….. } if then + + …. EXERCISE

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 24 Built-in Action Adopt Example 1: Example 2(a) and (b): Only goal base affected, belief base remains unchanged. beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). on(c,a). } adopt(on(c,a)) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } does not execute adopt(on(a,table)) adopt(on(a,b)) does not execute

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 25 Built-in Action Drop Example 1: Example 2: Only goal base affected, belief base remains unchanged. Like insert and delete actions always executes. beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ } drop(on(a,b), on(b,table))) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ } drop(on(a,b))

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 26 Drop Action drop(on(b,a), not(on(c,table))) Is goal in goal base dropped? Check: does goal imply on(b,a), not(on(c,table)) ? A: Yes, so goal is removed by drop action. knowledge{ block(X) :- on(X, Y). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]):- on(X,Y),tower([Y|T]). } goals{ on(a,table), on(b,a), on(c,b), on(d,table), on(e,table), on(f,e), on(g,f), on(h,g), on(i,h). } EXERCISE

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 27 Changing Agent’s Mind: Goals Changing One’s Mind Pattern: If a “better” goal G2 can be set, then replace a current goal G1 with that new goal G2. Example in Tower World Agent: program{ … if goal( holding(X) ), bel( not(clear(X) ), constructiveMove(Z,Y) then drop( holding(X) ) + adopt( holding(Z) ). … } program{ … if goal( G1 ), then drop( G1 ) + adopt( G2 ). … }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 28 Updating Agent’s Mental State One way to update beliefs with percepts: First, delete everything agent believes. Example: remove all block and on facts. Second, insert new information about current state provided from percepts into belief base. Example: insert block and on facts for every percept(block(…)) and percept(on(…)). Assumes that environment is fully observable with respect to block and on facts. Downside: not very efficient…

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 29 Percept Update Pattern A typical pattern for updating is: Rule 1 If the agent –perceives block X is on top of block Y, and –does not believe that X is on top of Y Then insert on(X,Y) into the belief base. Rule 2 If the agent –believes that X is on top of Y, and –does not perceive block X is on top of block Y Then remove on(X,Y) from the belief base. Assumes full observability with respect to on facts.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 30 Percepts and Event Module Percepts are processed in G OAL by means of event rules, i.e. rules in the event module. Event module is executed every time that agent receives new percepts. event module{ program{ <… rules …> }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 31 Implementing Pattern Rule 1 Rule 1 If the agent –perceives block X is on top of block Y, and –does not believe that X is on top of Y Then insert on(X,Y) into the belief base. event module { program{ % assumes full observability. if bel( percept(on(X,Y)), not(on(X,Y)) ) then insert(on(X,Y)). … } INCORRECT!

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 32 Implementing Pattern Rule 1 Rule 1 If the agent perceives block X is on top of block Y, and does not believe that X is on top of Y, then insert on(X,Y) into the belief base. We want to apply this rule for all percept instances that match it! event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). … } Content Percept Base percept(on(a,table)) percept(on(b,table)) percept(on(c,table)) percept(on(d,table)) … insert(on(a,table)) insert(on(b,table)) insert(on(c,table)) insert(on(d,table)) … Assuming empty belief base

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 33 Implementing Pattern Rule 2 Rule 2 If the agent –believes that X is on top of Y, and –does not perceive block X is on top of block Y Then remove on(X,Y) from the belief base. 1.We want that all rules are applied! By default the event module applies ALL rules in linear order. 2.Note that none of these rules fire if nothing changed. event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). forall bel(on(X,Y), not(percept(on(X,Y)))) do delete(on(X,Y)). }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 34 Event Module for Tower World Rules for every type of percept, e.g., for Tower World: blocks present; whether gripper is holding a block or not; a block being on top of another or on the table. event module { program{ % assumes full observability. forall bel( block(X), not(percept(block(X))) ) do delete( block(X) ). forall bel( percept(block(X)), not(block(X)) ) do insert( block(X) ). forall bel( holding(X), not(percept(holding(X))) ) do delete( holding(X) ). forall bel( percept(holding(X)), not(holding(X)) ) do insert( holding(X) ). forall bel( on(X,Y), not(percept(on(X,Y))) ) do delete( on(X,Y) ). forall bel( percept(on(X,Y)), not(on(X,Y)) ) do insert( on(X,Y) ). }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 35 Initially… Agent Knows Nothing In most environments an agent initially has no information about the state of the environment, e.g. Tower World, Wumpus World, … Represented by an empty belief base: There is no need to include a belief base in this case in a G OAL agent. It is ok to simply have no belief base section. beliefs{ }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 36 Processing Percepts Types of percepts: “send always” “send once”, e.g., place( ) “send on change”, e.g., at( ) “send on change with negation”, e.g., in( ) How to handle these different type of percepts? See previous slides

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 37 Pattern for Send Once “Send once” percepts are sent only once when the agent first connects to an environment. Use the init module to insert the beliefs you want to use in the agent’s belief base. init module { … program{ forall bel( percept(place(X) ) do insert( place(X) ). … }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 38 Send on Change Percept “send on change” percepts are sent once when a change occurs: Note that pattern for “send always” percepts does not work for “send on change” due to second rule in that pattern. event module { % send always pattern program{ … forall bel(on(X,Y), not(percept(on(X,Y)))) do delete(on(X,Y)). } State 1 State 2State 3State 4 State 5 state(traveling)state(arrived) -

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 39 Pattern for Send on Change “Send on change” percepts are sent only when something in the environment has changed. Rule: remove old belief and insert the new percept. Put rule in event module. event module { … program{ forall bel( state(Old), percept(state(New) ) do delete(state(Old)) + insert(state(New)). … } Could also have used ‘ if ’ here: At most one state percept received.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 40 Send on Change with Negation “Send on change with negation” percepts: Positive fact sent once when it becomes true in environment Negative fact sent once when it becomes false in environment Rule: insert positive and remove negative facts. Put rules in event module. event module { … program{ forall bel(percept(in(RoomID)) do insert(in(RoomID)). forall bel(percept(not(in(RoomID))) do delete(in(RoomID)).... }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 41 Combining Information In BW4T it is most important to remember in which place a colored block can be found. Idea: combine at location with ‘color’ percept using a new ‘block’ predicate. NB: make sure by earlier rule that at belief is correct! event module { … program{ … forall bel( percept(color(BlockID, ColorID)), at(PlaceID) ) do insert( block(BlockID, ColorID, PlaceID) ). forall bel( at(PlaceID), block(BlockID, ColorID, PlaceID), percept(not(color(BlockID, ColorID))) ) do delete( block(BlockID, ColorID, PlaceID) ). … }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 42 Summarizing Two types of rules: –if then. is applied at most once (if multiple instances chooses randomly) –forall do. is applied once for each instantiation of parameters that satisfy condition. Main module by default: –checks rules in linear order –applies first applicable rule (also checks action precondition!) Event module by default: –Checks rules in linear order –Applies all applicable rules (rules may enable/disable each other!) Program section modifiers: [order=random], [order=linear], [order=linearall], [order=randomall] Built-in actions: insert, delete, adopt, drop.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems ACTIONS IN ENVIRONMENTS

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 44 Instantaneous versus Durative Instantaneous actions Actions in the Blocks World environment are instantaneous, i.e. they do not take time. Wumpus World actions are of this type as well. Durative actions Actions in the Tower World environment take time. When a G OAL agent sends an action to such an environment, the action will not be completed immediately.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 45 Durative Actions Deciding on a next action typically gets more complicated when actions take time! Agents may decide to perform an action before finishing another. Three cases: 1.Sending an action B while another action A is still ongoing overrides and terminates previous action A… Example: actions in Tower World, goto action in UT Action B that is sent while another action A is still ongoing is performed in parallel… both actions are performed. Example: shooting while running in UT Action B that is sent while another action A is still ongoing is simply ignored… nothing new happens. Less frequent.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 46 Durative Actions and Sensing While durative actions are performed an agent may receive percepts. Useful to monitor progress of action. Tower World Example: Gripper may no longer be holding block because user removed it. BW4T Example: Other robot is blocking entrance to room.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 47 Specifying Durative Actions Tower World has two actions: pickup and putdown block both take time. How should we specify these actions?  delayed effect problem. Solution: –do not specify postcondition, –make sure action effects are handled by event rules.

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 48 Specifying Durative Actions Tower World action specification: Postcondition may also be “empty”: post { } Better practice is to indicate that you have not forgotten to specify it by using post { true }. actionspec{ pickup(X) { pre{ clear(X), not(holding(Y)) } post{ true } } putdown(X,Y) { pre { holding(X), clear(Y) } post { true } } nil { pre { true } post { true } }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 49 Multiple Environment Actions Warning: It may not be useful to combine environment actions. Environment actions may cancel each other out What happens with the following rule? Nothing… if gripper is already at “home position” program{ … if goal( holding(X) ) then pickup(X) + nil. … }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 50 Multiple Environment Actions Warning: It may not be useful to combine environment actions. Environment actions may cancel each other out What happens with the following rule? If robot is at “PlaceID” then maybe no pickup! program{ … if bel( block(BlockID, ColorId, PlaceID) ), goal( holding(ColorId) ) then goTo(PlaceID) + pickUp(BlockID) + goTo(‘DropZone’). … }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems WRITING GOAL AGENTS

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 52 General Guidelines –Design representation of environment –Code belief base –Identify goals –Code goal base –Identify percepts & events –Write event rules –Identify actions –Write action specifications –Determine action selection strategy –Write action rules a b c d e

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems BW4T ASSIGNMENT

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 54 Beliefs: Goals: Setting Goals Adopt goals that agent will eventually believe. For example, use atBlock(Block) en in(Room). No need to use drop action in this case! in(‘RoomA1’) goal adopted - belief inserted goal removed

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 55 ?? PARSING ERRORS ?? Inspect PARSE INFO tab: Prolog Parser error at line 13, position 23 in robot.goal: Missing token; expected ENDTOKEN but got '{'. Pure Prolog code in knowledge section! Also check Console Tab for other errors. Stack overflow -> non-terminating Prolog rule! knowledge{ … % Assignment 3.3: insert a predicate "nextColorInSeq(Color)“ nextColorInSeq(Color){ % ???? sequenceIndex(Q):- sequence(Q). % ???? } % ???? }

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 56 !! DOCUMENTATION !! Document your code using COMMENTS! Solution can be provided in about 100 lines of code. But then add at least half that amount to document and explain your code

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 57 TESTING & SUBMITTING Test your multi-agent system on multiple maps! See doc for how to change maps. Your mas should work also on Banana map…! Hand in both.mas file and.goal file(s)!

Koen Hindriks Multi-Agent Systems Koen HindriksMulti-Agent Systems 58 Organisation Tutorial this week: –Assignment 3: BW4T agent Next week: modules & goal management Let us know any problems you have at: