1. Distributed Problem Solving & Planning By: Sachin Kamboj CISC 889: Agents, Web Services and Grid Computing Fall 2003

2. Outline Introduction Motivation Distributed Problem Solving Strategies –Task Sharing –Result Sharing –Distributed Planning –Distributed Planning and Execution Comparison with Web Services and Grid Computing Introduction Motivation Distributed Problem Solving Strategies –Task Sharing –Result Sharing –Distributed Planning –Distributed Planning and Execution Comparison with Web Services and Grid Computing

3. Introduction Distributed Problem Solving –Agents work together to solve problems that require collective effort. –Requires: Coherence –Need to want to work together Competence –Need to know how to work together Distributed Planning –Problem Solving in which the agents work together to construct a plan. Distributed Problem Solving –Agents work together to solve problems that require collective effort. –Requires: Coherence –Need to want to work together Competence –Need to know how to work together Distributed Planning –Problem Solving in which the agents work together to construct a plan.

4. Motivation Resources may be distributed –Using distributed resources concurrently can allow a speedup of problem solving due to parallelism. Resources may be distributed –Using distributed resources concurrently can allow a speedup of problem solving due to parallelism.

5. Motivation Problem solving capabilities may be distributed –Separate processes may solve different parts of the problem. Problem solving capabilities may be distributed –Separate processes may solve different parts of the problem. Online Auction on EBay Product Search Credit Card Billing Credit Card Billing Bid Manager Product Delivery Product Delivery

6. Motivation Data may be distributed –Example: In distributed sensor networks Data may be distributed –Example: In distributed sensor networks Agent 3 Agent 1Agent 2 Agent 4

7. Motivation Results may be distributed –The results may have to be acted upon by multiple agents. Results may be distributed –The results may have to be acted upon by multiple agents.

8. Distributed Problem Solving Strategies Task Sharing Result Sharing Distributed Planning Distributed Planning and Execution Task Sharing Result Sharing Distributed Planning Distributed Planning and Execution

9. Task Sharing

10. Task Decomposition –Generate a set of tasks to be passed to others –Decompose large tasks into subtasks that can be tackled by different agents Task Allocation –Assign subtasks to appropriate agents Task Accomplishment –Agents accomplish task, either individually or through further decomposition Result Synthesis –Combine the results to generate the final results Task Decomposition –Generate a set of tasks to be passed to others –Decompose large tasks into subtasks that can be tackled by different agents Task Allocation –Assign subtasks to appropriate agents Task Accomplishment –Agents accomplish task, either individually or through further decomposition Result Synthesis –Combine the results to generate the final results

11. Task Sharing Example: Towers of Hanoi Task Decomposition: –Means-End Analysis: Move the largest disk that is misplaced to its destination peg. Get to the state where the largest disk can be moved. –Move the other disks on top of the largest disk to a peg other than the destination of the largest so that the largest can be moved. Task Allocation: –Allocate sub-problems to identical agents capable of solving it. Task Accomplishment: –If the recursive sub-problems have same start and goal states terminate. Result Synthesis Task Decomposition: –Means-End Analysis: Move the largest disk that is misplaced to its destination peg. Get to the state where the largest disk can be moved. –Move the other disks on top of the largest disk to a peg other than the destination of the largest so that the largest can be moved. Task Allocation: –Allocate sub-problems to identical agents capable of solving it. Task Accomplishment: –If the recursive sub-problems have same start and goal states terminate. Result Synthesis

12. Task Sharing Example: Towers of Hanoi

13. Task Sharing in Heterogeneous Systems Motivation: –Difficult to build agents that are competent in every possible task. –Will result in agents that are “Jacks of all trades and masters of none” –Omni-capable agents would only use a small proportion of their capabilities at any one time. Motivation: –Difficult to build agents that are competent in every possible task. –Will result in agents that are “Jacks of all trades and masters of none” –Omni-capable agents would only use a small proportion of their capabilities at any one time. Car Design Agent Engine Transmission Aerodynamics Brakes … … … …

14. Solution: –Build specialist agents that can work together to solve problems beyond their individual capabilities. –Dynamic combination of heterogeneous agents… Solution: –Build specialist agents that can work together to solve problems beyond their individual capabilities. –Dynamic combination of heterogeneous agents… Task Sharing in Heterogeneous Systems Car Design Agent Engine Design Agent Engine Design Agent Transmission Design Agent Transmission Design Agent Body Design Agent Body Design Agent Brakes Design Agent Brakes Design Agent Fuel Injection Design Agent Fuel Injection Design Agent Cylinder Design Agent Cylinder Design Agent

15. Task Sharing in Heterogeneous Systems Problem: –How can an agent that has to solve a problem find the “specialists”? Naïve Solution: –Have a table that identifies the capabilities of the agents. –Use table to select appropriate agents and give them the sub-problem. –Problems: How is selection done if several candidate agents exist? Problem: –How can an agent that has to solve a problem find the “specialists”? Naïve Solution: –Have a table that identifies the capabilities of the agents. –Use table to select appropriate agents and give them the sub-problem. –Problems: How is selection done if several candidate agents exist?

16. Task Sharing in Heterogeneous Systems Selection of Agents Announce a sub-problem to specific agents. Request return bids describing acceptance/availability Assign sub-problem to the highest bidder Problem: –What if no agents bid for a sub-problem? Solution: Use one of the following: –Broadcast Contracting –Retry –Announcement Revision –Alternative Decomposition Announce a sub-problem to specific agents. Request return bids describing acceptance/availability Assign sub-problem to the highest bidder Problem: –What if no agents bid for a sub-problem? Solution: Use one of the following: –Broadcast Contracting –Retry –Announcement Revision –Alternative Decomposition

17. Result Sharing

18. Results Sharing Motivation: –Results of different problem solvers on the same task may differ… –Sharing results can improve group performance by increasing: Confidence Completeness Precision Timeliness Agents need to know how to assimilate results from different agents –Assimilation may be non-trivial/costly/require large amounts of communication and incur overheads in management. Motivation: –Results of different problem solvers on the same task may differ… –Sharing results can improve group performance by increasing: Confidence Completeness Precision Timeliness Agents need to know how to assimilate results from different agents –Assimilation may be non-trivial/costly/require large amounts of communication and incur overheads in management.

19. Functionally Accurate Cooperation Distributed Vehicle Monitoring Agent 3 Agent 1Agent 2 Agent 4

20. Agents lack information about what is happening elsewhere –This information might impact their calculations. –As a result agents form tentative results that might be unnecessary Functionally Accurate: –Agents get the answer eventually –But with many possible false starts Cooperative: –Iterative exchange between agents is required Agents lack information about what is happening elsewhere –This information might impact their calculations. –As a result agents form tentative results that might be unnecessary Functionally Accurate: –Agents get the answer eventually –But with many possible false starts Cooperative: –Iterative exchange between agents is required Functionally Accurate Cooperation

21. Advantages: –Exchanging tentative partial solutions can impact: Completeness –Partial solutions are synthesized into larger solutions that solve more of the overall solution Precision –Agents may use results from other agents to refine their own solutions Confidence –Confidence measures can be combined to increase overall confidence in the solution. Advantages: –Exchanging tentative partial solutions can impact: Completeness –Partial solutions are synthesized into larger solutions that solve more of the overall solution Precision –Agents may use results from other agents to refine their own solutions Confidence –Confidence measures can be combined to increase overall confidence in the solution.

22. Functionally Accurate Cooperation Problems: –Iterative exchange of partial results should lead to some agent having enough information to keep moving the overall problem solving forward –A large amount of communication overhead is involved in the sharing of partial results. –May result in wasted computations –May lead to Distraction: Agents gravitate towards doing the same problem solving actions. Problems: –Iterative exchange of partial results should lead to some agent having enough information to keep moving the overall problem solving forward –A large amount of communication overhead is involved in the sharing of partial results. –May result in wasted computations –May lead to Distraction: Agents gravitate towards doing the same problem solving actions.

23. Shared Repositories Motivation Sharing partial results involves a lot of communication overhead Agent 1 Agent 2 Agent 3 Agent 4 Agent 5 Agent 6 Agent 7 Agent 9 Agent 8

24. Shared Repositories Instead of agents multicasting partial results to all other agents, the results are stored in a central repository –Also known as a blackboard Instead of agents multicasting partial results to all other agents, the results are stored in a central repository –Also known as a blackboard Agent 1 Agent 2 Agent 3 Agent 4 Agent 5 Agent 6 Agent 7 Agent 9 Agent 8 Shared Repository

25. Shared Repositories Support search through alternative designs –Agents with different design criteria can revise and critique the alternatives Similar to a distributed constraint satisfaction problem Support search through alternative designs –Agents with different design criteria can revise and critique the alternatives Similar to a distributed constraint satisfaction problem

26. Negotiated Search Negotiation: –An agent generates a proposal and other agents review it: If an agent does not like the proposal, it rejects it –Agents might also generate counter-proposals –Proposals are reviewed iteratively Negotiated Search: –A use of shared repositories –Agents concurrently investigate multiple solution paths –Conflicts between solution are resolved using negotiation –Agents have operators for initiating solutions, extending solutions, critiquing solutions and relaxing solution requirements Negotiation: –An agent generates a proposal and other agents review it: If an agent does not like the proposal, it rejects it –Agents might also generate counter-proposals –Proposals are reviewed iteratively Negotiated Search: –A use of shared repositories –Agents concurrently investigate multiple solution paths –Conflicts between solution are resolved using negotiation –Agents have operators for initiating solutions, extending solutions, critiquing solutions and relaxing solution requirements

27. Distributed Constrained Heuristic Search Used to handle distributed resource allocation An agent is associated with each resource DCHS works as follows: –Agent begins with a problem topology –Agent determines resource requirement and computes a demand profile –Agent sends profile to the resource –Resource computes aggregate demand and informs the agents making the demands –Agent uses aggregate demand to identify preferred resource –Agent requests the resource to reserve the specified interval –Resource grants reservation if possible –Agent processes the response from the resource Used to handle distributed resource allocation An agent is associated with each resource DCHS works as follows: –Agent begins with a problem topology –Agent determines resource requirement and computes a demand profile –Agent sends profile to the resource –Resource computes aggregate demand and informs the agents making the demands –Agent uses aggregate demand to identify preferred resource –Agent requests the resource to reserve the specified interval –Resource grants reservation if possible –Agent processes the response from the resource

28. Organizational Structuring An alternative strategy for reducing communication overhead Exploits the task decomposition structure/ organizational structure of agents –Define roles, resources and preferences for the agents –Associates with each agent the type of tasks that it can do Agents need to be informed of partial results that can influence how they carry out their responsibilities –Agents need not be told of results that do not affect their actions Make use of stored pattern-response rules An alternative strategy for reducing communication overhead Exploits the task decomposition structure/ organizational structure of agents –Define roles, resources and preferences for the agents –Associates with each agent the type of tasks that it can do Agents need to be informed of partial results that can influence how they carry out their responsibilities –Agents need not be told of results that do not affect their actions Make use of stored pattern-response rules

29. Communication Strategies Concerned with timing issues Agents need to know not only what results to send but also when to send them Sending a result too soon can: –Clutter up the recipient memory –Distract the recipient away from useful work Failure to send results can: –Lead to delays –Result in a failure of the system to converge on reasonable solutions Concerned with timing issues Agents need to know not only what results to send but also when to send them Sending a result too soon can: –Clutter up the recipient memory –Distract the recipient away from useful work Failure to send results can: –Lead to delays –Result in a failure of the system to converge on reasonable solutions

30. Communication Strategies When possibility of cluttering memory is short: –Send all partial results If exchange of partial results leads a subset of agents into redundant exploration of a part of the solution space: –Refrain from sending partial results Alternatively, only send results when requested –Especially when space of results formed is large and only a few are needed by others. When possibility of cluttering memory is short: –Send all partial results If exchange of partial results leads a subset of agents into redundant exploration of a part of the solution space: –Refrain from sending partial results Alternatively, only send results when requested –Especially when space of results formed is large and only a few are needed by others.

31. Distributed Planning

32. Traditional Planning Set of states –Conjunction of terms that are true about the world Set of operators –Preconditions Things that have to be true about the world to apply the operator –Effects Changes to the current state (Add List, Delete List) Goals Planning is a backward search from the goal state to the start state –At each step, the agent tries to find an operator whose effect achieves a goal or a precondition of another operator that has already been applied Set of states –Conjunction of terms that are true about the world Set of operators –Preconditions Things that have to be true about the world to apply the operator –Effects Changes to the current state (Add List, Delete List) Goals Planning is a backward search from the goal state to the start state –At each step, the agent tries to find an operator whose effect achieves a goal or a precondition of another operator that has already been applied

33. Traditional Planning (Example) Finish Have (Drill), Have (Milk), Have (Bananas), At (Home) Buy (Drill) At (HWS), Sells (HWS, Drill) Buy (Milk) At (SM), Sells (SM, Milk) Buy (Bananas) At (SM), Sells (SM, Bananas) Go (HWS) At (x) Go (SM) At (x) Start At (Home), Sells (SM, Bananas), Sells (SM, Milk), Sells (HW, Drill) Go (HWS) At (Home) Go (SM) At (HWS)

34. Traditional Planning (Simplifying Assumptions) Planning agent is omniscient The action that the agents can perform have definite outcomes The goals presented to the agents are categorical –They are either achieved or not –There is no partial satisfaction of goals The agent is the only source of change in the environment The goals presented remain unchanged throughout the process of planning and execution Actions do not have temporal extent or duration of occurrence Planning agent is omniscient The action that the agents can perform have definite outcomes The goals presented to the agents are categorical –They are either achieved or not –There is no partial satisfaction of goals The agent is the only source of change in the environment The goals presented remain unchanged throughout the process of planning and execution Actions do not have temporal extent or duration of occurrence

35. Traditional Planning (Consequences of Underlying Assumptions) There is no need to interleave planning and execution –Because agent has all the knowledge it needs at planning time No need for re-planning –A plan can always be executed successfully Planning problems do not compete with each other for the planning agent’s attention. –Goals are all known at the outset and remain fixed throughout planning and execution Assumptions fail to hold for realistic situations in which planning is applied… There is no need to interleave planning and execution –Because agent has all the knowledge it needs at planning time No need for re-planning –A plan can always be executed successfully Planning problems do not compete with each other for the planning agent’s attention. –Goals are all known at the outset and remain fixed throughout planning and execution Assumptions fail to hold for realistic situations in which planning is applied…

36. Distributed Planning (Example Scenario) You have a project due on Monday –You decide to work on it over the weekend Exact time is not fixed: will depend on other plans –Tentative Plans: Saturday Morning: Run errands Saturday Night: Watch ‘The Matrix’ Sunday: Finish off project –Saturday Afternoon Receive an invitation from a friend to her birthday party on Sunday evening –Revise Plans: Saturday Morning: Run errands Saturday Evening/Night: Work on Project Sunday Morning: Finish off project Sunday Night: Go to friends birthday party You have a project due on Monday –You decide to work on it over the weekend Exact time is not fixed: will depend on other plans –Tentative Plans: Saturday Morning: Run errands Saturday Night: Watch ‘The Matrix’ Sunday: Finish off project –Saturday Afternoon Receive an invitation from a friend to her birthday party on Sunday evening –Revise Plans: Saturday Morning: Run errands Saturday Evening/Night: Work on Project Sunday Morning: Finish off project Sunday Night: Go to friends birthday party

37. Distributed Planning Aims to overcome some of the limitations of traditional planning –Handle the example scenario just presented Specialization of distributed problem solving, where the problem to be solved is to design a plan Distributed means: –Planning is distributed –Plans are distributed –Both planning and plans are distributed Aims to overcome some of the limitations of traditional planning –Handle the example scenario just presented Specialization of distributed problem solving, where the problem to be solved is to design a plan Distributed means: –Planning is distributed –Plans are distributed –Both planning and plans are distributed

38. Agent 2Agent 1 Go (SM) At (Home) Go (HWS) At (Home) Partial Order Planning (Example) Finish Have (Drill), Have (Milk), Have (Bananas), At (Home) Buy (Drill) At (HWS), Sells (HWS, Drill) Buy (Milk) At (SM), Sells (SM, Milk) Buy (Bananas) At (SM), Sells (SM, Bananas) Start At (Home), Sells (SM, Bananas), Sells (SM, Milk), Sells (HW, Drill)

39. Distributed Planning Centralized Planning for Distributed Plans Formulation of plans that are to be executed in a distributed fashion in a centralized manner Example: –A partial order planner can generate plans that do not have a strict ordering between actions Such actions can be executed in parallel by multiple agents Algorithm: –Generate a partial order plan –Decompose plan into sub-plans –Insert synchronization actions into sub-plans –Allocate sub-plan to agents using task passing mechanisms –Initiate plan execution and monitor progress Algorithm is a specialization of task sharing Formulation of plans that are to be executed in a distributed fashion in a centralized manner Example: –A partial order planner can generate plans that do not have a strict ordering between actions Such actions can be executed in parallel by multiple agents Algorithm: –Generate a partial order plan –Decompose plan into sub-plans –Insert synchronization actions into sub-plans –Allocate sub-plan to agents using task passing mechanisms –Initiate plan execution and monitor progress Algorithm is a specialization of task sharing

40. Distributed Planning Distributed Planning for Centralized Plans Formulating a complex plan may require collaboration among a variety of cooperative planning specialists Similar to task-sharing and result-sharing problem solving –Overall problem formulation task is decomposed and distributed among various planning specialists Each planning specialist generates its portion of the plan Interaction between specialists might be through the exchange of partially specified plans Example: –Machining of tools in the manufacturing domain Formulating a complex plan may require collaboration among a variety of cooperative planning specialists Similar to task-sharing and result-sharing problem solving –Overall problem formulation task is decomposed and distributed among various planning specialists Each planning specialist generates its portion of the plan Interaction between specialists might be through the exchange of partially specified plans Example: –Machining of tools in the manufacturing domain

41. Distributed Planning Distributed Planning for Distributed Plans Both the planning process and its results are distributed Multi-agent plans are not represented in entirety anywhere in the system –No one agent knows the complete plan –Distributed pieces of a plan should be compatible Techniques that can be used: –Plan merging –Iterative plan formation –Negotiation in distributed planning Both the planning process and its results are distributed Multi-agent plans are not represented in entirety anywhere in the system –No one agent knows the complete plan –Distributed pieces of a plan should be compatible Techniques that can be used: –Plan merging –Iterative plan formation –Negotiation in distributed planning

42. Distributed Planning for Distributed Plans Plan Merging Each agent formulates its own plan individually Agents work together to ensure that their separate plans can be executed without conflict –Local plans are exchanged –Messaging and/or timing commitments are imposed to resolve negative plan interactions Centralized plan coordination approach –An agent collects together the individual plans –Agent analyzes the plans to discover the sequence of actions that leads to a conflict Reachability Analysis –Enumerate all possible states of the world that can be reached from the initial state by executing actions Plans are modified to remove conflicts Each agent formulates its own plan individually Agents work together to ensure that their separate plans can be executed without conflict –Local plans are exchanged –Messaging and/or timing commitments are imposed to resolve negative plan interactions Centralized plan coordination approach –An agent collects together the individual plans –Agent analyzes the plans to discover the sequence of actions that leads to a conflict Reachability Analysis –Enumerate all possible states of the world that can be reached from the initial state by executing actions Plans are modified to remove conflicts

43. Distributed Planning for Distributed Plans Iterative Plan Formation Not all problems allow agents to develop their own local plans Require agents to search through larger spaces of plans rather than each proposing a single specific plan –Each agent might construct a set of all feasible plans –The planning process consists of fitting together subsets of agents plans Example methods: –Plan Combination Search –Distributed Hierarchical Planning Not all problems allow agents to develop their own local plans Require agents to search through larger spaces of plans rather than each proposing a single specific plan –Each agent might construct a set of all feasible plans –The planning process consists of fitting together subsets of agents plans Example methods: –Plan Combination Search –Distributed Hierarchical Planning

44. Distributed Planning and Execution Plan merging: –Each agent individually builds a plan. –Individual plans are coordinated –Results of coordination are executed. If one or more of the agents should fail in carrying out their plans, the whole coordinated plan set might fail Post-planning coordination –Done after one or more plans fail –Methods: Contingency planning –Each agent formulates alternative plans to respond to possible contingencies Monitoring and replanning –If there is a deviation from a plan stop all execution and repeat the plan-coordinate-execute cycle Plan merging: –Each agent individually builds a plan. –Individual plans are coordinated –Results of coordination are executed. If one or more of the agents should fail in carrying out their plans, the whole coordinated plan set might fail Post-planning coordination –Done after one or more plans fail –Methods: Contingency planning –Each agent formulates alternative plans to respond to possible contingencies Monitoring and replanning –If there is a deviation from a plan stop all execution and repeat the plan-coordinate-execute cycle

45. Distributed Planning and Execution Pre-planning coordination Coordination is done in such a way that no matter what happens during execution they agents will remain coordinated with each other Introduces coordination restriction –Such restrictions usually take the form of social laws A prohibition against particular choices of actions in particular contexts For example, entering an intersection on a red light –Restrictions are derived by working back from undesirable states: Combination of actions that lead to those states are found Restrictions are imposed so that those actions cannot occur Coordination is done in such a way that no matter what happens during execution they agents will remain coordinated with each other Introduces coordination restriction –Such restrictions usually take the form of social laws A prohibition against particular choices of actions in particular contexts For example, entering an intersection on a red light –Restrictions are derived by working back from undesirable states: Combination of actions that lead to those states are found Restrictions are imposed so that those actions cannot occur

46. Distributed Planning and Execution Interleaving Planning, Coordination & Execution In certain situations, it might be impractical to distinguish between planning and execution –The environment may be highly dynamic Assumptions made by the agent while planning may no longer be true –A single agent cannot decide on the future course of action without exchanging info with other agents Example situations which required interleaving: –Distributed sensor networks –Dynamic resource allocation May make use of partial global planning In certain situations, it might be impractical to distinguish between planning and execution –The environment may be highly dynamic Assumptions made by the agent while planning may no longer be true –A single agent cannot decide on the future course of action without exchanging info with other agents Example situations which required interleaving: –Distributed sensor networks –Dynamic resource allocation May make use of partial global planning

47. Comparison with Web Services and Grid Computing Method of distributed problem solving depends on the application –Includes a large variety of techniques –There is no one “standard” or silver bullet. Same methods may be used in web services and grid computing –Distributed constrained heuristic search may be used in grid computing for the allocation of resources –Web services choreography is essentially a form of planning with preconditions, post-conditions, triggering events and flows Method of distributed problem solving depends on the application –Includes a large variety of techniques –There is no one “standard” or silver bullet. Same methods may be used in web services and grid computing –Distributed constrained heuristic search may be used in grid computing for the allocation of resources –Web services choreography is essentially a form of planning with preconditions, post-conditions, triggering events and flows

48. References Edmund H. Durfee: “Distributed Problem Solving and Planning”, in Chapter 3, Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, Gerhard Weiss (Editor)

49. Thank You!!