1. May 14, 20081 Organization Design and Dynamic Resources Huzaifa Zafar Computer Science Department University of Massachusetts, Amherst

2. May 14, 20082 Organization Design The organization of a multi-agent system is the collection of roles, relationships, and authority structures which govern its behavior - [Horling & Lesser 05] Organization Design v/s Operational Design Long Term v/s Short term Used to guide Data Flow Resource Allocation Coordination Pattern … etc

3. May 14, 20083 Dynamic Resources Dynamic Resources are those resources where some characteristics of the resource changes over time Example - Network Routing Cost of communication changes as network loads change Paths in multi-hop communication changes as links fail Environmental interference changes over time Example 2 - Battery Power Consumption More usage of power implies faster battery consumption Less available power implies an agent can take up less responsibility.

4. May 14, 20084 Outline How can we make better use of resource allocation given knowledge of the Organization design? Network Routing eCQRouting Experimental Analysis How can we redesign/adapt our organization to the changing resource? Problem setup Challenges we face in solving this problem Example applications

5. May 14, 20085 Motivation Application Layer Network Layer Agent AAgent B Application Message Organization Knowledge, Message priority Effect of message loss on performance

6. May 14, 20086 Introduction Objectives: Significant number of network exploration messages required to support multi-hop communication In turn reduces available bandwidth for application messages Reduce this number in order to increase application level bandwidth Further regulate the number of exploration messages based on: Priority of messages Relationship between rate of message loss and performance Use application level organizational estimates of direction and priority of communication in network level routing protocols

7. May 14, 20087 Routing At each time step do: Each destination-agent sends out an exploration message All other agents in the network receive this exploration message and use the corresponding time delay to predict cost of sending messages to the corresponding destination Agents develop policies for sending messages based on costs Policy dictates next hop when multi-hop routing Cost of sending exploration messages? eCQRouting: At each time step do: Should I as the destination-agent send a message? How much confidence do I as a source-agent have on the policies?

8. May 14, 20088 eCQRouting: Organizational Input Direction and priority of communication Effect of message loss on performance Minimum path-confidence Exploration-decision frequency Learning rate ( α ) - For Q-Learing

9. May 14, 20089 eCQRouting Step 1 Each agent has access to a weighted graph representing direction and priority of communication between agent roles in the network No network-level topological information Use the graph to determine if an agent is a destination-agent {Cluster-Head and Regional- Agents}. All agents are source-agents

10. May 14, 200810 Example Network Sensor Agent Regional Node Cluster Head Data Messages Exploration Messages Exploration messages are sent along with Data messages, causing interference and reduction in bandwidth

11. May 14, 200811 eCQRouting Step 2.1: source-agent Uses time delay in receiving exploration messages along with Q-Learning to determine local policies The policy of an agent determines the next best hop to a given destination Confidence represents how well the Q-Value reflects the current state of the network Confidence degrades with time in the absence of exploration messages Calculated at source: The lower the confidence of an agent, the less its Q-Values (and in turn policies) change with updates Time delay in receiving exploration messagesCurrent Confidence in Q-Value Learning rate

12. May 14, 200812 eCQRouting Step 2.2: destination-agent Exploration Objective: Determine the cost of sending a message from a source Every cycle: Regulate this threshold depending on the organization (later in this talk) Confidence has dropped below a threshold A minimum path-confidence threshold is provided as input Source agent communicates its confidence Source-agents use exploration messages to estimate time required to sending application messages to the destination

13. May 14, 200813 Example Network Benefits : Lower number of exploration messages Exploration messages are of a smaller size Q-Table is smaller

14. May 14, 200814 eCQRouting Step 2.3: Exploration based on message priority More frequent exploration by high priority destinations (messages to the corresponding destination have high priority) Destination agent changes threshold depending on message priority Q-Values of application messages to high priority destinations are more accurate, with low priority messages less accurate.

15. May 14, 200815 eCQRouting Step 2.4: Exploration based on message loss Source agents: Determine the rate of message loss to the destination Send message loss rate to the destination Destination agents: Explore more frequently when current paths have significant application-level performance degradation Agents tolerate high message-loss rates if the corresponding performance degradation is low

16. May 14, 200816 eCQRouting Step 2.4 : Exploration based on message loss

17. May 14, 200817 CNAS Collaborative Network for Atmospheric Sensing Power-Aware, Agent-Based nodes Hierarchical Organization Sensor Agents collect data Cluster Heads aggregate data and guide sensor agents Cluster Heads send aggregated data to regional agents

18. May 14, 200818 CASA - Collaborative Adaptive Sensing of the Atmosphere Considerably higher bandwidth requirement than CNAS 4 Roles; Radars, Feature Detectors, Feature Repositories and Optimizers Roles higher in the hierarchy communicate with higher priority

19. May 14, 200819 Experiment - Bandwidth Increase Networks range from 4 agents to 100 agents Agents are randomly placed such that density remains constant as network size increases 1 Cluster Head for every 3 Sensor Agents; placed randomly in the network 35% additional application bandwidth in the network of size 100 when compared to OLSR

20. May 14, 200820 Experiment - Robust Performance Network of 160 agents 4 Optimizer agents; 4 Feature- Repository/Feature-Detector agents; Rest Radar agents More robust performance degradation with message loss Insignificant difference between the two threshold modification algorithms

21. May 14, 200821 Conclusions Reduce network exploration messages Selected agents explore depending on organization knowledge Each agent explores only if the confidence in Q-Value of the path is below a threshold Regulate path-confidence threshold Priority of messages - high priority destinations explore more often Effect of message loss on performance - Significant effect implies more exploration to find alternative paths

22. May 14, 200822 Future Work - Problem Setup Resource - Network Routing Given - A basic organization Question 1: How is this organization represented? Wireless Networks Cost of sending messages fluctuate regularly Adhoc Networks Agents enter and leave the network dynamically Agent Failures Agents are unable to communicate with their neighbors Emergent Organization?

23. May 14, 200823 Challenges Effect of change in organization on the network Message interferences Changes in costs with changes in traffic Effects of mobility of nodes Goodness of Organization How do we determine if one organization is better than another organization? Cost of evaluating the organization Effect of time spent evaluating on the MAS Reorganization/Adaptation costs Time spent in developing the new organization Cost of updating all agents with the new organization

24. May 14, 200824 Experimental Analysis Reorganizing CNAS Re-ordering the leader agent priority lists Regional nodes RoboRescue Fire Hazards Organizing agents based on locations of fire hazards Predicting (or detecting) environmental changes Communication Costs Reorganizing to reduce communication costs/limitations

25. May 14, 200825 Questions and Discussion