1. MULTI-AGENT BASED SCHEDULING D. Ouelhadj ASAP (Automated Scheduling Optimisation and Planning) Research Group School of Computer Science and IT University of Nottingham, UK Open Issues in Grid Scheduling, 2003

2. Contents 1. Introduction 1. Introduction 2. Multi-agent systems 2. Multi-agent systems 3. Multi-agent based scheduling 3. Multi-agent based scheduling 4. Multi-agent systems for integrated and 4. Multi-agent systems for integrated and dynamic scheduling of steel production dynamic scheduling of steel production 5. Conclusion 5. Conclusion Open Issues in Grid Scheduling, 2003

3. Introduction Open Issues in Grid Scheduling, 2003 Characteristics of most scheduling systems developed in manufacturing environments: Centralised or hierarchical. Centralised or hierarchical. Tractable. Tractable. Stochastic. Stochastic. Resource level Supervisor level Intermediate levels Supervisor level Resource level Centralised and hierarchical scheduling

4. Introduction Open Issues in Grid Scheduling, 2003 Classical scheduling techniques: Operational research-based techniques: branch and bound, etc. Artificial intelligence-based techniques: heuristics, meta- heuristics, hyper-heuristics, knowledge-based systems, case-based reasoning, fuzzy logic, etc. Distributed Scheduling systems using MULTI-AGENTS

5. Motivations Open Issues in Grid Scheduling, 2003 Real-life scheduling problems are usually physically or functionally distributed (air traffic control, manufacturing systems, health care, etc.). Complex systems are beyond direct control. They operate through the cooperation of many interacting subsystems, which may have their independent interest, and modes of operation. Complexity of real-life scheduling problems dictates a local point of view. When the problems are too extensive to be analysed as a whole, solutions based on local approaches are more efficient. Centralised structures are difficult to maintain and reconfigure, inflexible, inefficient to satisfy real-world needs, costly in the presence of failures, and the amount of knowledge to manage is very large.

6. Motivations Open Issues in Grid Scheduling, 2003 Need for integration of multiple legacy systems and expertise. Heterogeneity. Heterogeneous environments may use different data and models, and operate in different modes. Robustness and reliability against failures. Scalability and flexibility. Computational efficiency. Agents can operate asynchronously and in parallel, which can result in increased overall speed. Clarity of design and reusability. Costs. It may be much more cost-effective than a centralised system, since it could be composed of simple subsystems of low unit cost.

7. What is a multi-agent system Open Issues in Grid Scheduling, 2003 communication action perception Agent environment autonomy goal-driven r reactivity and proactivity social ability persistent mobility adaptability autonomy goal-driven r reactivity and proactivity social ability persistent mobility adaptability An agent is an intelligent entity that is situated in some environment, and that is capable of flexible and autonomous action in this environment in order to meet its design objectives. By flexible we mean that the system must be responsive, proactive, and social Wooldrige and Jennings (1995). A Multi-Agent System is a system composed of a population of autonomous agents, which interact with each other to reach common objectives, while simultaneously each agent pursues individual objectives Ferber (1997).

8. Cooperation in multi-agent systems Open Issues in Grid Scheduling, 2003 Contract Net Protocol The contract net protocol is a high level protocol for achieving efficient cooperation introduced by Smith (1980) based on a market-like protocol. Task announcement Contract Bid agent Task announcement

9. Multi-agent-based scheduling Open Issues in Grid Scheduling, 2003 Resource agent: Local Scheduling Local autonomy. An agent has the responsibility for carrying out local scheduling for one or more (functional or physical) components, such as machines and jobs. Agents have the ability to observe their environment and to communicate and cooperate with other agents in order to ensure that local scheduling leads to a globally desirable schedule. Autonomy allows the agents to respond to local variations, increasing the flexibility of the system. Concurrency. Negotiation-based decision making instead of totally pre-planned scheduling. Robustness: fast detection of and recovery from the failures. Open and dynamic scheduling structures. Announcement of production requirements Local scheduling I am free in that period broken down Resource agents

10. Multi-agent-based scheduling architectures Open Issues in Grid Scheduling, 2003 Autonomous. Mediator.

11. Multi-agent-based scheduling architectures Open Issues in Grid Scheduling, 2003 Autonomous architectures Manufacturing entities Physical or functional agents (resources, parts, tasks, etc) Agents representing manufacturing entities (resources, tasks, etc.) have the ability to define their local schedules, react locally to local changes, and cooperate directly with each other to generate the global optimal and robust schedules.

12. Multi-agent-based scheduling architectures Open Issues in Grid Scheduling, 2003 Mediator architectures Mediator agent Coordinator for global scheduling Mediator agent Physical or functional agents (resources, parts, tasks, etc) Manufacturing entities A mediator architecture has a basic structure of autonomous cooperating local agents that are capable of negotiation with each other in order to achieve production targets.That basic structure is extended with mediator agents to coordinate the behaviour of the local agents to generate the global optimal and robust schedules.

13. A multi-agent system for integrated and dynamic scheduling of steel production Open Issues in Grid Scheduling, 2003

14. Integration : how to integrate the scheduling systems of Integration : how to integrate the scheduling systems of the continuous caster and the hot strip mill ? the continuous caster and the hot strip mill ? Dynamic scheduling: Robustness against failures ? Dynamic scheduling: Robustness against failures ? Use Of MULTI-AGENT SYSTEMS Steel production scheduling Open Issues in Grid Scheduling, 2003

15. Multi-agent architecture proposed ` User agent HSM Agent SY Agent CC-1 Agent CC-3 Agent CC-2 Agent user Continuous Casters Slabs Hot Strip Mill Slabyard coils Ladle Open Issues in Grid Scheduling, 2003

16. Dynamic scheduling of the HSM and CC agents Presence of real-time events On the CC agent: steel with wrong chemical compositions. On the HSM agent: non availability of slabs. Robust predictive-reactive scheduling first constructs a predictive schedule and then modifies the schedule in response to real-time events so as to minimise deviation between the performance measure values of the realised and predictive schedule. Open Issues in Grid Scheduling, 2003

17. Dynamic scheduling of the HSM and CC agents Predictive schedules are generated using tabu search. Robust predictive-reactive schedules are generated using: Utility, stability, and robustness measures. Rescheduling strategies: complete rescheduling and schedule repair. Open Issues in Grid Scheduling, 2003

18. Dynamic scheduling of the HSM and CC agents Utility, stability and robustness measure the effect of real-time events, and are used to select the best rescheduling strategy (schedule repair or complete rescheduling) to react to real- time events. Utility measures the change in the value of the schedule objective function following the schedule revision. Stability measures the deviation from the original predictive schedule caused by schedule revision. Robustness combines the maximisation of utility and the minimisation of stability. Open Issues in Grid Scheduling, 2003

19. Rescheduling strategies Schedule repair and complete rescheduling strategies On the CC agent Insert- at- end Schedule repair (IESR) Insert-Heat Schedule Repair (IHSR) Shift Schedule Repair (SHSR) Swap Schedule Repair (SWSR) Hybrid Schedule Repair (HBSR) Complete Rescheduling (CR) On the HSM agent Do-nothing (DON) Simple Replacement (SR) Closed Schedule Repair (CSR) Open Schedule Repair (OSR) Hybrid Closed Schedule Repair (HCSR) Hybrid Open Schedule Repair (HOSR) Partial Reschedule (PR) Complete Rescheduling (CR) Open Issues in Grid Scheduling, 2003

20. Negotiation protocol for inter-agent cooperation The negotiation protocol is a two-level bidding mechanism based on the contract net protocol involving negotiation at HSMA-SYA and SYA-CCA(s) levels. At the HSMA-SYA negotiation level, the HSMA requests the supply of slabs from the SYA. At the SYA-CCA (s) negotiation level, the SYA requests the production of slabs not available in the slabyard from the CCA(s). A commitment duration is attached to the the negotiation messages to specify the time windows by which the agents must respond to a given negotiation message. Open Issues in Grid Scheduling, 2003

21. Negotiation protocol for inter-agent cooperation The negotiation protocol incorporates a decommitment mechanism to allow the agents to decommit by specifying appropriate contracts alternatives in response to future real-time events. Open Issues in Grid Scheduling, 2003

22. Negotiation protocol for inter-agent cooperation Steps of the negotiation protocol HSM agent SY agent 1.HSMA-announcement for the supply of the slabs for the current turn. CC-1 agent 2. SYA-Announcement for the production of slabs not available in the SY. HSM agent SY agent 4. SYA-bid 3. CCA-bid(s) 6. Forward of the contract, or renegotiation of the non- satisfied slabs. CC-1 agent CC-n agent HSM agent SY agent CC-1 agent CC-n agent AnnouncingBidding Contracting or renegotiating 5. Establishment of a contract, or renegotiation of the non-satisfied slabs. CC-n agent Open Issues in Grid Scheduling, 2003

23. Prototype developed in simulation A prototype has been developed in Microsoft Visual C++/MFC. Cooperation between the agents is done with the exchange of asynchronous messages formatted in XML using MSMQ. Open Issues in Grid Scheduling, 2003

24. Prototype developed in simulation Open Issues in Grid Scheduling, 2003

25. Evaluation of the performance of the local and global predictive schedules Open Issues in Grid Scheduling, 2003

26. Average frequency of schedule repair and complete rescheduling strategies On the CC agent On the HSM agent Open Issues in Grid Scheduling, 2003

27. Performance of the utility and stability measures On the CC agent On the HSM agent Open Issues in Grid Scheduling, 2003

28. Conclusion Dynamic and autonomous distributed scheduling. The dynamic scheduling problem is distributed across a set of agents. Local autonomy allows the agents to respond to local variations and self-adaptation to real-time events, increasing the robustness and flexibility of the system. The cooperation protocol allows the agents to cooperate and coordinate their local tasks in order to generate desirable globally predictive and robust schedules. Dynamic task allocation. Open Issues in Grid Scheduling, 2003

29. Conclusion Natural load-balancing as busy agents do not need to bid. Increased Flexibility. Robustness against failures. Heterogeneity. Open and extensible scheduling architectures: Agents can be introduced and removed dynamically. Reduced complexity. Reduced costs. Open Issues in Grid Scheduling, 2003