1. 1 Phase 1 Problems ● Small size – 20 to 150 methods – 10 to 20 agents ● Problem Classes – One static + one dynamically arriving problem – Uncertainty in duration/quality – Too much work to do (only class with more than 70 methods) – Non Local Effects (hard enables + soft facilitates) – Syncronization 1

2. 2 Phase 2 ● Much more complex scenario generator – mix and match various problem classes, now called templates – better control of the assignment of agents to methods ● within a template & between problems – More dynamics (both new tasks & changes to old) – New NLEs (hinder, disable) – New QAFs (Sum-And, Exactly-One) ● Much larger problems – 10–100 agents – 725–10,000 methods 2

3. Experimental Environment ● Scenario Generator generates random scenarios. ● Each scenario contains one or more problems. ● Problems are independent, except that they may overlap in time. Agents can only do one thing at a time. ● Problems consist of Template Instances. ● Templates comprise tasks and methods. ● Methods are individual executable actions.

4. Experimental Environment ● Each problem, and some complex templates, are broken into a sequential set of time windows. ● Each window has an earliest start time and a deadline. ● We can independently control how tight these start time and deadlines are relative to method execution time and how much windows overlap with one another. ● Non-local effects (NLEs) also are limited to be from earlier windows to later windows.

5. 5 Templates ● Simple – Static single-window problems from Phase 1 ● Syncronization – Single sinch point from Phase 1 ● Dynamic Simple – Dynamically arriving simple single-window ● NLE Chain – Multiple windows chained together with enables or facilitates – Recursive template for each window 5

6. 6 Templates ● Multi-Synch – Multiple sych points under a single window ● Enables Tracks – Multiple “tracking” tasks, each with a series of linearly enabled subtasks ● Contingency – multiple “preparation” methods enable multiple “completion” methods. – A crucial task with multiple outcomes determines what “completion” method is actually needed at runtime 6

7. 7 Templates ● Second Chance Dynamic – “preparation” task almost impossible, enables “completion” task. – an “easier” preparation task arrives, allowing an easier second chance at completion ● Circular Soft NLE – Soft NLEs (facilitates, hinders) are set up in cycles, creating a hard optimization problem 7

8. 8 Simple Templates

9. 9 Syncronization Template

10. 10 Dynamic Template

11. 11 NLE Chain Template

12. 12 Multi-Synch Template

13. 13 Enables Tracks Template

14. 14 Contingency Template

15. 15 Second Chance Template

16. 16 Circular Soft NLE Template

17. 17 Experimental Classes [Phase 2] ● General Mix ● Negative Interdependence ● Very Dynamic ● Circular Soft Interdependencies ● Tight Deadlines ● Uncertainty ● Contingency ● Big Interdependence ● 100 Agent Mix 17

18. 18 General Mix ● Basic template mix for most problems ● 10–70 Agents ● 400–3000 Methods ● 1 fast fallback methods ● 2 redundant methods ● Template Mix – Simple[Sum] – Simple[SumAnd] – Dynamic – NLEChain 18 - Multi Synch - Enables Tracks - Contingency - CircularSoftNLE[facilitates]

19. 19 General Mix, 10 Agents, Actual

20. 20 General Mix, 10 Agents, Detail Circular Soft NLE Template Enables Tracks Template NLE Chain (start)

21. 21 Negative Interdependence ● Simple Template with lots of hinders and disables 21

22. 22 Very Dynamic ● Dynamic Template with changes to deadlines, release times, and quality/duration distributions 22 Random changes to: Release Times Deadlines Quality Distributions Duration Distributions

23. 23 Circular Soft Interdependencies ● Circular facilitation creates hard optimization problem 23

24. 24 Tight Deadlines ● Simple Templates with Sum and tight deadlines (must use alternative fallback or redundant methods) 24

25. 25 Uncertainty ● Simple and Dynamic Templates; high uncertainty in duration/quality; 20% method failure Uncertainty make scheduling difficult

26. 26 Contingency

27. 27 Big Interdependence ● NLE Template Mix: NLE Chains, Enables Tracks, Circular Soft NLEs plus random NLEs

28. 28 100 Agent Mix (actual) ● Similar to General Mix without Dynamic changes 28