1. Temporal Planning and Resource Allocation Stefanie Chiou, Rob Kochman, and Gary Look

2. Running Plans in the Real World Need to account for time and resources when creating plans Papers featured: "Executing Reactive, Model-Based Programs through Graph-Based Temporal Planning" by Phil Kim, Brian C. Williams, and Mark Abramson (IJCAI ’01) "Managing Multiple Tasks in Complex, Dynamic Environments" by Michael Freed (AAAI ’98).

3. Paper Executing Reactive, Model-based Programs through Graph-based Temporal Planning by Phil Kim, Brian Williams, and Mark Abramson

4. Familiar Examples Mars Climate Orbiter: 12/11/98Mars Polar Lander: 1/3/99

5. Motivation Embedded programming is hard Easier to reason about state when programming

6. Overview/Contributions RMPL provides a new programming paradigm for programming robust systems of cooperative autonomous agents TPN -> synthesis of temporal, causal link, and HTN planning A “holy grail” for autonomous agents Planner that implements these ideas

7. RMPL Intro RMPL supports four types of reasoning about system interactions reasoning about contingencies scheduling inferring hidden state controlling hidden state This paper focuses on first two interaction types

8. (Model-based) Embedded Programs Embedded programs interact with plant sensors/actuators: Read sensors Set actuators Model-based programs interact with plant state: Read state Write state Embedded Program S Plant Obs Cntrl Model-based Embedded Program S Plant Programmer must map between state and sensors/actuators. Model-based executive maps between sensors, actuators to states. setState getState

9. Model-based Embedded Program Breakdown Model-based Embedded Program S Plant Model-based executive maps between sensors, actuators to states. Model-based Executive getState setState Sensor data Actuator commands

10. Example: The model-based program sets engine = thrusting, and the deductive controller... Determines that valves on the backup engine will achieve thrust, and plans needed actions. Deduces that a valve failed - stuck closed Plans actions to open six valves Fuel tank Oxidizer tank Deduces that thrust is off, and the engine is healthy

11. Time and Contingency Constructs in RMPL if c thennext A do A maintaining C A,B (concurrency) A;B (serialization) A[l,u] (temporal bounds) Choose{A,B} (choose)

12. RMPL Code Example Group-Enroute()[l,u] = { choose { do { Group-Fly-Path(PATH1) [l*90%,u*90%]; } maintaining PATH1_OK, do { Group-Fly-Path(PATH2) [l*90%,u*90%]; } maintaining PATH2_OK }; { Group-Transmit(FAC,ARRIVED_TAI)[0,2], do { Group-Wait(TAI_HOLD1,TAI_HOLD2)[0,u*10%] } watching PROCEED_OK } A B Path 1 Path 2 Choosing a route from A to B

13. RMPL’s Representation of Time and Contingencies Important to find a plan quickly Idea: use a plan graph Generalization of Simple Temporal Network (STN) TPN defined (STN + conditionals + choices)

14. STN example StartEnd

15. Temporal Planning Networks (TPN) A temporal planning network is just a generalization of a STN Includes ability to represent conditionals and choices

16. TPN Example Ask(Proceed=Ok)

17. RMPL -> TPN conversion A [l,u]: invoke activity A between l and u time units

18. RMPL -> TPN conversion c [l,u]: Assert that condition c is true now until [l,u]

19. RMPL -> TPN conversion If c thennext A [l,u]: Execute A for [l,u], if condition c is currently satisfied

20. RMPL -> TPN conversion do A [l,u] maintaining c : Execute A for [l,u], and ensure that condition c holds throughout

21. RMPL -> TPN conversion A [l 1,u 1 ], B [l 2,u 2 ] : Concurrently execute A for [l 1,u 1 ], and B for [l 2,u 2 ]

22. RMPL -> TPN conversion A [l 1,u 1 ]; B [l 2,u 2 ] : Execute A for [l 1,u 1 ], and then B for [l 2,u 2 ]

23. RMPL -> TPN conversion choose {A [l 1,u 1 ]; B [l 2,u 2 ]} : Reduces to A [l 1,u 1 ] or B [l 2,u 2 ] non-deterministically

24. Kirk Compiles RMPL program into a TPN Searches TPN for a temporally consistent plan Temporally consistent plan is “embedded” into the TPN.

25. Kirk Phase1 Select plan from TPN Essentially a graph traversal Check plan for temporal consistency Start

26. Selecting the Plan Start

27. Checking for Temporal Consistency Convert TPN to a distance graph Run Bellman-Ford to check for negative cycles (if any found, inconsistent)

28. Converting TPNs to Distance Graphs The interval [a ij,b ij ] represents the statement: a ij ≤ T j -T i ≤ b ij This is equivalent to: T j -T i ≤ b ij and T i -T j ≤ -a ij 0 3 1 4 2 [10,20] [30,40] [10,20] [60,70] [40,50] 0 3 1 4 2 20 40 20 70 50 -60 -40 -10 -30 -10

29. Checking for Temporal Consistency Convert TPN to a distance graph Run Bellman-Ford algorithm to check for negative cycles:

30. Bellman-Ford Algorithm initializeCosts(G, s) for i=1 to |V(G)|-1 for each edge (u,v) in E(G) updateCost(u, v, w) for each edge (u, v) in E(G) if cost(v) > cost(u) + w(u. v) return false return true

31. Bellman-Ford Example 0     20 40 20 70 50 -60 -40 -10 -30 -10 Source

32. Bellman-Ford Example 0  20   40 20 70 50 -60 -40 -10 -30 -10 Source

33. Bellman-Ford Example 0  20  60 20 40 20 70 50 -60 -40 -10 -30 -10 Source

34. Bellman-Ford Example 0 50 20  60 20 40 20 70 50 -60 -40 -10 -30 -10 Source

35. Bellman-Ford Example 0 50 20 100 60 20 40 20 70 50 -60 -40 -10 -30 -10 Source

36. Bellman-Ford Example 0 50 20 70 60 20 40 20 70 50 -60 -40 -10 -30 -10 Source

37. Bellman-Ford Example 0 30 20 70 60 20 40 20 70 50 -60 -40 -10 -30 -10 Source

38. Bellman-Ford Example 0 30 20 70 50 20 40 20 70 50 -60 -40 -10 -30 -10 Source

39. Kirk Phase 2 Resolve threats and open conditions Analogous to threats and open conditions in causal link planning Identify intervals of inconsistent constraints using Floyd-Warshall Order intervals to resolve threats Close open conditions by making sure open conditions satisfied by some action in the plan

40. Why This Paper? It’s useful for our term project

41. Vision "Managing Multiple Tasks in Complex, Dynamic Environments" by Michael Freed (AAAI ’98). Achieve goals in “task environments” Complex Time-pressured Uncertain Co-existing/Interacting

42. APEX Goal: ATC Goal: simulate human air traffic controllers Largely routine activity Complexity due to many simple tasks Interruptions necessary

43. APEX Goal: ATC

46. Resource Conflicts Separate tasks make incompatible demands What to do? Determine relative priority of tasks Assign control to winner Deal with the loser

47. Conflict Resolution Strategies Shed Eliminate low importance tasks When (Demand > Availability) Delay/Interrupt Introduces complications Circumvent Select methods that use different resources

48. APEX Architecture: Two Parts Resource Architecture Set of resources Cognitive Perceptual Motor Action Selection Component Resource Architecture World actuators perception commands events

49. Procedure Definition Language (PDL) Example: Turning on headlights

50. Procedure Definition Language (PDL) (procedure (index (turn-on-headlights) (step s1 (clear-hand left-hand)) (step s2 (determine-loc headlight-ctl => ?loc)) (step s3 (grasp knob left-hand ?loc) (waitfor ?s1 ?s2)) (step s4 (pull knob left-hand 0.4) (waitfor ?s3)) (step s5 (ungrasp left-hand) (waitfor ?s4)) (step s6 (terminate) (waitfor ?s5))) Example: Turning on headlights

51. Detecting Conflicts Must determine: Which tasks should be checked and when Preconditions satisfied Resources become available Whether conflict exists between specified tasks Direct and indirect control

52. PROFILE Clause Denotes resource requirements for a procedure (profile ( )) (profile (left-hand 8 10))

53. Prioritization of Tasks Used when: New resource conflict detected New information potentially changes a previous prioritization decision

54. Prioritization Example: Reprioritize (procedure (index (drive-car))... (step s8 (monitor-behind)) (step s9 (reprioritize ?s8) (waitfor (sound-type ?sound car-horn) (loudness ?sound ?db (?if (> ?db 30)))))) (urgency ?y)))

55. Assigning Priority (step s5 (monitor-fuel-gauge) (priority 3)) (step s6 (monitor-left-traffic) (priority ?x)) (step s7 (monitor-ahead) (priority (+ ?x ?y)))

56. General Priority Form (priority (importance ) (urgency )) (step s5 (monitor-fuel-gauge) (priority (run-empty) (importance 6) (urgency 2)) (priority (delay-to-other-task) (importance ?x) (urgency 3)) (priority (excess-time-cost refuel) (importance ?x) (urgency ?y)))

57. Importance vs. Urgency Depends on workload priority b = S*I b + (S max -S)U b S is subjective workload (a heuristic approximation of actual workload); I b and U b represent importance and urgency for a specified basis

58. Interruption: RESET... (step s4 (turn-on-headlights)) (step s5 (reset) (waitfor (suspended ?s4))

59. Coping with Interruption Wind-down activities Suspension-time activities Wind-up activities

60. Wind-down Activities: an Example (step s15 (pull-over) (waitfor (suspended ?self)) (priority (avoid-accident) (importance 10) (urgency 10)))

61. Interrupt Costs Wind-down, suspension, and wind-up activities incur cost Ongoing task has its priority increased in proportion to interrupt cost (interrupt-cost 5)

62. Slack Time (step s17 (suspend ?self) (waitfor (shape ?object traffic-signal) (color ?object red))) (step s18 (monitor-object ?object) (waitfor ?s17)) (step s19 (reprioritize ?self) (waitfor (color ?object green)))

63. Computing Priority IC = interruption cost U = urgency I = importance S = workload

64. Conflict Resolution Strategies Shed Eliminate low importance tasks When (Demand > Availability) Delay/Interrupt Introduces complications Circumvent Select methods that use different resources

65. Evaluation and Future Work Strengths and Weaknesses ATC application has identified issues Computing overall priority from base priorities Suppression of base priorities Other priority issues: A, B need X A, C need Y Priority of A must exceed that of B+C