Temporal Planning and Resource Allocation Stefanie Chiou, Rob Kochman, and Gary Look

Running Plans in the Real World Need to account for time and resources when creating plans Papers featured: "Executing Reactive, Model-Based Programs through Graph-Based Temporal Planning" by Phil Kim, Brian C. Williams, and Mark Abramson (IJCAI ’01) "Managing Multiple Tasks in Complex, Dynamic Environments" by Michael Freed (AAAI ’98).

Paper Executing Reactive, Model-based Programs through Graph-based Temporal Planning by Phil Kim, Brian Williams, and Mark Abramson

Familiar Examples Mars Climate Orbiter: 12/11/98Mars Polar Lander: 1/3/99

Motivation Embedded programming is hard Easier to reason about state when programming

Overview/Contributions RMPL provides a new programming paradigm for programming robust systems of cooperative autonomous agents TPN -> synthesis of temporal, causal link, and HTN planning A “holy grail” for autonomous agents Planner that implements these ideas

RMPL Intro RMPL supports four types of reasoning about system interactions reasoning about contingencies scheduling inferring hidden state controlling hidden state This paper focuses on first two interaction types

(Model-based) Embedded Programs Embedded programs interact with plant sensors/actuators: Read sensors Set actuators Model-based programs interact with plant state: Read state Write state Embedded Program S Plant Obs Cntrl Model-based Embedded Program S Plant Programmer must map between state and sensors/actuators. Model-based executive maps between sensors, actuators to states. setState getState

Model-based Embedded Program Breakdown Model-based Embedded Program S Plant Model-based executive maps between sensors, actuators to states. Model-based Executive getState setState Sensor data Actuator commands

Example: The model-based program sets engine = thrusting, and the deductive controller... Determines that valves on the backup engine will achieve thrust, and plans needed actions. Deduces that a valve failed - stuck closed Plans actions to open six valves Fuel tank Oxidizer tank Deduces that thrust is off, and the engine is healthy

Time and Contingency Constructs in RMPL if c thennext A do A maintaining C A,B (concurrency) A;B (serialization) A[l,u] (temporal bounds) Choose{A,B} (choose)

RMPL Code Example Group-Enroute()[l,u] = { choose { do { Group-Fly-Path(PATH1) [l*90%,u*90%]; } maintaining PATH1_OK, do { Group-Fly-Path(PATH2) [l*90%,u*90%]; } maintaining PATH2_OK }; { Group-Transmit(FAC,ARRIVED_TAI)[0,2], do { Group-Wait(TAI_HOLD1,TAI_HOLD2)[0,u*10%] } watching PROCEED_OK } A B Path 1 Path 2 Choosing a route from A to B

RMPL’s Representation of Time and Contingencies Important to find a plan quickly Idea: use a plan graph Generalization of Simple Temporal Network (STN) TPN defined (STN + conditionals + choices)

STN example StartEnd

Temporal Planning Networks (TPN) A temporal planning network is just a generalization of a STN Includes ability to represent conditionals and choices

TPN Example Ask(Proceed=Ok)

RMPL -> TPN conversion A [l,u]: invoke activity A between l and u time units

RMPL -> TPN conversion c [l,u]: Assert that condition c is true now until [l,u]

RMPL -> TPN conversion If c thennext A [l,u]: Execute A for [l,u], if condition c is currently satisfied

RMPL -> TPN conversion do A [l,u] maintaining c : Execute A for [l,u], and ensure that condition c holds throughout

RMPL -> TPN conversion A [l 1,u 1 ], B [l 2,u 2 ] : Concurrently execute A for [l 1,u 1 ], and B for [l 2,u 2 ]

RMPL -> TPN conversion A [l 1,u 1 ]; B [l 2,u 2 ] : Execute A for [l 1,u 1 ], and then B for [l 2,u 2 ]

RMPL -> TPN conversion choose {A [l 1,u 1 ]; B [l 2,u 2 ]} : Reduces to A [l 1,u 1 ] or B [l 2,u 2 ] non-deterministically

Kirk Compiles RMPL program into a TPN Searches TPN for a temporally consistent plan Temporally consistent plan is “embedded” into the TPN.

Kirk Phase1 Select plan from TPN Essentially a graph traversal Check plan for temporal consistency Start

Selecting the Plan Start

Checking for Temporal Consistency Convert TPN to a distance graph Run Bellman-Ford to check for negative cycles (if any found, inconsistent)

Converting TPNs to Distance Graphs The interval [a ij,b ij ] represents the statement: a ij ≤ T j -T i ≤ b ij This is equivalent to: T j -T i ≤ b ij and T i -T j ≤ -a ij [10,20] [30,40] [10,20] [60,70] [40,50]

Checking for Temporal Consistency Convert TPN to a distance graph Run Bellman-Ford algorithm to check for negative cycles:

Bellman-Ford Algorithm initializeCosts(G, s) for i=1 to |V(G)|-1 for each edge (u,v) in E(G) updateCost(u, v, w) for each edge (u, v) in E(G) if cost(v) > cost(u) + w(u. v) return false return true

Bellman-Ford Example 0     Source

Bellman-Ford Example 0  20   Source

Bellman-Ford Example 0  20  Source

Bellman-Ford Example  Source

Bellman-Ford Example Source

Bellman-Ford Example Source

Bellman-Ford Example Source

Bellman-Ford Example Source

Kirk Phase 2 Resolve threats and open conditions Analogous to threats and open conditions in causal link planning Identify intervals of inconsistent constraints using Floyd-Warshall Order intervals to resolve threats Close open conditions by making sure open conditions satisfied by some action in the plan

Why This Paper? It’s useful for our term project

Vision "Managing Multiple Tasks in Complex, Dynamic Environments" by Michael Freed (AAAI ’98). Achieve goals in “task environments” Complex Time-pressured Uncertain Co-existing/Interacting

APEX Goal: ATC Goal: simulate human air traffic controllers Largely routine activity Complexity due to many simple tasks Interruptions necessary

APEX Goal: ATC

Resource Conflicts Separate tasks make incompatible demands What to do? Determine relative priority of tasks Assign control to winner Deal with the loser

Conflict Resolution Strategies Shed Eliminate low importance tasks When (Demand > Availability) Delay/Interrupt Introduces complications Circumvent Select methods that use different resources

APEX Architecture: Two Parts Resource Architecture Set of resources Cognitive Perceptual Motor Action Selection Component Resource Architecture World actuators perception commands events

Procedure Definition Language (PDL) Example: Turning on headlights

Procedure Definition Language (PDL) (procedure (index (turn-on-headlights) (step s1 (clear-hand left-hand)) (step s2 (determine-loc headlight-ctl => ?loc)) (step s3 (grasp knob left-hand ?loc) (waitfor ?s1 ?s2)) (step s4 (pull knob left-hand 0.4) (waitfor ?s3)) (step s5 (ungrasp left-hand) (waitfor ?s4)) (step s6 (terminate) (waitfor ?s5))) Example: Turning on headlights

Detecting Conflicts Must determine: Which tasks should be checked and when Preconditions satisfied Resources become available Whether conflict exists between specified tasks Direct and indirect control

PROFILE Clause Denotes resource requirements for a procedure (profile ( )) (profile (left-hand 8 10))

Prioritization of Tasks Used when: New resource conflict detected New information potentially changes a previous prioritization decision

Prioritization Example: Reprioritize (procedure (index (drive-car))... (step s8 (monitor-behind)) (step s9 (reprioritize ?s8) (waitfor (sound-type ?sound car-horn) (loudness ?sound ?db (?if (> ?db 30)))))) (urgency ?y)))

Assigning Priority (step s5 (monitor-fuel-gauge) (priority 3)) (step s6 (monitor-left-traffic) (priority ?x)) (step s7 (monitor-ahead) (priority (+ ?x ?y)))

General Priority Form (priority (importance ) (urgency )) (step s5 (monitor-fuel-gauge) (priority (run-empty) (importance 6) (urgency 2)) (priority (delay-to-other-task) (importance ?x) (urgency 3)) (priority (excess-time-cost refuel) (importance ?x) (urgency ?y)))

Importance vs. Urgency Depends on workload priority b = S*I b + (S max -S)U b S is subjective workload (a heuristic approximation of actual workload); I b and U b represent importance and urgency for a specified basis

Interruption: RESET... (step s4 (turn-on-headlights)) (step s5 (reset) (waitfor (suspended ?s4))

Coping with Interruption Wind-down activities Suspension-time activities Wind-up activities

Wind-down Activities: an Example (step s15 (pull-over) (waitfor (suspended ?self)) (priority (avoid-accident) (importance 10) (urgency 10)))

Interrupt Costs Wind-down, suspension, and wind-up activities incur cost Ongoing task has its priority increased in proportion to interrupt cost (interrupt-cost 5)

Slack Time (step s17 (suspend ?self) (waitfor (shape ?object traffic-signal) (color ?object red))) (step s18 (monitor-object ?object) (waitfor ?s17)) (step s19 (reprioritize ?self) (waitfor (color ?object green)))

Computing Priority IC = interruption cost U = urgency I = importance S = workload

Conflict Resolution Strategies Shed Eliminate low importance tasks When (Demand > Availability) Delay/Interrupt Introduces complications Circumvent Select methods that use different resources

Evaluation and Future Work Strengths and Weaknesses ATC application has identified issues Computing overall priority from base priorities Suppression of base priorities Other priority issues: A, B need X A, C need Y Priority of A must exceed that of B+C