Temporal Plan Execution: Dynamic Scheduling and Simple Temporal Networks Brian C. Williams 16.412J/6.834J December 2nd, 2002 1

Dynamic Scheduling and Task Dispatch Goals and Environment Constraints Temporal Network Solver Projective Task Expansion Temporal Planner Temporal Plan Dynamic Scheduling and Task Dispatch Task Dispatch Goals Modes Model-based Programming& Execution Task-Decomposition Execution Reactive Task Expansion Observations Commands

Dynamic Scheduling and Task Dispatch Goals and Environment Constraints Temporal Network Solver Projective Task Expansion Temporal Planner Temporal Plan Dynamic Scheduling and Task Dispatch Task Dispatch Goals Modes Model-based Programming& Execution Task-Decomposition Execution Reactive Task Expansion Observations Commands

Outline Temporal Representation Qualitative temporal relations Metric constraints Temporal Constraint Networks Temporal Plans Temporal Reasoning for Planning and Scheduling Flexible Execution through Dynamic Scheduling

Qualitative Temporal Constraints (Allen 83) x before y x meets y x overlaps y x during y x starts y x finishes y x equals y y after x y met-by x y overlapped-by x y contains x y started-by x y finished-by x y equals x X Y X Y X Y Y X X Y Y X Y X

Example: Deep Space One Remote Agent Experiment Timer Idle Max_Thrust Idle Th_Seg contained_by equals meets Start_Up Shut_Down Thr_Boundary Thrust Standby Th_Sega Idle_Seg Accum_NO_Thr Accum_Thr CP(Ips_Tvc) SEP_Segment Accum SEP Action Attitude Poke

Qualitative Temporal Constraints Maybe Expressed as Inequalities (Vilain, Kautz 86) x before y X+ < Y- x meets y X+ = Y- x overlaps y (Y- < X+) & (X- < Y+) x during y (Y- < X-) & (X+ < Y+) x starts y (X- = Y-) & (X+ < Y+) x finishes y (X- < Y-) & (X+ = Y+) x equals y (X- = Y-) & (X+ = Y+) Inequalities may be expressed as binary interval relations: X+ - Y- < [-inf, 0]

Metric Constraints Going to the store takes at least 10 minutes and at most 30 minutes. 10 < [T+(store) – T-(store)] < 30 Bread should be eaten within a day of baking. 0 < [T+(baking) – T-(eating)] < 1 day Inequalities, X+ < Y- , may be expressed as binary interval relations: - inf < [X+ - Y-] < 0

Metric Time: Quantitative Temporal Constraint Networks (Dechter, Meiri, Pearl 91) A set of time points Xi at which events occur. Unary constraints (a0 < Xi < b0 ) or (a1 < Xi < b1 ) or . . . Binary constraints (a0 < Xj - Xi < b0 ) or (a1 < Xj - Xi < b1 ) or . . .

Temporal Constraint Satisfaction Problem (TCSP) < Xi, Ti , Tij > Xi continuous variables {I1, . . . ,In } interval constraints where Ii = [ai,bi] interval Ti = (ai £ Xi £ bi) or . . . or (ai £ Xi £ bi) Tij = (a1£ Xi - Xj £ b1) or ... or (an £ Xi - Xj £ bn) [Dechter, Meiri, Pearl, aij89]

TCSP Are Visualized Using Directed Constraint Graphs 1 3 4 2 [10,20] [30,40] [60,inf] [20,30] [40,50] [60,70]

Simple Temporal Networks (Dechter, Meiri, Pearl 91) A set of time points Xi at which events occur. Unary constraints (a0 < Xi < b0 ) or (a1 < Xi < b1 ) or . . . Binary constraints (a0 < Xj - Xi < b0 ) or (a1 < Xj - Xi < b1 ) or . . . Sufficient to represent: most Allen relations simple metric constraints Can’t represent: Disjoint tokens

Simple Temporal Network Tij = (aij£ Xi - Xj £ bij) 1 3 4 2 [10,20] [30,40] [60,inf] [20,30] [40,50] [60,70]

Delta_V(direction=b, magnitude=200) A Completed Plan Forms an STN Thrust Goals Delta_V(direction=b, magnitude=200) Power Attitude Point(a) Turn(a,b) Point(b) Turn(b,a) Engine Off Warm Up Thrust (b, 200) Off

A Completed Plan Forms an STN       

Outline Temporal Representation Temporal Reasoning for Planning and Scheduling TCSP Queries Induced Constraints Plan Consistency Static Scheduling Flexible Execution through Dynamic Scheduling

TCSP Queries (Dechter, Meiri, Pearl, AIJ91) Is the TCSP consistent? What are the feasible times for each Xi? What are the feasible durations between each Xi and Xj? What is a consistent set of times? What are the earliest possible times? What are the latest possible times?

Flexible Sequence (Plans) Histories(?) Goals Projective Task Expansion Temporal Planner Scheduler Flexible Sequence (Plans) Task Dispatch Plan Runner Goals Modes Model-based Execution (w planner) Task-Decomposition Execution Reactive Task Expansion Observations Commands

TCSP Queries (Dechter, Meiri, Pearl, AIJ91) Is the TCSP consistent? Planning What are the feasible times for each Xi? What are the feasible durations between each Xi and Xj? What is a consistent set of times? Scheduling What are the earliest possible times? Scheduling What are the latest possible times?

To Query an STN Map to a Distance Graph Gd = < V,Ed > Edge encodes an upper bound on distance to target from source. Xj - Xi £ bij Xi - Xj £ - aij Tij = (aij£ Xj - Xi £ bij) 1 3 4 2 [10,20] [30,40] [40,50] [60,70] 70 1 3 4 2 20 50 -10 40 -30 -40 -60

Gd Induces Constraints Path constraint: i0 =i, i1 = . . ., ik = j Conjoined path constraints result in the shortest path as bound: where dij is the shortest path from i to j

Shortest Paths of Gd 70 1 2 4 3 20 50 -10 40 -30 -40 -60 d-graph

STN Minimum Network d-graph STN minimum network

Conjoined Paths are Computed using All Pairs Shortest Path (e. g Conjoined Paths are Computed using All Pairs Shortest Path (e.g., Floyd-Warshall’s algorithm ) 1. for i := 1 to n do dii 0; 2. for i, j := 1 to n do dij aij; 3. for k := 1 to n do 4. for i, j := 1 to n do 5. dij min{dij, dik + dkj}; k i j

Testing Plan Consistency No negative cycles: -5 > TA – TA = 0 70 1 2 4 3 20 50 -10 40 -30 -40 -60 d-graph

Latest Solution 20 40 1 2 -10 -30 -10 20 50 3 4 -40 -60 70 d-graph Node 0 is the reference. 20 40 1 2 -10 -30 -10 20 50 3 4 -40 -60 70 d-graph

Solution: Latest Times S1 = (d01, . . . , d0n) 20 40 1 3 -10 -30 -10 20 50 2 4 -40 -60 70

Earliest Solution 20 40 1 2 -10 -30 -10 20 50 3 4 -40 -60 70 d-graph Node 0 is the reference. 20 40 1 2 -10 -30 -10 20 50 3 4 -40 -60 70 d-graph

Solution: Earliest Times S1 = (-d10, . . . , -dn0) 20 40 1 3 -10 -30 -10 20 50 2 4 -40 -60 70

Scheduling: Feasible Values Latest Times X1 in [10, 20] X2 in [40, 50] X3 in [20, 30] X4 in [60, 70] Earliest Times d-graph

Scheduling without Search: Solution by Decomposition Select value for 1 15 [10,20] d-graph

Scheduling without Search: Solution by Decomposition Select value for 1 15 [10,20] d-graph

Solution by Decomposition Select value for 1 15 Select value for 2, consistent with 1 45 [40,50], 15+[30,40] d-graph

Solution by Decomposition Select value for 1 15 Select value for 2, consistent with 1 45 [45,50] d-graph

Solution by Decomposition Select value for 1 15 Select value for 2, consistent with 1 45 [45,50] d-graph

Solution by Decomposition Select value for 1 15 Select value for 2, consistent with 1 45 Select value for 3, consistent with 1 & 2 30 [20,30], 15+[10,20],45+[-20,-10] d-graph

Solution by Decomposition Select value for 1 15 Select value for 2, consistent with 1 45 Select value for 3, consistent with 1 & 2 30 [25,30] d-graph

Solution by Decomposition Select value for 1 15 Select value for 2, consistent with 1 45 Select value for 3, consistent with 1 & 2 30 [25,30] d-graph

Solution by Decomposition Select value for 1 15 Select value for 2, consistent with 1 45 Select value for 3, consistent with 1 & 2 30 d-graph Select value for 4, consistent with 1,2 & 3 O(N2)

In General Solving TCSPs is NP Hard Forward(I1, . . ., Ii) 1. if i = m then 2. M M union Solve-STP(I1, . . ., Im), and 3. Go-Back(I1, . . ., Im); 4. Ci+1 empty; 5. for every Ij in Di+1 do 6. if Consistent-STP (I1, . . . , Ii , Ij), then 7. for i, j := 1 to n do 8. dij min{dij, dik + dkj};

Outline Temporal Representation Temporal Reasoning for Planning and Scheduling Flexible Execution through Dynamic Scheduling

Dynamic Scheduling and Task Dispatch Goals and Environment Constraints Temporal Network Solver Projective Task Expansion Temporal Planner Temporal Plan Dynamic Scheduling and Task Dispatch Task Dispatch Goals Modes Model-based Programming& Execution Task-Decomposition Execution Reactive Task Expansion Observations Commands

Executing Flexible Temporal Plans [Muscettola, Morris, Pell et al.] Handling delays and fluctuations in task duration: Least commitment temporal plans leave room to adapt EXECUTIVE CONTROLLED SYSTEM flexible execution adapts through dynamic scheduling Assigns time to event when executed.

Issues in Flexible Execution How do we minimize execution latency? How do we schedule at execution time?

Time Propagation Can Be Costly EXECUTIVE CONTROLLED SYSTEM

Time Propagation Can Be Costly EXECUTIVE CONTROLLED SYSTEM

Time Propagation Can Be Costly EXECUTIVE CONTROLLED SYSTEM

Time Propagation Can Be Costly EXECUTIVE CONTROLLED SYSTEM

Time Propagation Can Be Costly EXECUTIVE CONTROLLED SYSTEM

Time Propagation Can Be Costly EXECUTIVE CONTROLLED SYSTEM

Issues in Flexible Execution How do we minimize execution latency? Propagate through a small set of neighboring constraints. How do we schedule at execution time?

Compile to Efficient Network EXECUTIVE CONTROLLED SYSTEM

Compile to Efficient Network EXECUTIVE CONTROLLED SYSTEM

Compile to Efficient Network EXECUTIVE CONTROLLED SYSTEM

Issues in Flexible Execution How do we minimize execution latency? Propagate through a small set of neighboring constraints. How do we schedule at execution time?

Issues in Flexible Execution How do we minimize execution latency? Propagate through a small set of neighboring constraints. How do we schedule at execution time? Through decomposition?

Dynamic Scheduling by Decomposition 10 2 -2 -1 1 Equivalent Distance Graph Representation A C B D [0,10] [-2,2] [-1,1] Simple Example

Dynamic Scheduling by Decomposition Compute APSP graph Decomposition enables assignment without search A C B D 10 2 -2 -1 1 11 Equivalent All Pairs Shortest Paths (APSP) Representation A C B D 10 2 -2 -1 1 Equivalent Distance Graph Representation

Assignment by Decomposition 11 10 <0,10> -1 B 1 A D <2,11> t = 0 -1 1 -2 C 2 10 <0,10> -2

Assignment by Decomposition 11 t = 3 10 -1 B 1 <0,0> A D <2,11> -1 1 -2 C 2 10 <0,10> -2

Assignment by Decomposition Solution: Assignments must monotonically increase in value. Execute first, all APSP neighbors with negative delays. But C now has to be executed at t =2, which is already in the past! 11 t = 3 10 -1 B 1 <0,0> A D <4,4> -1 1 -2 C 2 10 <2,2> -2

Dispatching Execution Controller Execute an event when enabled and active Enabled - APSP Predecessors are completed Predecessor – a destination of a negative edge that starts at event. Active - Current time within bound of task.

Dispatching Execution Controller Initially: E = Time points w/o predecessors S = {} Repeat: Wait until current_time has advanced st Some TP in E is active All time points in E are still enabled. Set TP’s execution time to current_time. Add TP to S. Propagate time of execution to TP’s APSP immediate neighbors. Add to A, all immediate neighbors that became enabled. TPx enabled if all negative edges starting at TPx have their destination in S. EXECUTIVE

Propagation is Focused Propagate forward along positive edges to tighten upper bounds. forward prop along negative edges is useless. Propagate backward along negative edges to tighten lower bounds. Backward prop along positive edges useless.

Propagation Example S = {A} B A D C 11 10 -1 <0,0> -1 1 1 -1 -2 -2 C 9 2 -2

Propagation Example S = {A} B A D C 11 <1,10> 10 -1 <0,0> -2 C 9 2 -2

Propagation Example S = {A} B A D C 11 <1,10> 10 -1 <0,0> -2 C 9 2 <0,9> -2

Propagation Example S = {A} B A D C 11 <1,10> 10 -1 <2,11> <0,0> -1 1 A D 1 -1 -2 C 9 2 <0,9> -2

Propagation Example S = {A} E = { C } B A D C 11 <1,10> 10 -1 <2,11> <0,0> -1 1 A D 1 -1 -2 C 9 2 <0,9> -2

Reducing Execution Latency Filtering: some edges are redundant remove redundant edges Execution time is O(n) worst case best case 11 10 t = 3 -1 B 1 <0,0> A D <2,11> -1 1 -2 C 2 10 <0,10> -2

Edge Dominance Eliminate edge that is redundant due to the triangle inequality AB + BC = AC C A B 150 -100 100 80 -50 -20

Edge Dominance Eliminate edge that is redundant due to the triangle inequality AB + BC = AC 150 A B -100 80 100 -50 -20 C

Edge Dominance Eliminate edge that is redundant due to the triangle inequality AB + BC = AC 150 A B -100 80 -50 -20 C

Edge Dominance Eliminate edge that is redundant due to the triangle inequality AB + BC = AC 150 A B -100 80 -50 C

An Example of Edge Filtering Start off with the APSP network A C B D -1 10 9 2 -2 1 11

An Example of Edge Filtering Start at A-B-C triangle 11 10 B -1 -1 1 A D 1 -1 -2 C 9 2 -2

An Example of Edge Filtering Look at B-D-C triangle 11 B -1 -1 1 A D 1 -1 -2 C 9 2 -2

An Example of Edge Filtering Look at B-D-C triangle 11 B -1 -1 1 A D 1 -1 -2 C 9 2

An Example of Edge Filtering Look at D-A-B triangle 11 B -1 -1 1 A D 1 -1 C 9 2

An Example of Edge Filtering Look at D-A-C triangle 11 B -1 1 A D 1 -1 C 9 2

An Example of Edge Filtering Look at B-C-D triangle B -1 1 A D 1 -1 C 9 2

An Example of Edge Filtering Look at B-C-D triangle B -1 1 A D 1 -1 C 9

An Example of Edge Filtering Resulting network has less edges than the original 9 1 1 A C B D -1 -1 A C B D 10 2 -2 -1 1

Avoiding Intermediate Graph Explosion Problem: APSP consumes O(n2) space. Solution: Interleave process of APSP construction with edge elimination Never have to build whole APSP graph

Additional Filtering Node Contraction Collapse two events with fixed time between them A |LA| |AL| L B |BL| |LB|

Additional Filtering Node Contraction Collapse two events with fixed time between them A |LA| |AL| L B |BL| |LB|

Additional Filtering Node Contraction Collapse two events with fixed time between them XY B |BL| |LB|

Dynamic Scheduling and Task Dispatch Goals and Environment Constraints Temporal Network Solver Projective Task Expansion Temporal Planner Temporal Plan Dynamic Scheduling and Task Dispatch Task Dispatch Goals Modes Model-based Programming& Execution Task-Decomposition Execution Reactive Task Expansion Observations Commands