Probabilistic Temporal Planning with Uncertain Durations Mausam Joint work with Daniel S. Weld University of Washington Seattle

Motivation Three features of real world planning domains Concurrency Calibrate while rover moves Uncertain Effects ‘Grip a rock’ may fail Uncertain Durative actions Wheels spin, so speed uncertain

Contributions Novel Challenges Large number of decision epochs Results to manage this blowup in different cases Large branching factors Approximation algorithms Five planning algorithms  DUR prun : optimal  DUR samp : near-optimal  DUR hyb : anytime with user defined error  DUR exp : super-fast  DUR arch : balance between speed and quality Identify fundamental issues for future research

Outline of the talk Background Theory Algorithms and Experiments Summary and Future Work

Outline of the talk Background MDP Decision Epochs: happenings, pivots Theory Algorithms and Experiments Summary and Future Work

Markov Decision Process S : a set of states, factored into Boolean variables. A : a set of actions P r ( S£A£S! [0,1]): the transition model C ( A! R ) : the cost model s 0 : the start state G : a set of absorbing goals unit duration

GOAL of an MDP Find a policy ( S ! A ) which: minimises expected cost of reaching a goal for a fully observable Markov decision process if the agent executes for indefinite horizon. Algorithms Value iteration, Real Time Dynamic Programming, etc. iterative dynamic programming algorithms

Definitions (Durative Actions) Assumption: (Prob.) TGP Action model Preconditions must hold until end of action. Effects are usable only at the end of action. Decision epochs: time point when a new action may be started. Happenings: A point when action finishes. Pivot: A point when action could finish.

Outline of the talk Background Theory Explosion of Decision Epochs Algorithms and Experiments Summary and Future Work

Decision Epochs (TGP Action Model) Deterministic Durations [Mausam&Weld05] : Decision Epochs = set of happenings Uncertain Durations: Non-termination has information! Theorem: Decision Epochs = set of pivots

Illustration: A bimodal distribution Duration distribution of a Expected Completion Time

Conjecture if all actions have duration distributions independent of effects unimodal duration distributions then Decision Epochs = set of happenings

Outline of the talk Background Theory Algorithms and Experiments Expected Durations Planner Archetypal Durations Planner Summary and Future Work

Planning with Durative Actions MDP in an augmented state space X 1 : Application of b on X X a b c Time

Uncertain Durations: Transition Fn a, b 0.25 a b b a b a a b action a : uniform(1,2) action b : uniform(1,2)

Branching Factor If n actions m possible durations r probabilistic effects Then Potential Successors (m-1)[(r+1) n – r n – 1] + r n

Algorithms Five planning algorithms  DUR prun : optimal  DUR samp : near-optimal  DUR hyb : anytime with user defined error  DUR exp : super-fast  DUR arch : balance between speed and quality

Expected Durations Planner (  DUR exp ) assign each action a deterministic duration equal to the expected value of its distribution. build a deterministic duration policy for this domain. repeat execute this policy and wait for interrupt (a) action terminated as expected – do nothing (b) action terminated early – replan from this state (c) action terminated late – revise a’s deterministic duration and replan for this domain until goal is reached

Planning Time

Multi-modal distributions Recall: conjecture holds only for unimodal distributions happenings if unimodal Decision epochs = pivots if multimodal

Multi-modal Durations: Transition Fn a, b 0.25 a b b a b a a b action a : uniform(1,2) action b : 50% : 1 50% : 3

Multi-modal Distributions Expected Durations Planner (  Dur exp ) One deterministic duration per action Big approximation for multi-modal distribution Archetypal Durations Planner (  Dur arch ) Limited uncertainty in durations One duration per mode of distribution

Planning Time (multi-modal)     

Expected Make-span (multi-modal)     

Outline of the talk Background Theory Algorithms and Experiments Summary and Future Work Observations on Concurrency

Summary Large number of Decision Epochs Results to manage explosion in specific cases Large branching factors Expected Durations Planner Archetypal Durations Planner (multi-modal)

Handling Complex Action Models So Far: Probabilistic TGP Preconditions hold over-all. Effects usable only at end. What about: Probabilistic PDDL2.1 ? Preconditions at-start, over-all, at-end Effects at-start, at-end Decision epochs must be arbitrary points.

Ramifications Result independent of uncertainty!! Existing decision epoch planners are incomplete. SAPA, Prottle, etc. All IPC winners p, : q a b G G q : p q p preconditions effects

PDDL2.1 (NO UNCERTAINTY!) Theorem: Restricting decision epochs to pivots causes ‘incompleteness’ A problem may be incorrectly deemed unsolvable Exciting future work! p, : q a b G G q : p q p preconditions effects

Related Work Tempastic (Younes and Simmons’ 04) Generate, Test and Debug Prottle (Little, Aberdeen, Thiebaux’ 05) Planning Graph based heuristics Uncertain Durations w/o concurrency Foss and Onder’05 Boyan and Littman’00 Bresina et.al.’02, Dearden et.al.’03