1. Probabilistic Temporal Planning with Uncertain Durations Mausam Joint work with Daniel S. Weld University of Washington Seattle

2. Motivation Three features of real world planning domains Concurrency Calibrate while rover moves Uncertain Effects ‘Grip a rock’ may fail Uncertain Durative actions Wheels spin, so speed uncertain

3. Contributions Novel Challenges Large number of decision epochs Results to manage this blowup in different cases Large branching factors Approximation algorithms Five planning algorithms  DUR prun : optimal  DUR samp : near-optimal  DUR hyb : anytime with user defined error  DUR exp : super-fast  DUR arch : balance between speed and quality Identify fundamental issues for future research

4. Outline of the talk Background Theory Algorithms and Experiments Summary and Future Work

5. Outline of the talk Background MDP Decision Epochs: happenings, pivots Theory Algorithms and Experiments Summary and Future Work

6. Markov Decision Process S : a set of states, factored into Boolean variables. A : a set of actions P r ( S£A£S! [0,1]): the transition model C ( A! R ) : the cost model s 0 : the start state G : a set of absorbing goals unit duration

7. GOAL of an MDP Find a policy ( S ! A ) which: minimises expected cost of reaching a goal for a fully observable Markov decision process if the agent executes for indefinite horizon. Algorithms Value iteration, Real Time Dynamic Programming, etc. iterative dynamic programming algorithms

8. Definitions (Durative Actions) Assumption: (Prob.) TGP Action model Preconditions must hold until end of action. Effects are usable only at the end of action. Decision epochs: time point when a new action may be started. Happenings: A point when action finishes. Pivot: A point when action could finish.

9. Outline of the talk Background Theory Explosion of Decision Epochs Algorithms and Experiments Summary and Future Work

10. Decision Epochs (TGP Action Model) Deterministic Durations [Mausam&Weld05] : Decision Epochs = set of happenings Uncertain Durations: Non-termination has information! Theorem: Decision Epochs = set of pivots

11. Illustration: A bimodal distribution Duration distribution of a Expected Completion Time

12. Conjecture if all actions have duration distributions independent of effects unimodal duration distributions then Decision Epochs = set of happenings

13. Outline of the talk Background Theory Algorithms and Experiments Expected Durations Planner Archetypal Durations Planner Summary and Future Work

14. Planning with Durative Actions MDP in an augmented state space X 1 : Application of b on X. 0 2 4 6 X a b c Time

15. Uncertain Durations: Transition Fn a, b 0.25 a b b a b a a b action a : uniform(1,2) action b : uniform(1,2)

16. Branching Factor If n actions m possible durations r probabilistic effects Then Potential Successors (m-1)[(r+1) n – r n – 1] + r n

17. Algorithms Five planning algorithms  DUR prun : optimal  DUR samp : near-optimal  DUR hyb : anytime with user defined error  DUR exp : super-fast  DUR arch : balance between speed and quality

18. Expected Durations Planner (  DUR exp ) assign each action a deterministic duration equal to the expected value of its distribution. build a deterministic duration policy for this domain. repeat execute this policy and wait for interrupt (a) action terminated as expected – do nothing (b) action terminated early – replan from this state (c) action terminated late – revise a’s deterministic duration and replan for this domain until goal is reached

19. Planning Time

20. Multi-modal distributions Recall: conjecture holds only for unimodal distributions happenings if unimodal Decision epochs = pivots if multimodal

21. Multi-modal Durations: Transition Fn a, b 0.25 a b b a b a a b action a : uniform(1,2) action b : 50% : 1 50% : 3

22. Multi-modal Distributions Expected Durations Planner (  Dur exp ) One deterministic duration per action Big approximation for multi-modal distribution Archetypal Durations Planner (  Dur arch ) Limited uncertainty in durations One duration per mode of distribution

23. Planning Time (multi-modal)     

24. Expected Make-span (multi-modal)     

25. Outline of the talk Background Theory Algorithms and Experiments Summary and Future Work Observations on Concurrency

26. Summary Large number of Decision Epochs Results to manage explosion in specific cases Large branching factors Expected Durations Planner Archetypal Durations Planner (multi-modal)

27. Handling Complex Action Models So Far: Probabilistic TGP Preconditions hold over-all. Effects usable only at end. What about: Probabilistic PDDL2.1 ? Preconditions at-start, over-all, at-end Effects at-start, at-end Decision epochs must be arbitrary points.

28. Ramifications Result independent of uncertainty!! Existing decision epoch planners are incomplete. SAPA, Prottle, etc. All IPC winners p, : q a b G G q : p q p preconditions effects

29. PDDL2.1 (NO UNCERTAINTY!) Theorem: Restricting decision epochs to pivots causes ‘incompleteness’ A problem may be incorrectly deemed unsolvable Exciting future work! p, : q a b G G q : p q p preconditions effects

30. Related Work Tempastic (Younes and Simmons’ 04) Generate, Test and Debug Prottle (Little, Aberdeen, Thiebaux’ 05) Planning Graph based heuristics Uncertain Durations w/o concurrency Foss and Onder’05 Boyan and Littman’00 Bresina et.al.’02, Dearden et.al.’03