1. State Agnostic Planning Graphs William Cushing Daniel Bryce Arizona State University {william.cushing, dan.bryce}@asu.edu Special thanks to: Subbarao Kambhampati, David E. Smith, Menkes van den Briel, Romeo Sanchez, J. Benton

2. Motivation  Reachability analysis (via Planning Graphs)  Sets of planning graphs are useful  Progression search  Belief-space planning  Replanning  Robustification  Local search ……  …but highly redundant  PGs overlap (duplicate information)  PGs are inflexible (fixed source)  Generalize PG to multiple sources p5 q5 r5 p6 o pq o pr o 56 p5p5 pqr56pqr56 o pq o pr o 56 pqrst567pqrst567 o ps o qt o 67 q5q5 qtr56qtr56 o qt o qr o 56 qtrsp567qtrsp567 o qs o tp o 67 r5r5 rqp56rqp56 o rq o rp o 56 rqpst567rqpst567 o rs o qt o 67 p6p6 pqr67pqr67 o pq o pr o 67 pqrst678pqrst678 o ps o qt o 78 Introduction 1 3 4 1 3 o 12 o 34 2 1 3 4 5 o 12 o 34 o 23 o 45 2 3 4 5 3 5 o 34 o 56 3 4 5 o 34 o 45 o 56 66 7 o 67 1 5 1 5 o 12 o 56 2 1 3 5 o 12 o 23 o 56 2 66 7 o 67 G oGoG G oGoG G oGoG G oGoG G oGoG 1 3 3 5 1 5 h( )=5

3. Overview Scratch State Agnostic Graph Planning Graphs BuildPG(A) BuildSAG() 1.Labeled Uncertainty Graph [LUG] 2.(Belief) State Agnostic LUG [SA LUG ] 3.Optimized (Belief) State Agnostic LUG [SLUG] ExtractH(A,B) Reachability Queries ExtractH(A,B) Technique: Transform BuildPG(A) into BuildSAG() Introduction

4. Domain  Agent has a bag of letters and digits  “ac45” = {f a =T, f c =T, f 4 =T, f 5 =T, …=F}  Agent can trade letters freely  Trade c for e (o ce ): “ac45” => “ae45”  Trace c for a (o ca ): “ac45” => “a45”  Movement on a clique (zenotravel [fly])  Agent can increment digits  Increment 5 (o 56 ): “ac45” => “ac46”  Increment 4 (o 45 ): “ac45” => “ac5”  Movement along a ray (blocksworld [unstack])

5.  Envelope of Progression Tree (Relaxed Progression)  Linear vs. Exponential Growth  Used for:  Search  Reachability Heuristics p5 q5 r5 p6 q6 t5 p5p5 p7 s6 pqr56pqr56 pqrst567pqrst567 o pq o pr o 56 o qt o 67 o ps o pq o pr o 56 o ps o qt o 67 [Kambhampati et al, ECP, 1997] Planning Graph Basics

6. Multiple Graphs 1 3 3 5 1 5 1 3 4 1 3 o 12 2 1 3 4 5 o 34 o 23 2 3 4 5 3 5 o 34 o 56 3 4 5 o 34 o 45 o 56 66 7 o 67 1 5 1 5 o 56 2 1 3 5 o 12 o 56 2 66 7 o 67 h( )=5 G oGoG G oGoG G G G Heuristics for belief-space  G D. Bryce and S. Kambhampati, “Heuristic Guidance Measures for Conformant Planning”, In ICAPS’04, 2004. o 34 o 45 o 12 o 23 oGoG oGoG oGoG

7. 11 o 12 2 1 o 23 2 Unioned Graphs 1 3 3 5 1 5 3 4 3 o 34 3 4 5 o 45 11 o 12 2 1 o 23 2 3 4 5 3 5 o 34 o 56 3 4 5 o 34 o 45 o 56 66 7 o 67 11 o 12 2 1 3 o 23 2 55 o 56 5 66 7 o 67 G oGoG G oGoG G oGoG G oGoG G oGoG G oGoG G oGoG Heuristics for belief-space  G

8. Unioned Graphs 11 o 12 2 1 o 23 2 3 4 5 3 5 o 34 o 56 3 4 5 o 34 o 45 o 56 66 7 o 67 G oGoG G oGoG G oGoG h( )=1 Heuristics for belief-space  G 1 3 3 5 1 5 3 5

9. h( )=5 Labeled Graph [LUG] 1 3 4 5 1 3 5 o 12 o 34 o 56 2 1 3 4 5 o 12 o 34 o 23 o 45 o 56 2 66 7 o 67 oGoG G GG oGoG oGoG Heuristics for belief-space 1 3 3 5 1 5  G D. Bryce, S. Kambhampati, and D. Smith, “Planning in Belief Space with a Labelled Uncertainty Graph”, In AAAI Workshop on Markov Processes, 2004.

10. Labeled Graph [LUG] 1 ^ 3 ^ -5 ^ γ 3 ^ 5 ^ -1 ^ γ 1 ^ 5 ^ -3 ^ γ 1 ^ 3v5 ^ -3v-5 ^ γ 3 ^ 1v5 ^ -1v-5 ^ γ 5 ^ 1v3 ^ -1v-3 ^ γ 1v3 ^ 3v5 ^ 1v5 ^ -1v-3v-5 ^ γ 1 3 4 5 o 12 o 34 o 56 2 6 oGoG G 1 3 4 5 o 12 o 34 o 23 o 45 o 56 2 6 7 o 67 G oGoG G oGoG  Binary Decision Diagrams  Initialize: and/projection  Operator: and/preconditions  Literal: or/supporters Heuristics for belief-space D. Bryce, S. Kambhampati, and D. Smith, “Planning in Belief Space with a Labelled Uncertainty Graph”, In AAAI Workshop on Markov Processes, 2004. 1 3 3 5 1 5 1 3 5  G γ = “everything else false”

11. Multiple (Labeled) Graphs 1 3 3 5 1 3 3 5 1 5 1 3 1 5 3 5 1 5 1 3 5 1 3 5 1 3 5 1 3 5 G oGoG G oGoG G oGoG G oGoG Single graph for progression D. Bryce, S. Kambhampati, and D. Smith, “Planning Graph Heuristics for Belief Space Search”, ASU CSE TR, 2005.

12. Unioned (Labeled) Graph 1 3 4 5 o 12 o 34 o 23 o 45 o 56 2 6 7 o 67 GG oGoG oGoG 1 3 4 5 2 6 G 1 3 5 o 12 o 34 o 56 oGoG 1 3 3 5 1 5 Single graph for progression

13.  Introduce labels for beliefs over labels for states Labeled (Labeled) Graph [SA LUG ] 1 3 4 5 1 3 5 o 12 o 34 o 56 2 1 3 4 5 o 12 o 34 o 23 o 45 o 56 2 66 7 o 67 oGoG G GG oGoG oGoG 1 3 3 5 1 5 S r v S b S b v S g S r v S g S r ^S b ^S g => 1v3 ^ 3v5 ^ 1v5 ^ -1v-3v-5 ^ γ -S r => 5 ^ 1v3 ^ -1v-3 ^ γ -S b => 1 ^ 3v5 ^ -3v-5 ^ γ -S g => 3 ^ 1v5 ^ -1v-5 ^ γ Single graph for progression W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005.

14. Labeled (Labeled) Graph [SA LUG ] 1 3 4 5 1 3 5 o 12 o 34 o 56 2 1 3 4 5 o 12 o 34 o 23 o 45 o 56 2 66 7 o 67 oGoG G GG oGoG oGoG 1 3 3 5 1 5 Single graph for progression W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005.

15. Filtered Unioned (Labeled) Graph [SLUG] 1 3 1 5 1 3 4 5 1 3 5 o 12 o 34 o 56 2 1 3 4 5 o 12 o 34 o 23 o 45 o 56 2 66 7 o 67 oGoG G GG oGoG oGoG 3 5 Don’t let the name fool you!  Ignore irrelevant labels  Largest LUG == all LUGs Optimized single graph W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005.

16. Belief Space ProblemsClassical Problems Conformant Contingent Empirical Results

17. Conclusion  Developed general agnosticism (SAG)  Removed dependence on world state (PG  LUG)  Removed dependence on belief state (LUG  SA LUG )  Dramatically improved performance ({LUG,SA LUG } ~> SLUG)  Empirically demonstrated  Internal performance boost  Favorable external comparison  SAG has rich connections to:  Constraint propagation (vs. branching)  Lazy evaluation  Memoization

18. Further Details  Heuristics for belief space in the CAltAlt planner  D. Bryce and S. Kambhampati, “Heuristic Guidance Measures for Conformant Planning”, In ICAPS’04, 2004.  Labeled Uncertainty Graph in the CAltAlt planner  D. Bryce, S. Kambhampati, and D. Smith, “Planning in Belief Space with a Labelled Uncertainty Graph”, In AAAI Workshop on Markov Processes, 2004.  Heuristics and LUG in the POND and CAltAlt planners  D. Bryce, S. Kambhampati, and D. Smith, “Planning Graph Heuristics for Belief Space Search”, ASU CSE TR, 2005.  SLUG: Improvement to LUG for POND  W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005.  CLUG: propagating numeric information  D. Bryce and S. Kambhampati, “Cost Sensitive Reachability Heuristics for Handling State Uncertainty”, In UAI’05, 2005.