1. Minh Do - PARC Planning with Goal Utility Dependencies J. Benton Department of Computer Science Arizona State University Tempe, AZ j.benton@asu.edu Subbarao Kambhampati Department of Computer Science Arizona State University Tempe, AZ rao@asu.edu Minh Do Embedded Reasoning Area Palo Alto Research Center Palo Alto, CA minh.do@parc.com Menkes van den Briel Department of Computer Science Arizona State University Tempe, AZ j.benton@asu.edu

2. Minh Do - PARC Planning with Goal Utility Dependencies Classical vs. Over-Subscription Planning Classical Planning Initial state Set of goals Actions Find a plan that achieves all goals (prefer plans with fewer actions) Over-Subscription Planning Initial state Goals with differing utilities Actions with differing costs Find a plan with highest net benefit (cumulative utility – cumulative cost) (best plan may not achieve all the goals) 1/19

3. Minh Do - PARC Planning with Goal Utility Dependencies Goal Dependencies cost dependencies Actions achieving different goals interact positively or negatively Goals may complement or substitute each other utility dependencies goal interactions exist as two distinct types Handled by previous planners AltWlt, Optiplan, Sapa PS Investigated in this paper a 2/19

4. Minh Do - PARC Planning with Goal Utility Dependencies Complementary Goals Cost: 10 Util: 20 Cost: 100 Util: 50 Cost: 110 Util: 300 3/19

5. Minh Do - PARC Planning with Goal Utility Dependencies Substitute Goals Cost: 15000 Util: 30000 Cost: 45000 Util: 35000 4/19

6. Minh Do - PARC Planning with Goal Utility Dependencies Conditional Dependency Cost: 500 Util: 1000 Cost: 500 Util: 0

7. Minh Do - PARC Planning with Goal Utility Dependencies Challenges 1.Modeling goal utility dependencies 2.Doing planning in the presence of utility (and cost) dependencies 5/19

8. Minh Do - PARC Planning with Goal Utility Dependencies Modeling Goal Dependencies General Additive Independence (GAI) Model (Bacchus & Grove, 1995) Utility over sets of dependent goals Util: 20Util: 50Util: 300 6/19

9. Minh Do - PARC Planning with Goal Utility Dependencies Planning Approaches Bounded-horizon Optimal using Integer Linear Programming (ILP): Supports action cost and goal utility naturally Handles the objective function of maximizing utility-cost tradeoffs easily Concern: scaling up with complex problems Heuristic Search: Scale up well Challenge: Developing good heuristics Pursue two different approaches 7/19

10. Minh Do - PARC Planning with Goal Utility Dependencies I I G AT_STORE AT_HOME IN_CAR AT_OFFICE AT_HOME t = 1t = 2t = 3 Shoes Car G1SC IP Encoding IP Constraints: Capture the flow of each variable from I to G Interaction between different flows IPPlan (van den Briel et. al., 2005) 8/19

11. Minh Do - PARC Planning with Goal Utility Dependencies iPUD : Extended G1SC Encoding Remove hard constraints on goal achievement. Introduce a new binary variable for each related goal set S. Add constraints to ensure that S is achieved when achieved (and vice versa). New objective function capturing goal utility dependencies. 9/19

12. Minh Do - PARC Planning with Goal Utility Dependencies SPUDS : Heuristic Search Approach I Forward State-space Planning Search using A* Node evaluation: g = U(G S ) – Cost(P S ) | h(S): expected additional benefit Output better quality solutions given more time (anytime) S1S1 S2S2 S3S3 S4S4 S5S5 Extending Sapa PS (2004) 10/19

13. Minh Do - PARC Planning with Goal Utility Dependencies Heuristic: Relaxed Plan Step 1: Estimate the lowest cost relaxed plan P + achieving all remaining goals I S all remaining goals A3A3 A1A1 A4A4 A2A2 relaxed plan Going backward, greedily select lowest cost action achieving all remaining goals Introduced in FF (Hoffman & Nebel, 2001) 11/19

14. Minh Do - PARC Planning with Goal Utility Dependencies Heuristic S A3A3 A1A1 A4A4 A2A2 Step 2: Build Cost-dependencies between actions in P + Build the supported goal set: GS(A 1 ) = {G 3 }, GS(A 2 ) = {G 1,G 2,G 3 } G1G1 G2G2 G3G3 12/19

15. Minh Do - PARC Planning with Goal Utility Dependencies Heuristic Step 3: Extract the optimal relaxed plan within P + S A3 A2 G1 G2 G3 Removing costly goals and actions (solely) supporting them Set up and solve an IP encoding using GS(A) and goal utility dependencies functions f(S) A1 A4 13/19

16. Minh Do - PARC Planning with Goal Utility Dependencies Heuristic: Summary Step 1: Estimate the lowest cost relaxed plan P + achieving all remaining goals Step 2: Build cost-dependencies between goals in P + Step 3: Find the optimal relaxed plan within P + 14/19 Approximate the relaxed plan with the best utility-cost tradeoff

17. Minh Do - PARC Planning with Goal Utility Dependencies Experimental Setup IPC problems in 4 benchmark domains: Satellite, ZenoTravel, TPP, Rovers Goals randomly selected as “hard” or “soft” Goal utilities and action costs randomly generated within upper/lower bounds. Goal dependencies randomly generated Compared iPud, SPUDS (three heuristics) and Sapa PS. 15/19

18. Minh Do - PARC Planning with Goal Utility Dependencies Time-bounded Solution Quality Problems 11-20 for each domain: Total of 40 hardest problems 16/19

19. Minh Do - PARC Planning with Goal Utility Dependencies Quality & Time Comparison 20 problems for each domain (harder going from left to right) Planning time limit: 600 secs 17/19

20. Minh Do - PARC Planning with Goal Utility Dependencies Summary Goal utility dependency in GAI framework iPud: Bounded-horizon optimal plans SPUDS: Heuristic forward search cost dependencies + utility dependencies IP encoding for heuristic improvement 18/19

21. Minh Do - PARC Planning with Goal Utility Dependencies Future Work Quantitative + Qualitative preference (PrefPlan by Brafman & Chernyavsky) PDDL3 preference model Residual cost as in AltWlt (Sanchez & Kambhampati) 19/19

22. Minh Do - PARC Planning with Goal Utility Dependencies Thank You !