1. Planning with Conceptual Models Mined from User Behavior By Thomas J. Walsh and Michael L. Littman Rutgers University Department of Computer Science {thomaswa, mlittman}@cs.rutgers.edu AAAI Workshop on Acquiring Planning Knowledge via Demonstration July 23, 2007

2. Motivating Example TravelerInfo(t1) => [city1, dest1] CityLookup(city1) => [vancouver] FlightLookup(city1, dest1, 25) => [] FlightLookup(city1, dest1, 50) => [fnum1, air_Canada, f1_Airport, $1200] [fnum2, delta, f2_Airport, $800] ReserveFlight(t1, fnum2, f2_Airport, dest1) => reservation12

3. Problem Description An agent is given a trace of user(s) interacting with an environment via queries. –Describes interactions with web services or databases –Hospital reservations, flights, shopping, clerical tasks –These environments may be too large for active exploration. The agent’s task is to find a “goal” instance. –The goal instance will be judged as in the traces. –Mimic functionality, not specific interactions Trace TravelerInfo(t1). ReserveFlight(t1, fnum2, f2_Airport, dest1) Plan TravelerInfo(“bob”) gets Cities FlightLookup(Cities, 25) Gets Flights and Prices Choose the cheapest.

4. Assumptions Query success/failures are independent for different queries. The response of a query to a fixed set of parameters will not change (stationary) Each query has a single Input/Output concept Part relationships are clearly indicated (e.g. XML).

5. Prior Work Description Logics with Action Formalisms [Baader et al.] Relational Reinforcement Learning (RRL) [Dzeroski et al.] –Both of these plan at an entirely conceptual level and are given the goal instance beforehand Concept models for generalized policies [Martin et al.] –Only mimics user behavior –Chose between instances arbitrarily –Language was complex ( ALNI ) Workflow induction / Process mining looks to build Petri Nets from user traces [van der Aalst et al.] –Related to grammar induction –Often concerned with getting exact models

6. Rule Mining Conceptual Model Traveler TripBetween StartCity Agent Environment Queries Agent’s Instance Model Traveler: t1 TB: c1, c2 SC: c1 Trace TravelerInfo(t1). ReserveFlight(t1, fnum2, f2_Airport, dest1) Language    oneOf Planning/ Execution MOPLEX Architecture (MOdel, PLan, EXecute)

7. Language Constructors We need to represent Concepts, C (Traveler) rather than Instances (Alice) C  C p (primitives, e.g. Traveler) C   R.C (  ReserveFlight.Reservation) C  oneOf{I 1 …I n } (oneOf{25, 50}) C   C (hasPart,  FlightNum  ) C  C  C (  FlightNum  AirLine  Subset of the Description Logic ALO Language    oneOf

8. Building the Model Input Concepts –FlightLookup(City1, Dest1, 25) –Concepts for “known” parameters –TripBetween  (  StartCity   DestCity) –Concepts for “out of the blue” parameters –CitiesAndRadius   TripBetween   oneOf{25} –Finally, link the Input to the Output –CitiesAndRadius   TripBetween   oneOf{25, 50}   FlightLookup.FlightAndPrice

9. Building the Model Output Concepts –FlightLookup(city1, dest1, 50)  [fnum1, airCanada, f1_Airport, $1200] –Manage the non-numerics –Flight  (  FlightNum   Airline   Airport) –Make another concept for numerics (mimics reification) –FlightAndPrice  (  Flight   Price)

10. UML Diagram Conceptual Model Traveler TripBetween StartCity

11. Rules Outside the modeling language itself –Must be applicable immediately Allow us to choose between multiple instances [fnum1, air_Canada, f1_Airport, $1200] [fnum2, delta, f2_Airport, $800] Current implementation just compares numbers –Machine learning classifiers – ILP classifiers –Traditional data mining Rule Mining

12. Planning We focus on using traditional planning techniques on the Conceptual Model itself. –Planning how to instantiate the concepts Cost Structure –Part/Whole edges have 0 cost –“Query edges” have a cost for the query. Agent’s Instance Model Traveler: t1 TB: c1, c2 SC: c1

13. Minimize Parallel Query Cost Assumes no failure, using structure above, query costs of 1, and breadth first search we get: Also have extensions for stochastic queries

14. Amazon Web Services Amazon web-services let users to get information about customers, products, and merchants. I1D963BXJGFASA 2007-07-04 1 0 B000MG62M2 Deluxe Task Chair with Arms

15. Two “Traces” ● ListSearch(Jane@DoeCorp.com) => [ID345] ● ListLookup(ID345) => [[BeanBagChair, ID111],3] [[DeluxeOfficeChair, ID222],1] ● ItemLookup(ID111) => [$35, [IDM2, www.goodDeal.com], new] [$55, [IDM3, www.badDeal.com], new] ● CreateCart(ID111, IDM2,3) => [www.amazon..., IDC543, 1] ● CustomerSearch(John@DoeCorp.com) => [ID976, John] ● CustListLookup(ID976) => [ID112] ● ListLookup(ID112) => [[Lamp, ID333],4] [[Drill, ID444], 1] ● ItemLookup(ID333) => [$50, [IDM2, www.goodDeal.com], new] [$75, [IDM3, www.badDeal.com], new] ● CreateCart(ID111, IDM2,1) => [www.amazon..., IDC543, 1] Buying products on people’s wish lists – Find the item they want the most of, and buy one cheap.

16. Amazon Web Service App

17. Handling Query Failure ? List 2 Chairs 1 Album

18. Conclusions Developed a system that learns an ontology and plans to instantiate concepts. System is robust to query failures Deployed the system in a real world task. Looking forward –More expressive languages and reasoning –More powerful rule mining procedures –Different planning criteria –More “real world” applications using web-services

19. Acknowledgements Special thanks to: DARPA IPTO and the Integrated Learning Project Team Alex Borgida (Rutgers) for many helpful discussions on conceptual modeling

20. References Baader, F., Lutz, C. Milicic, M., Sattler, U., Wolter, F. “Integrating descrition logics and action formalisms: First results”. AAAI, 2005. Dzeroski, S. DeRaedt, L. Driessens, K. “Relational Reinforcement Learning.” Machine Learning 43(1), 7-52, 2001 Martin M. and Geffner, H. “Learning generalized policies from planning examples using concept languages”. Journal of Applied Intelligence (20), 9-19, 2004. van der Aalst, W. and Weijters, A. “Process mining: a research agenda” Computers in Industry 53(3), 231- 244, 2004.