1. AI – Week 16 Machine Learning Applied to AI Planning Lee McCluskey, room 2/09 Email lee@hud.ac.uklee@hud.ac.uk http://scom.hud.ac.uk/scomtlm/cha2555/

2. Artform Research Group Machine Learning applied to AI Planning 2 main ways --- n Refining the knowledge you already have Automated Skill Acquisition - learning heuristics / macros. Make planners generate plans more quickly, generate better plans, solve problems they couldn’t solve before. n Gaining new knowledge from the environment Automated Knowledge Acquisition: learning the domain model. Make planners be able to plan in new environments by eg inducing operator schema from example plans

3. Artform Research Group Part 1: Refining the knowledge you already have n Learning macros cf week 13 – RECALL Example.. move(R,A,B) pre: at(R,A), A \= Beff+: at(R,B) eff-: at(R,A) load(R,C,L) pre: at(R,L), at(C,L) eff+: in(C,R) eff-: at(C,L) unload(R,C,L) pre: at(R,L), in(C,R) eff+: at(C,L) eff-: in(C,R) Planning problem and solution: initial state = at(r1, london), at(r2, london), at(c1,paris), at(c2,paris) goal = in(c1,r1), in(c2,r1) Solution = [move(r1, london, paris), load(r1, c1,paris), load(r1, c1,paris)]

4. Artform Research Group WP Technique – cf week 13 WP( T, G) = WP'(reverse(T),G) WP'( empty list, G) = G WP'(T,G) = WP'( tail(T), pre(head(T)) U (G \ eff+(head(T))). G = in(c1,r1), in(c2,r1) head(T) = load(r1, c1,paris)]

5. Artform Research Group WP Technique – cf week 13 WP( T, G) = WP'(reverse(T),G) WP'( empty list, G) = G WP'(T,G) = WP'( tail(T), pre(head(T)) U (G \ eff+(head(T))). G = in(c1,r1), in(c2,r1) head(T) = load(r1, c1,paris)] at(r1,paris), at(c1,paris), in(c2,r1)

6. Artform Research Group WP Technique – cf week 13 WP( T, G) = WP'(reverse(T),G) WP'( empty list, G) = G WP'(T,G) = WP'( tail(T), pre(head(T)) U (G \ eff+(head(T))). G = in(c1,r1), in(c2,r1) head(T) = load(r, c1,paris)] at(r1,paris), at(c1,paris), in(c2,r1) [load(r1, c1,paris), move(r1,london,paris)] ….

7. Artform Research Group WP Technique – cf week 13 WP( T, G) = WP'(reverse(T),G) WP'( empty list, G) = G WP'(T,G) = WP'( tail(T), pre(head(T)) U (G \ eff+(head(T))). G = in(c1,r1), in(c2,r1) head(T) = load(r1, c1,paris)] at(r1,paris), at(c1,paris), in(c2,r1) [load(r1, c1,paris), move(r1,london,paris)] …. at(r1,london), london =/ paris, at(c1,paris), at(c2,paris)

8. Artform Research Group Macro Creation Assemble Macro: (G, WP(T,G), T) Means “to solve G when WP(T,G) is in the initial state, use T” ({in(c1,r1), in(c2,r1)}, {at(r1,london), london =/ paris, at(c1,paris), at(c2,paris)}, [ move(r1, london, paris), load(r1, c1,paris), load(r1, c1,paris) ] ) Generalised Macro: ({in(?c1,?r1), in(?c2,?r1)}, {at(?r1,?1l), ?l1 =/ ?l2, at(?c1,?l2), at(?c2,?l2)}, [ move(?r1, ?l1, ?l2), load(?r1, ?c1,?l2), load(?r1, ?c1,?l2) ] )

9. Artform Research Group FORWARD, STATE SPACE SEARCH INITIAL STATE A1 A2 A3 SOLUTION NODE

10. Artform Research Group MACRO USE : FORWARD, STATE SPACE SEARCH WITH MACROS INITIAL STATE Could get to a solution in a one step!! = macro X provides the steps in [A1,A2,A3] So no need for long search. A1 A2 A3 SOLUTION NODE X

11. Artform Research Group MACRO USE : FORWARD, STATE SPACE SEARCH WITH MACROS - PROBLEMS INITIAL STATE BUT number of instances of (generalised) macros may be huge  This called the UTILITY problem. Machine Learning systems much be CAREFUL in what is learned. A1 A2 A3 SOLUTION NODE X

12. Artform Research Group Part 2: Gaining New Knowledge From the Environment n Knowledge Acquisition and Knowledge Engineering are processes of eliciting and encoding knowledge in a way that intelligent processes can use effectively. KA/KE are normally done by people. n Some AI programs need an explicit KB and some don’t Before we look at programs that learn by acquiring knowledge for themselves, lets consider what KA/KE mean.

13. Artform Research Group KA/KE for AI Planning: Terminology DOMAIN = APPLICATION AREA Symbolic World Domain Model Knowledge Engineering (KE) Knowledge Acquisition Domain Model Language Real or Conceptualised World

14. Artform Research Group General KB created by “manual” KA/KE [(one_or_both_of Segment1 and Segment2 are_flown_at_subsonic_speed) & ( the_Aircraft_on(Segment1) and the_Aircraft_on(Segment2) meet_mnps) & ( the_Aircraft_on(Segment1) and the_Aircraft_on(Segment2) are_jets & (the_Profile_containing(Segment1) & the_Profile_containing(Segment2) are_wholly_or_partly_in_the_ mnps_airspace) ] => [(the_basic_min_longitudinal_sep _Val_in_mins_required_for Segment1 and Segment2) = 10 …. ETC

15. Artform Research Group Planning KB created by “manual” KA/KE Eg PDDL domain model and problem spec files …………… (:action stack :parameters ( ?P - block ?A - block) :precondition (and (clear ?A) (holding ?P)) :effect (and (not (holding ?P)) (not (clear ?A))(on ?P ?A) )) ……………

16. Artform Research Group KA/KE in AI Planning KA/KE for planning is hard because: - Its knowledge about CHANGE, rather than static or purely descriptive knowledge i.e. its hard to represent knowledge about time, events, actions, processes etc - Planning is a synthetic task i.e. it puts together or creates something – harder than e.g. diagnosis - KE different from SE because we are trying to encode “rich” data to be acted on by reasoning processes which rely on heuristics or approximation to work.

17. Artform Research Group Using Machine Learning for KA/KE One Way: Give a learning system a number of PLANS, or let it “observe” plans e.g. fetch_jack jack1 boot1 remove_wheel wheel0 hub0 jack0 jack_up hub1 jack1 Get the learning system to learn the details of the actions by “inducing” the operator schema

18. Workflow Example (:action remove_wheel :parameters (?Wheel1 - wheel ?Hub2 - hub ?Jack3 - jack) :precondition (and (wheel_state2 ?Wheel1 ?Hub2) (hub_state1 ?Hub2 ?Jack3 ?Wheel1) (jack_state1 ?Jack3 ?Hub2)) :effect (and (wheel_state1 ?Wheel1) (not (wheel_state2 ?Wheel1 ?Hub2)) (hub_state0 ?Hub2 ?Jack3) (not (hub_state1 ?Hub2 ?Jack3 ?Wheel1)) (jack_state0 ?Jack3 ?Hub2) (not (jack_state1 ?Jack3 ?Hub2))) ) Trace of Changing a Car Wheel Meaning of “remove_wheel” Action in Planning-ready format fetch_jack jack1 boot1 remove_wheel wheel0 hub0 jack0 jack_up hub1 jack1 put_on_wheel wheel2 hub0 jack0 fetch_wrench wrench0 boot0 jack_down hub1jack1 putaway_wrench wrench1 boot0 ………………..etc Induction

19. Inducing Domain Models - Game Example Send to home.. Move to Free cell.... Move to Free Cell.... Move to column.... Send to home... (:action sendtohome : parameters (?card - card ?suit - suitsort ?vcard - denomination ?homecard - card ?vhomecard ?cols ?ncols - denomination) :precondition (and (clear ?card) (bottomcol ?card) (home ?homecard) (suit ?card ?suit) (suit ?homecard ?suit) (value ?card ?vcard) (value ?homecard ?vhomecard) (successor ?vcard ?vhomecard) (colspace ?cols) (successor ?ncols ?cols)) : effect (and (home ?card) (colspace ?ncols) (not (home ?homecard)) (not (clear ?card)) (not (bottomcol ?card)) (not (colspace ?cols)))) Trace of FreeCell Game Meaning of “Send to home” Action in Planning-ready format Induction

20. Artform Research Group Conclusion n Learning can be classed as Knowledge Refinement (eg learning to perform better) or Knowledge Acquisition (eg learning new domains, new knowledge) n This can be applied to planning: -Knowledge Refinement: Learning macros (this week) -Knowledge Acquisition: Learning PDDL operator schema (next week) [-- or how you can get a program to do your coursework ]