© Daniel S. Weld 1 Logistics PS3 Project Additional problem Reading Planning & CSP for “today” SAT plan paper review for Wed

© Daniel S. Weld 2 PSet 2 Student(s) = s is a student Takes(s, c, q) = s takes c during quarter q IsQtr(q) = q is a quarter Passes(s, c) = s passes course c Higher(h, g) = grade h is higher than g IsGradeIn(g, c) = g is a grade in course c Every student who takes French passes it Needs time argument Needs person & time args  s,q [student(s)  isQtr(q)  takes(x, French, q)]  passes(s, French) But what if Joe takes French in the fall, fails and then doesn’t take French in the winter? The antecedent is false… but P  Q =  P  Q So the formula is true?!? What’s the fix??

© Daniel S. Weld 3 PSet 2 Higher(h, g) = grade h is higher than g GradeOf(g, s, c, q) = s has grade g in course c during q The best score in Greek is always better than that french  g, q [  s gradeOf(s, g, French, q)]  [  t, h gradeOf(t, h, Greek, q)  Higher(h, g)]

© Daniel S. Weld Topics Agency Problem Spaces SearchKnowledge Representation PlanningUncertainty MDPs Supervised Learning Reinforcement Learning

© Daniel S. Weld 5 Immediate Outline Constraint satisfaction Defn – factoring state spaces Backtracking policies Variable-ordering heuristics & preprocessing The planning problem Searching world states Graphplan SATplan Reachability analysis & heuristics Planning under uncertainty

© Daniel S. Weld 6 Constraint Satisfaction Kind of search in which States are factored into sets of variables Search = assigning values to these variables Structure of space is encoded with constraints Backtracking-style algorithms work E.g. DFS for SAT (i.e. DPLL) But other techniques add speed Propagation Variable ordering Preprocessing

© Daniel S. Weld 7 Chinese Constraint Network Soup Total Cost < $30 Chicken Dish Vegetable RiceSeafood Pork Dish Appetizer Must be Hot&Sour No Peanuts No Peanuts Not Chow Mein Not Both Spicy

© Daniel S. Weld 8 CSPs in the real world Scheduling Space Shuttle Repair Airport gate assignments Transportation Planning Supply-chain management Computer Configuration Diagnosis UI Optimization Etc...

Adapting to Device Characteristics

© Daniel S. Weld 10 Binary Constraint Network Set of n variables: x 1 … x n Value domains for each variable: D 1 … D n Set of binary constraints (also “relations”) R ij  D i  D j Specifies which values pair (x i x j ) are consistent V for each country Each domain = 4 colors R ij enforces 

© Daniel S. Weld 11 Binary Constraint Network Partial assignment of values = tuple of pairs {...(x, a)…} means variable x gets value a... Tuple=consistent if all constraints satisfied Tuple=full solution if consistent + has all vars Tuple {(x i, a i ) … (x j, a j )} = consistent w/ a set of vars {x m … x n } iff  a m … a n such that {(x i, a i )…(x j, a j ), (x m, a m )…(x n, a n )} } = consistent

© Daniel S. Weld 12 N Queens Variables = board columns Domain values = rows R ij = {(a i, a j ) : (a i  a j )  (|i-j|  |a i -a j |) e.g. R 12 = {(1,3), (1,4), (2,4), (3,1), (4,1), (4,2)} Q Q Q {(x 1, 2), (x 2, 4), (x 3, 1)} consistent with (x 4 ) Shorthand: “{2, 4, 1} consistent with x 4 ”

© Daniel S. Weld 13 CSP as a search problem? What are states? (nodes in graph) What are the operators? (arcs between nodes) Initial state? Goal test? Q Q Q

© Daniel S. Weld 14 Chronological Backtracking (BT) (e.g., depth first search) Q Q Q Q Q Q Q Q Q Q Q Consistency check performed in the order in which vars were instantiated If c-check fails, try next value of current var If no more values, backtrack to most recent var

© Daniel S. Weld 15 Backjumping (BJ) Similar to BT, but more efficient when no consistent instantiation can be found for the current var Instead of backtracking to most recent var… BJ reverts to deepest var which was c-checked against the current var BJ Discovers (2, 5, 3, 6) inconsistent with x 6 No sense trying other values of x 5 Q Q Q Q Q

© Daniel S. Weld 16 5 Conflict-Directed Backjumping (CBJ) More sophisticated backjumping behavior Each variable has conflict set CS Set of vars that failed c-checks w/ current val Update this set on every failed c-check When no more values to try for x i Backtrack to deepest var, x d, in CS(x i ) And updateCS(x d ):=CS(x d )  CS(x i )-{x d } CBJ Discovers (2, 5, 3) inconsistent with {x 5, x 6 } Q Q Q Q Q x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 CS(x 5 ) 1,2,3 CS(x 6 ) 1, 2,3,5

© Daniel S. Weld 17 BT vs. BJ vs. CBJ {

© Daniel S. Weld 18 Forward Checking (FC) Perform Consistency Check Forward Whenever a var is assigned a value Prune inconsistent values from As-yet unvisited variables Backtrack if domain of any var ever collapses Q Q Q Q Q FC only visits consistent nodes but not all such nodes skips (2, 5, 3, 4) which CBJ visits But FC can’t detect that (2, 5, 3) inconsistent with {x 5, x 6 }

© Daniel S. Weld 19 Number of Nodes Explored BT=BM BJ=BMJ=BMJ2 CBJ=BM-CBJ FC-CBJ FC More Fewer =BM-CBJ2

© Daniel S. Weld 20 Number of Consistency Checks BMJ2 BT BJ BMJ BM-CBJ CBJ FC-CBJ BM BM-CBJ2 FC More Fewer

© Daniel S. Weld 21 Dynamic variable ordering In the N-queens examples we assumed First x 1 then x 2 then... But this order not required Any order ok with respect to completeness A good order leads to huge speedup A good heuristic (MRV): Choose variable w/ minimum remaining values This is easy if one is doing FC

© Daniel S. Weld 22 DVO MRV => WOW!! Algo17 Queens21 Queens27 Queens FC-CBJ mrv FC-CBJ FC CBJ BJ BT

© Daniel S. Weld 23 Preprocessing Strategies Even FC-CBJ is O(b d ) time worst case Sometimes useful to preprocess before doing exponential search spend polynomial time to achieve local consistency Arc consistency Consider all pairs of vars Can values be eliminated from a domain ala FC Propagate O(d 2 ) time where d= number of vars

© Daniel S. Weld 24 Constraint Satisfaction Recap CSP = Factoring a state space Chronological Backtracking (BT) Backjumping (BJ) Conflict-Directed Backjumping (CBJ) Forward checking (FC) Dynamic variable ordering heuristics Preprocessing Strategies

© Daniel S. Weld 25 Immediate Outline The planning problem Searching world states Constraint satisfaction Graphplan SATplan Reachability analysis & heuristics Planning under uncertainty

© Daniel S. Weld 26 Ways to make “plans” Generative Planning Reason from first principles (knowledge of actions) Requires formal model of actions Case-Based Planning Retrieve old plan which worked on similar problem Revise retrieved plan for this problem Reinforcement Learning Act ”randomly” - noticing effects Learn reward, action models, policy

© Daniel S. Weld 27 Generative Planning Input Description of (initial state of) world (in some KR) Description of goal (in some KR) Description of available actions (in some KR) Output Controller E.g. Sequence of actions E.g. Plan with loops and conditionals E.g. Policy = f: states -> actions

© Daniel S. Weld 28 Input Representation Description of initial state of world E.g., Set of propositions: ((block a) (block b) (block c) (on-table a) (on-table b) (clear a) (clear b) (clear c) (arm-empty)) Description of goal: i.e. set of worlds or ?? E.g., Logical conjunction Any world satisfying conjunction is a goal (and (on a b) (on b c))) Description of available actions

© Daniel S. Weld 29 Simplifying Assumptions Environment Percepts Actions What action next? Static vs. Dynamic Fully Observable vs. Partially Observable Deterministic vs. Stochastic Instantaneous vs. Durative Full vs. Partial satisfaction Perfect vs. Noisy

© Daniel S. Weld 30 Classical Planning Environment Static Fully Observable Deterministic Instantaneous Full Perfect I = initial state G = goal state OiOi (prec)(effects) [ I ] OiOi OjOj OkOk OmOm [ G ]

© Daniel S. Weld 31 StaticDeterministicObservable Instantaneous Propositional “Classical Planning” Dynamic Replanning/ Situated Plans Durative Temporal Reasoning Continuous Numeric Constraint reasoning (LP/ILP) Stochastic Contingent/Conformant Plans, Interleaved execution MDP Policies POMDP Policies Partially Observable Contingent/Conformant Plans, Interleaved execution Semi-MDP Policies

© Daniel S. Weld 32 Today’s Hot Research Areas Durative Actions Simultaneous actions, events, deadline goals Planning Under Uncertainty Modeling sensors; searching belief states [ I ] OiOi OjOj OkOk ? ObOb OaOa OcOc

© Daniel S. Weld 33 Representing Actions Situation Calculus STRIPS PDDL UWL Dynamic Bayesian Networks

© Daniel S. Weld 34 How Represent Actions? Simplifying assumptions Atomic time Agent is omniscient (no sensing necessary). Agent is sole cause of change Actions have deterministic effects STRIPS representation World = set of true propositions Actions: Precondition: (conjunction of literals) Effects (conjunction of literals) a a a north11 north12 W0W0 W2W2 W1W1

© Daniel S. Weld 35 STRIPS Actions Action = function: worldState  worldState Precondition says where function defined Effects say how to change set of propositions a a north11 W0W0 W1W1 precond: (and (agent-at 1 1) (agent-facing north)) effect: (and (agent-at 1 2) (not (agent-at 1 1))) Note: strips doesn’t allow derived effects; you must be complete!

© Daniel S. Weld 36 Action Schemata (:operator pick-up :parameters ((block ?ob1)) :precondition (and (clear ?ob1) (on-table ?ob1) (arm-empty)) :effect (and (not (clear ?ob1)) (not (on-table ?ob1)) (not (arm-empty)) (holding ?ob1))) Instead of defining: pickup-A and pickup-B and … Define a schema: Note: strips doesn’t allow derived effects; you must be complete! }

© Daniel S. Weld 37 Immediate Outline Constraint satisfaction The planning problem Searching world states Regression Heuristics Graphplan SATplan Reachability analysis & heuristics Planning under uncertainty

© Daniel S. Weld 38 Planning as Search Nodes Arcs Initial State Goal State World states Actions The state satisfying the complete description of the initial conds Any state satisfying the goal propositions

© Daniel S. Weld 39 Forward-Chaining World-Space Search A C B C B A Initial State Goal State

© Daniel S. Weld 40 Backward-Chaining Search Thru Space of Partial World-States D C B A E D C B A E D C B A E * * * Problem: Many possible goal states are equally acceptable. From which one does one search? A C B Initial State is completely defined D E

© Daniel S. Weld 41 Regression Regressing a goal, G, thru an action, A Yields the weakest precondition G’ Such that: if G’ is true before A is executed G is guaranteed to be true afterwards A G precond effect G’ Represents a set of world states

© Daniel S. Weld 42 Regression Example pick-up :parameters ((block ?ob1)) :precondition (and (clear ?ob1) (on-table ?ob1) (arm-empty)) :effect (and (not (clear ?ob1)) (not (on-table ?ob1)) (not (arm-empty)) (holding ?ob1))) A G precond effect G’ (and (holding C) (on A B)) (and (clear A) (on-table A) (arm-empty) (on A B)) Disjunction  preconditions

© Daniel S. Weld 43 Conditional Effects

© Daniel S. Weld 44 Regressing Conditional Effects A G precond effect G’ (and (at keys home) (at paycheck bank)) (and (at briefcase bank) (in keys briefcase) (not (in paycheck briefcase)) (at paycheck bank)) bank home

© Daniel S. Weld 45 Plan Space Forward chaining thru world-states Backward chaining thru world-states

© Daniel S. Weld 46 “Causal Link” Planning Nodes Arcs Initial State Goal State Partially specified plans Adding + deleting actions or constraints (e.g. <) to plan The empty plan (Actually two dummy actions…) A plan which when simulated achieves the goal Need efficient way to evaluate quality (percentage of preconditions satisfied) of partial plan … Hence causal link datastructures

© Daniel S. Weld 47 Plan-Space Search pick-from-table(C) pick-from-table(B) pick-from-table(C) put-on(C,B) How represent plans? How test if plan is a solution?