KU NLP Structures and Strategies for State Space Search Depth-First and Breadth-First Search q A search algorithm must determine the order in which states are examined. q Breadth-first search explores the space  A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U (Figure 3.13, p100, tp34)  Figure 3.15 (p 103, tp37) q Depth-first search goes deeper into the search space whenever this is possible.  A, B, E, K, S, L, T, F, M, C, G, N, H, O, P, U, D, I, Q, J, R (Figure 3.13, p100, tp34)  Figure 3.17 (p 105, tp42)

KU NLP Structures and Strategies for State Space Search34 Search Example (Fig p100)

KU NLP Structures and Strategies for State Space Search35 Breadth-First Search (1) Procedure breadth_first_search; begin open := [Start]; closed := []; while open  [] do begin remove leftmost state from open, call it X; if X is a goal then return(success) else begin generate children of X; put X on closed; /* loop check */eliminate children of X on open or closed; /* queue */ put remaining children on right end of open end return(failure) end.

KU NLP Structures and Strategies for State Space Search36 Breadth-First Search(2) 1. open=[A]; closed=[] 2. open=[B,C,D]; closed=[A] 3. open=[C,D,E,F]; closed=[B,A] 4. open=[D,E,F,G,H]; closed=[C,B,A] 5. open=[E,F,G,H,I,J]; closed=[D,C,B,A] 6. open=[F,G,H,I,J,K,L]; closed=[E,D,C,B,A] 7. open=[G,H,I,J,K,L,M] (as L is already on open); closed=[F,E,D,C,B,A] 8. open=[H,I,J,K,L,M,N]; closed=[G,F,E,D,C,B,A] 9. And so on until either U is found or open=[]

KU NLP Structures and Strategies for State Space Search37 Breadth first search (3)

KU NLP Structures and Strategies for State Space Search38 Breadth-First Search (4) q OPEN lists states that have been generated but whose children have not been examined. q CLOSED records states that have already been examined. q OPEN is maintained as a queue, FIFO data structure.  States are added to the right of the list and removed from the left q Breadth-first search is guaranteed to find the shortest path from the start state to the goal. q If the path is required for a solution, we can store ancestor information along with each state.  open = [(D,A), (E,B), (F,B), (G,C), (H,C)]  closed = [(C,A), (B,A), (A,nil)]

KU NLP Structures and Strategies for State Space Search39 Depth-First Search (1) Procedure depth_first_search; begin open := [Start]; closed := []; while open  [] do begin remove leftmost state from open, call it X; if X is a goal then return(success) else begin generate children of X; put X on closed; eliminate children of X on open or closed; put remaining children on left end of open end return(failure) end.

KU NLP Structures and Strategies for State Space Search40 Depth-First Search (2) 1. open=[A]; closed=[] 2. open=[B,C,D]; closed=[A] 3. open=[E,F,C,D]; closed=[B,A] 4. open=[K,L,F,C,D]; closed=[E,B,A] 5. open=[S,L,F,C,D]; closed=[K,E,B,A] 6. open=[L,F,C,D]; closed=[S,K,E,B,A] 7. open=[T,F,C,D]; closed=[L,S,K,E,B,A] 8. open=[F,C,D]; closed=[T,L,S,K,E,B,A] 9. open=[M,C,D], as L is already on closed; closed=[F,T,L,S,K,E,B,A] 10. open=[C,D]; closed=[M,F,T,L,S,K,E,B,A] 11. open=[G,H,D]; closed=[C,M,F,T,L,S,K,E,B,A]

KU NLP Structures and Strategies for State Space Search41 Depth-First Search (3) q OPEN is maintained as a stack, or LIFO data structure.  The state are both added and removed from the left end of OPEN. q is not guaranteed to find the shortest path.

KU NLP Structures and Strategies for State Space Search42 Depth first search (4)

KU NLP Structures and Strategies for State Space Search43 Depth-First vs. Breadth-First  The choice depends on the specific problem being solved.  Significant features include the importance of finding the shortest path, the branching of the state space, the available time and space resources, the average length of paths to a goal node, and whether we want all solutions or only the first solution.  Breadth-First  always finds the shortest path to a goal node.  If there is a bad branching factor, the combinatorial explosion may prevent the algorithm from finding a solution.  Depth-First  may not waste time searching a large number of shallow state in the graph.  can get lost deep in a graph missing shorter paths to goal.

KU NLP Structures and Strategies for State Space Search Using the State Space to Represent Reasoning with the Predicate Calculus q State Space Description of a Logical System q AND/OR Graphs q Further Examples and Applications  MACSYMA (integration)  Where is Fred?  The Financial Advisor  English Grammar

KU NLP Structures and Strategies for State Space Search State Space Description of a Logical System (p107) q Predicate calculus can be used as the formal specification language for making nodes distinguishable as well as for mapping the nodes of a graph onto the state space. q Inference rules can be used to create and describe the arcs between states. q Problems in the predicate calculus, such as determining whether a particular expression is a logical consequence of a given set of assertions, may be solved using search.

KU NLP Structures and Strategies for State Space Search46 Example propositional calculus (1) q A set of assertions : q  p; r  p; v  q; s  r; t  r; s  u; s; t; q State space graph of a set of implications p qr u v  the arcs correspond to logical implications (  )  propositions given true (s and t) correspond to the given data of the problem ts

KU NLP Structures and Strategies for State Space Search47 Example propositional calculus (2) q Propositions that are logical consequences of the given set of assertions correspond to the nodes that may be reached along a directed path from a state representing a true proposition.  [s,r,p] corresponds to the sequence of inferences: s and s  r yields r. r and r  p yields p. q Determining whether a given proposition is a logical consequence of a set of propositions becomes a problem of finding a path from a boxed node to the goal node.

KU NLP Structures and Strategies for State Space Search And/Or graphs (1)  If the premises of an implication are connected by an  operator, they are called AND nodes, and the arcs from this node are joined by a curved link.  q  r  p is represented by q And/Or graph is actually a specialization of a type of graph known as a hypergraph, which connects nodes by sets of arcs.

KU NLP Structures and Strategies for State Space Search And/Or graphs (2) q Definition : HYPERGRAPH  A hypergraph consists of : N, a set of nodes. H, a set of hyperarcs.  Hyperarcs are also known as k-connections, where K is the cardinality of the set of descendant nodes. If k=1, OR node, If k>1, AND nodes.

KU NLP Structures and Strategies for State Space Search AND/OR graphs (3)

KU NLP Structures and Strategies for State Space Search51 And/Or graph Search   operator(and nodes) indicates a problem decomposition in which the problem is broken into subproblems such that all of the subproblems must be solved to solve the original problem.   operator indicates a selection, a point at which a choice may be made between alternative problem-solving strategies.

KU NLP Structures and Strategies for State Space Search52 Example 3.3.3: Integration (1) q One example of an and/or graph is a program for symbolically integrating mathematical functions. (Fig. 3.22, p112, tp53) q In performing integrations, break an expression into sub-expressions that may be integrated independently and then combine the results algebraically into one solution expression.  Decomposition of a problem into independent subproblems can be represented by AND nodes in the graph. q Simplification of an expression can be done by various algebraic substitutions  A number of different substitutions are represented by OR nodes of the graph. q An obvious example of Goal-directed search

KU NLP Structures and Strategies for State Space Search53 Example 3.3.3: Integration (2)

KU NLP Structures and Strategies for State Space Search54 Example 3.3.4: “Where is fred?”(1) q Given facts and rules 1. collie(fred). 2. master(fred, sam). 3. day(saturday). 4.  (warm(saturday)). 5. trained(fred). 6.  X[spaniel(X)  (collie(X)  trained(X))  gooddog(X)] 7.  (X,Y,Z)[goodog(X)  master(X,Y)  location(Y,Z)  location(X,Z)] 8. day(saturday)  warm(saturday)  location(sam, park). 9. day(saturday)   (warm(saturday))  location(sam, museum).

KU NLP Structures and Strategies for State Space Search55 Example 3.3.4: “Where is fred?”(2)

KU NLP Structures and Strategies for State Space Search56 Example 3.3.4: “Where is fred?”(3) q To determine Fred’s location  caluse 7: location(fred, Z)  The premises of the rule 7 must be proved.  gooddog(fred)  master(fred,Y)  location(Y,Z).  rule 6 & fact 1 & fact 5  gooddog(fred) is proved.  master(fred,sam) by fact 2. {sam/Y}  location(sam, Z) must be proved.  Rule 7 is failed (because gooddog(sam) is not true)  rule 8 is failed. (because warm(saturday) is not true)  rule 9 & fact 3 & fact 4  location(sam, museum)  Location(fred, museum) q Goal driven search of the AND/OR graph

KU NLP Structures and Strategies for State Space Search57 Example 3.3.5: Investment Advice q an example of goal directed depth first search with backtracking q Goal: investment(X)  try investment(savings).  must prove savings_account(inadequate).  amount_saved(X)  dependents(Y)   greater(X, minsavings(Y))  (given X=20,000, Y=2)  greater(X,minsavings(Y)) is false.  backtrack to investment(stocks).  savings_account(adequate)  income(adequate).  both are proved true.

KU NLP Structures and Strategies for State Space Search58 Example 3.3.5: Investment Advice

KU NLP Structures and Strategies for State Space Search59 Example 3.3.6: English Grammar(1) q A set of rewrite rules for parsing sentences can be represented by AND/OR graph. q The rules are used to parse sequence of words, i.e. to determine whether they are well-formed sentences. q Simple subset of English grammar: 1. Sentence  NP VP6. ART  a 2. NP  N7. ART  the 3. NP  ART N8. N  man 4. VP  V9. N  dog 5. VP  V NP10. V  likes 11. V  bites

KU NLP Structures and Strategies for State Space Search60 Example 3.3.6: English Grammar(2)

KU NLP Structures and Strategies for State Space Search61 Example 3.3.6: English Grammar(3) q AND/OR graph(Fig. 3.25, tp60) define rewrite rules  AND nodes correspond to RHS of the rule.  Multiple rules with the same conclusion from the OR node. q An expression is well formed in a grammar if it consists entirely of terminal symbols and there is a series of substitutions in the expression using rewrite rules that reduce it to the SENTENCE symbol. q A date-driven parsing algorithm parse the input sentence constructing the parse tree.  “The dog bites the man.” (Fig. 3.26, p119, tp62) q Rewrite rules are used to generate legal sentences.

KU NLP Structures and Strategies for State Space Search62 Example 3.3.6: English Grammar(4)