Introduction to search Chapter 3

Why study search? §Search is a basis for all AI l search proposed as the basis of intelligence l all learning algorithms, e.g., can be seen as searching some space of hypothesis

Newell & Simon: physical symbol hypothesis §“A physical-symbol system has the necessary & sufficient means for general intelligent action”

Physical-systems §“A physical-system consists of a set of entities, called symbols, which are physical patterns that can occur as components of another type of entity called an expression… Thus a symbol structure is composed of a number of instances (or tokens) of symbols related in some physical way (such as one token being next to another). At any instant of time, the system will contain a collection of these symbol structures. Besides these structures, the system also contains a collection of processes that operate on expressions to produce other expressions: processes of creation, modification, reproduction, & destruction. A physical-symbol system is a machine that produces through time an evolving collection of symbol structures.”

Intelligence as search §“The solutions to problems are represented as symbol structures. A physical-symbol system exercises its intelligence in problem- solving by search -- that is, by generating & progressively modifying symbol structures until it produces a solution structure.”

The search task §Given l a description of the initial state l list of legal actions preconditions effects l goal test -- tells you if a goal has been reached §Find l an ordered list of actions to perform in order to go from the initial state to the goal

Search as a graph §Nodes l search-space states §Directed arcs l legal actions from each state to another §Example §Rather than being given, however, we often build the graph (implicitly) as we go; we aren’t given the graph explicitly

Trip as search §State: geographical location (city, street, etc.) §Actions: following a road §Initial state: Madison §Goal state: Minneapolis

ID3 as search §Search space: space of all decision trees possible using feature set §Operator (action): add a node (expand the tree) §State (node): decision tree §Initial state: single leaf node §Goal state: tree that properly categorizes (separates) the training examples

ID3 heuristic §Problem: search space very large; given that we can’t look at all of the possibilities, what’s a good next state? -- info gain (heuristic) §Info gain can be seen as a scoring function that guides us through the space of possible decision trees

General search methods §Given a set of search-space nodes, which one should we consider next? l Youngest [depth-first search] most recently created/expanded node l Oldest [breadth-first search] least-recently created/expanded node l Best [best-first search] requires a scoring function l Random simulated annealing

Example of search strategies

“Considering” a node §If goal-state?(node) then done else expand(node) l expand = add previously unvisited children (states that be gotten to from node) to the OPEN list l if we want the cheapest path, we must check that we can’t find a cheaper route example

General search algorithm

Search algorithm explained §Task: find a route from the start node to the goal node §Desire: try all/many possible paths §OPEN list allows backtracking from a path that “dies out”; saves other possible ways l nodes under consideration (to be expanded) §CLOSED list prevents us from repeatedly trying a node & ending up in an infinite loop l nodes that have been processed

Search danger: infinite spaces §We must be careful that our search doesn’t go forever l the goal we are looking for may not be in the search space but because the space is infinite we might not know this l the goal may be in the search space but we go down an infinite branch l example

Implementing search strategies §Breadth l queue: first in, first out l put new nodes at the back of list §Depth l stack: last in, first out l put new nodes at the front of list §Best l priority queue l add new nodes then sort list §The next node to consider is popped off the front of the OPEN list

BFS tradeoffs §pros l guaranteed to find a solution if it exists l guaranteed to find the shortest solution (in terms of arcs) §cons l OPEN becomes too big [O(b d )] l example

DFS tradeoffs §pros l might find solution quickly l needs less space for OPEN [O(b * d)] l example §cons l can get stuck (run forever) in infinite spaces l might not find shortest solution

Best-first search tradeoffs §pro l can use domain specific knowledge §con l requires a good heuristic (scoring) function

Fixing DFS: iterative deepening §Combines the strengths of breadth- & depth-first search §do DFS but with depth limited to k l if find solution, done (return path, etc.) l else, increment k & start over §don’t save partial solutions from previous iterations; too much memory required §due to exponential growth, most work is at the bottom §guaranteed to find the shortest solution, but OPEN doesn’t get too big (as in BFS)

Iterative deepening idea §We do only a little more work overall, but our storage needs are much less l we get the good aspects of BFS -- shortest solution l without the negative aspects -- OPEN list too big

Search space examples §Water jug problem §Missionaries & cannibals §Define l state representation (k-rep) l initial state l goal state l operators §Then draw search space