Problem Spaces & Search CSE 473 A handful of GENERAL SEARCH TECHNIQUES lie at the heart of practically all work in AI We will encounter the SAME PRINCIPLES again and again in this course, whether we are talking about GAMES LOGICAL REASONING MACHINE LEARNING These are principles for SEARCHING THROUGH A SPACE OF POSSIBLE SOLUTIONS to a problems.
473 Topics Agents & Environments Problem Spaces Search & Constraint Satisfaction Knowledge Repr’n & Logical Reasoning Machine Learning Uncertainty: Repr’n & Reasoning Robotics © D. Weld, D. Fox
Weak Methods “In the knowledge lies the power…” [Feigenbaum] But what if no knowledge???? Generate & test: As weak as it gets © D. Weld, D. Fox
Example: Fragment of 8-Puzzle Problem Space © D. Weld, D. Fox
Problem Space / State Space Search thru a Problem Space / State Space Input: Set of states Operators [and costs] (successor function) Start state Goal state [test] Output: Path: start a state satisfying goal test [May require shortest path] State Space Search is basically a GRAPH SEARCH PROBLEM STATES are whatever you like! OPERATORS are a compact DESCRIPTION of possible STATE TRANSITIONS – the EDGES of the GRAPH May have different COSTS or all be UNIT cost OUTPUT may be specified either as ANY PATH (REACHABILITY), or a SHORTEST PATH © D. Weld, D. Fox
Example: Route Planning Input: Set of states Operators [and costs] Start state Goal state (test) Output: States = cities Start state = starting city Goal state test = is state the destination city? Operators = move to an adjacent city; cost = distance Output: a shortest path from start state to goal state SUPPOSE WE REVERSE START AND GOAL STATES? © D. Weld, D. Fox
Example: N Queens Q Input: Set of states Operators [and costs] Start state Goal state (test) Output: © D. Weld, D. Fox
Search Strategies Blind Search Informed Search Constraint Satisfaction Adversary Search Depth first search Breadth first search Iterative deepening search Iterative broadening search © D. Weld, D. Fox
Search Strategies A search strategy is defined by picking the order of node expansion Strategies are evaluated along the following dimensions: completeness: does it always find a solution if one exists? time complexity: number of nodes generated space complexity: maximum number of nodes in memory optimality: does it always find a least-cost solution? Time and space complexity are measured in terms of b: maximum branching factor of the search tree d: depth of the least-cost solution m: maximum depth of the state space (may be ∞) © D. Weld, D. Fox
Breadth First Search Maintain queue of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? Yes a O(b^d) b c O(b^d) g h d e f © D. Weld, D. Fox
Memory a Limitation? Suppose: 2 GHz CPU 1 GB main memory 100 instructions / expansion 5 bytes / node 200,000 expansions / sec Memory filled in 100 sec … < 2 minutes © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? © D. Weld, D. Fox
Depth First Search Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? Not for infinite spaces O(b^m) O(bm) © D. Weld, D. Fox
Iterative Deepening Search DFS with limit; incrementally grow limit Evaluation Complete? Time Complexity? Space Complexity? Yes a b e O(b^d) c f d i O(bd) g h j k l © D. Weld, D. Fox
Iterative deepening search Number of nodes generated in a depth-limited search to depth d with branching factor b: NDLS = b0 + b1 + b2 + … + bd-2 + bd-1 + bd Number of nodes generated in an iterative deepening search to depth d with branching factor b: NIDS = (d+1)b0 + d b^1 + (d-1)b^2 + … + 3bd-2 +2bd-1 + 1bd For b = 10, d = 5, NDLS = 1 + 10 + 100 + 1,000 + 10,000 + 100,000 = 111,111 NIDS = 6 + 50 + 400 + 3,000 + 20,000 + 100,000 = 123,456 Overhead = (123,456 - 111,111)/111,111 = 11% © D. Weld, D. Fox
Speed BFS Nodes Time Iter. Deep. Nodes Time 8 Puzzle 2x2x2 Rubik’s Assuming 10M nodes/sec & sufficient memory BFS Nodes Time Iter. Deep. Nodes Time 8 Puzzle 2x2x2 Rubik’s 15 Puzzle 3x3x3 Rubik’s 24 Puzzle 105 .01 sec 106 .2 sec 1013 6 days 1019 68k yrs 1025 12B yrs 105 .01 sec 106 .2 sec 1017 20k yrs 1020 574k yrs 1037 1023 yrs © D. Weld, D. Fox Adapted from Richard Korf presentation
When to Use Iterative Deepening N Queens? Q Q Q Q © D. Weld, D. Fox
Forwards vs. Backwards start end © D. Weld, D. Fox
vs. Bidirectional © D. Weld, D. Fox
Repeated states Failure to detect repeated states can turn a linear problem into an exponential one! Graph search: Stores expanded nodes in a set called Closed and only adds new nodes to the fringe © D. Weld, D. Fox
Graph search © D. Weld, D. Fox
Problem All these methods are slow (blind) Solution add guidance (“heurstic estimate”) “informed search” © D. Weld, D. Fox