Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 5 of 41 Wednesday 03.

Slides:



Advertisements
Similar presentations
Review: Search problem formulation
Advertisements

Uninformed search strategies
Artificial Intelligence Presentation
Artificial Intelligence Chapter 9 Heuristic Search Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.
Artificial Intelligence Problem Solving Eriq Muhammad Adams
CS 480 Lec 3 Sept 11, 09 Goals: Chapter 3 (uninformed search) project # 1 and # 2 Chapter 4 (heuristic search)
Blind Search1 Solving problems by searching Chapter 3.
Search Strategies Reading: Russell’s Chapter 3 1.
May 12, 2013Problem Solving - Search Symbolic AI: Problem Solving E. Trentin, DIISM.
1 Blind (Uninformed) Search (Where we systematically explore alternatives) R&N: Chap. 3, Sect. 3.3–5.
Solving Problem by Searching Chapter 3. Outline Problem-solving agents Problem formulation Example problems Basic search algorithms – blind search Heuristic.
Lets remember about Goal formulation, Problem formulation and Types of Problem. OBJECTIVE OF TODAY’S LECTURE Today we will discus how to find a solution.
Touring problems Start from Arad, visit each city at least once. What is the state-space formulation? Start from Arad, visit each city exactly once. What.
CSC 423 ARTIFICIAL INTELLIGENCE
Artificial Intelligence (CS 461D)
Search Strategies CPS4801. Uninformed Search Strategies Uninformed search strategies use only the information available in the problem definition Breadth-first.
Artificial Intelligence Lecture No. 7 Dr. Asad Safi ​ Assistant Professor, Department of Computer Science, COMSATS Institute of Information Technology.
Computing & Information Sciences Kansas State University Lecture 4 of 42 CIS 530 / 730 Artificial Intelligence Lecture 4 of 42 William H. Hsu Department.
14 Jan 2004CS Blind Search1 Solving problems by searching Chapter 3.
CS 380: Artificial Intelligence Lecture #3 William Regli.
SE Last time: Problem-Solving Problem solving: Goal formulation Problem formulation (states, operators) Search for solution Problem formulation:
Review: Search problem formulation
Search I Tuomas Sandholm Carnegie Mellon University Computer Science Department [Read Russell & Norvig Chapter 3]
Artificial Intelligence Chapter 3: Solving Problems by Searching
CS 188: Artificial Intelligence Spring 2006 Lecture 2: Queue-Based Search 8/31/2006 Dan Klein – UC Berkeley Many slides over the course adapted from either.
CS 561, Session 6 1 Last time: Problem-Solving Problem solving: Goal formulation Problem formulation (states, operators) Search for solution Problem formulation:
Computing & Information Sciences Kansas State University Lecture 3 of 42 CIS 530 / 730 Artificial Intelligence Lecture 3 of 42 William H. Hsu Department.
Review: Search problem formulation Initial state Actions Transition model Goal state (or goal test) Path cost What is the optimal solution? What is the.
1 Problem Solving and Searching CS 171/271 (Chapter 3) Some text and images in these slides were drawn from Russel & Norvig’s published material.
Problem Solving and Search Andrea Danyluk September 11, 2013.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 3 Wednesday 27 August.
Computing & Information Sciences Kansas State University Lecture 2 of 42 CIS 530 / 730 Artificial Intelligence Lecture 2 of 42 William H. Hsu Department.
CS 440 / ECE 448 Introduction to Artificial Intelligence Spring 2010 Lecture #3 Instructor: Eyal Amir Grad TAs: Wen Pu, Yonatan Bisk Undergrad TAs: Sam.
Computing & Information Sciences Kansas State University Monday, 28 Aug 2006CIS 490 / 730: Artificial Intelligence Lecture 3 of 42 Monday, 28 August 2006.
CHAPTER 4: INFORMED SEARCH & EXPLORATION Prepared by: Ece UYKUR.
Search Tamara Berg CS 560 Artificial Intelligence Many slides throughout the course adapted from Dan Klein, Stuart Russell, Andrew Moore, Svetlana Lazebnik,
AI in game (II) 권태경 Fall, outline Problem-solving agent Search.
Informed Search Strategies Lecture # 8 & 9. Outline 2 Best-first search Greedy best-first search A * search Heuristics.
For Friday Finish reading chapter 4 Homework: –Lisp handout 4.
For Monday Read chapter 4, section 1 No homework..
Search with Costs and Heuristic Search 1 CPSC 322 – Search 3 January 17, 2011 Textbook §3.5.3, Taught by: Mike Chiang.
Review: Tree search Initialize the frontier using the starting state While the frontier is not empty – Choose a frontier node to expand according to search.
Lecture 3: Uninformed Search
Computing & Information Sciences Kansas State University Wednesday, 30 Aug 2006CIS 490 / 730: Artificial Intelligence Lecture 4 of 42 Wednesday, 30 August.
Search CPSC 386 Artificial Intelligence Ellen Walker Hiram College.
Advanced Artificial Intelligence Lecture 2: Search.
A General Introduction to Artificial Intelligence.
For Friday Read chapter 4, sections 1 and 2 Homework –Chapter 3, exercise 7 –May be done in groups.
Uninformed Search ECE457 Applied Artificial Intelligence Spring 2007 Lecture #2.
1 Kuliah 4 : Informed Search. 2 Outline Best-First Search Greedy Search A* Search.
Goal-based Problem Solving Goal formation Based upon the current situation and performance measures. Result is moving into a desirable state (goal state).
Uninformed search strategies A search strategy is defined by picking the order of node expansion Uninformed search strategies use only the information.
Slides by: Eric Ringger, adapted from slides by Stuart Russell of UC Berkeley. CS 312: Algorithm Design & Analysis Lecture #36: Best-first State- space.
Solving problems by searching A I C h a p t e r 3.
Computing & Information Sciences Kansas State University Friday, 05 Sep 2008CIS 490 / 730: Artificial Intelligence Lecture 4 of 42 Friday, 05 September.
Computing & Information Sciences Kansas State University Friday, 01 Sep 2006CIS 490 / 730: Artificial Intelligence Lecture 5 of 42 Friday, 01 September.
Blind Search Russell and Norvig: Chapter 3, Sections 3.4 – 3.6 CS121 – Winter 2003.
Search I Tuomas Sandholm Read Russell & Norvig Sections (Also read Chapters 1 and 2 if you haven’t already.) Next time we’ll cover topics related.
For Monday Read chapter 4 exercise 1 No homework.
Biointelligence Lab School of Computer Sci. & Eng. Seoul National University Artificial Intelligence Chapter 8 Uninformed Search.
Chapter 3 Solving problems by searching. Search We will consider the problem of designing goal-based agents in observable, deterministic, discrete, known.
Artificial Intelligence Solving problems by searching.
Chapter 3.5 Heuristic Search. Learning Objectives Heuristic search strategies –Best-first search –A* algorithm Heuristic functions.
Lecture 3: Uninformed Search
Last time: Problem-Solving
Artificial Intelligence (CS 370D)
Problem Solving and Searching
EA C461 – Artificial Intelligence
Searching for Solutions
Problem Solving and Searching
Presentation transcript:

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 5 of 41 Wednesday 03 September 2003 William H. Hsu Department of Computing and Information Sciences, KSU Reading for Next Class: Handouts: #1 – Nilsson, #2 – Ginsberg Chapter 5, Russell and Norvig 2e Search Strategies

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture Outline Today’s Reading: Sections , Russell and Norvig General Search Framework: Blind (Uninformed) versus Heuristic (Informed) Today: Uninformed Search Techniques –Breadth-First Search (BFS) Basic algorithm and analysis (worst-case time complexity) Generalization: uniform-cost Bidirectional search –Depth-First Search (DFS) Basic algorithm and analysis (worst-case time complexity) Specializations: depth-limited, iterative deepening –Avoiding repeated states This Week: Informed Search Strategies –Chapter 5, R&N –Search handouts in CIS library Wednesday: Constraint Satisfaction

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Problem Solving in Agent Frameworks –Recall: IA scenario from Chapter 2, R&N –Last week: design, spec, implementation of problem-solving IA “skeleton” Cost Measures in Search –Actual –Heuristic (estimated) –Offline (computational) –Online (path) Kinds of Search –Costs used Actual Actual + heuristic Offline + online –Uninformed (blind) Exhaustive Cost-based –Informed (heuristic): minimize actual + heuristic cost – next lecture Search and Problem-Solving Agents: Review

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence General Search [1]: Overview Generating Action Sequences –Initialization: start (initial) state –Test for goal condition Membership in goal state set (explicitly enumerated) Constraints met (implicit) –Applying operators (when goal state not achieved) Implementation: generate new set of successor (child) states Conceptual process: expand state Result: multiple branches (e.g., Figure 3.8 R&N) Intuitive Idea –Select one option –Ordering (prioritizing / scheduling) remaining options for later consideration –Iteration: choose, test, expand –Termination: solution is found or no states remain to be expanded Search Strategy: Selection of State to Be Expanded

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence General Search [2]: Algorithm function General-Search (problem, strategy) returns a solution or failure –// “a solution or failure”  option (SOME of Solution | NONE) –initialize search tree using initial state of problem –loop do if there are no candidates for expansion then return failure choose leaf node for expansion according to strategy If node contains a goal state then return corresponding solution else expand node and add resulting nodes to search tree –end Note: Downward Function Argument (Funarg) strategy Implementation of General-Search –Rest of Chapter 3, Chapter 4, R&N –See also Ginsberg (handout in CIS library today) Rich and Knight Nilsson: Principles of Artificial Intelligence

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Search Strategies: Criteria Completeness –Is strategy guaranteed to find solution when one exists? –Typical assumptions (requirements) for guaranteed existence of solution Finite depth solution Finite branch factor Minimum unit cost (if paths can be infinite) – discussion: why? Time Complexity –How long does it take to find solution in worst case? –Asymptotic analysis Space Complexity –How much memory does it take to perform search in worst case? –Analysis based on data structure used to maintain frontier Optimality –Does strategy find highest-quality solution when more than one exists? –Quality: defined in terms of node depth, path cost

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Uninformed (Blind) Search Strategies Breadth-First Search (BFS) –Basic algorithm: breadth-first traversal of search tree –Intuitive idea: expand whole frontier first –Advantages: can find optimal (minimum-depth) solution for finite search spaces –Disadvantages: intractable in practice (exponential complexity, high constants) Depth-First Search (DFS) –Basic algorithm: depth-first traversal of search tree –Intuitive idea: expand one path first and backtrack –Advantages: narrow frontier –Disadvantages: lot of backtracking in worst case; suboptimal and incomplete Search Issues –“Blind” No information about number of steps or path cost from state to goal i.e., no path cost estimator function (heuristic) –Criteria: completeness (convergence); optimality; time, space complexity Uniform-Cost, Depth-Limited, Iterative Deepening, Bidirectional

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Breadth-First Search [1]: Algorithm function Breadth-First-Search (problem) returns a solution or failure –return General-Search (problem, Enqueue-At-End) function Enqueue-At-End (e: Element-Set) returns void –// Queue: priority queue data structure –while not (e.Is-Empty()) if not queue.Is-Empty() then queue.last.next  e.head(); queue.last  e.head(); e.Pop-Element(); –return Implementation Details –Recall: Enqueue-At-End downward funarg for Insert argument of General-Search –Methods of Queue (priority queue) Make-Queue (Element-Set) – constructor Is-Empty() – boolean-valued method Remove-Front() – element-valued method Insert(Element-Set) – procedure, aka Queuing-Fn

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Breadth-First Search [2]: Analysis Asymptotic Analysis: Worst-Case Time Complexity –Branching factor: b (maximum number of children per expanded node) –Solution depth (in levels from root, i.e., edge depth): d –Analysis b i nodes generated at level i At least this many nodes to test Total:  I b i = 1 + b + b 2 + … + b d =  (b d ) Worst-Case Space Complexity:  (b d ) Properties –Convergence: suppose b, d finite Complete: guaranteed to find a solution Optimal: guaranteed to find minimum-depth solution (discussion – why?) –Very poor worst-case time complexity (see Figure 3.12, R&N)

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Uniform-Cost Search [1]: A Generalization of BFS Generalizing to Blind, Cost-Based Search Justification –BFS: finds shallowest (min-depth) goal state –Not necessarily min-cost goal state for general g(n) –Want: ability to find least-cost solution Modification to BFS –Expand lowest-cost node on fringe –Requires Insert function to insert into increasing order –Alternative conceptualization: Remove-Front as Select-Next –See: Winston, Nilsson BFS: Specific Case of Uniform-Cost Search –g(n) = depth(n) –In BFS case, optimality guaranteed (discussion: why?)

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Uniform-Cost Search [2]: Example Figure 3.13, Russell and Norvig (p. 76) Requirement for Uniform-Cost Search to Find Min-Cost Solution –Monotone restriction: g(Successor(n))  g(n) + cost (n, Successor(n))  g(n) –Intuitive idea Cost increases monotonically with search depth (distance from root) i.e., nonnegative edge costs –Discussion Always realistic, i.e., can always be expected in real-world situations? What happens if monotone restriction is violated?

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Depth-First Search [1]: Algorithm function Depth-First-Search (problem) returns a solution or failure –return General-Search (problem, Enqueue-At-Front) function Enqueue-At-Front (e: Element-Set) returns void –// Queue: priority queue data structure –while not (e.Is-Empty()) temp  queue.first; queue.first  e.head(); queue.first.next  temp; e.Pop-Element(); –return Implementation Details –Enqueue-At-Front downward funarg for Insert argument of General-Search –Otherwise similar in implementation to BFS –Exercise (easy) Recursive implementation See Cormen, Leiserson, and Rivest (1990) – Introduction to Algorithms

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Depth-First Search [2]: Analysis Asymptotic Analysis: Worst-Case Time Complexity –Branching factor: b (maximum number of children per expanded node) –Max depth (in levels from root, i.e., edge depth): m –Analysis b i nodes generated at level i At least this many nodes to test Total:  I b i = 1 + b + b 2 + … + b m =  (b m ) Worst-Case Space Complexity:  (bm) – Why? Example: Figure 3.14, R&N Properties –Convergence: suppose b, m finite Not complete: not guaranteed to find a solution (discussion – why?) Not optimal: not guaranteed to find minimum-depth solution –Poor worst-case time complexity

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Depth-Limited Search: A Bounded Specialization of DFS Intuitive Idea –Impose cutoff on maximum depth of path –Search no further in tree Analysis –Max search depth (in levels from root, i.e., edge depth): l –Analysis b i nodes generated at level i At least this many nodes to test Total:  I b i = 1 + b + b 2 + … + b l =  (b l ) Worst-Case Space Complexity:  (bl) Properties –Convergence: suppose b, l finite and l  d Complete: guaranteed to find a solution Not optimal: not guaranteed to find minimum-depth solution –Worst-case time complexity depends on l, actual solution depth d

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Iterative Deepening Search: An Incremental Specialization of DFS Intuitive Idea –Search incrementally –Anytime algorithm: return value on demand Analysis –Solution depth (in levels from root, i.e., edge depth): d –Analysis b i nodes generated at level i At least this many nodes to test Total:  I b i = 1 + b + b 2 + … + b d =  (b d ) Worst-Case Space Complexity:  (bd) Properties –Convergence: suppose b, l finite and l  d Complete: guaranteed to find a solution Optimal: guaranteed to find minimum-depth solution (discussion – why?)

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Bidirectional Search: A Concurrent Variant of BFS Intuitive Idea –Search “from both ends” –Caveat: what does it mean to “search backwards from solution”? Analysis –Solution depth (in levels from root, i.e., edge depth): d –Analysis b i nodes generated at level i At least this many nodes to test Total:  I b i = 1 + b + b 2 + … + b d/2 =  (b d/2 ) Worst-Case Space Complexity:  (b d/2 ) Properties –Convergence: suppose b, l finite and l  d Complete: guaranteed to find a solution Optimal: guaranteed to find minimum-depth solution –Worst-case time complexity is square root of that of BFS

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Comparison of Search Strategies Asymptotic Analysis: Worst-Case Time Complexity –Branching factor: b (maximum number of children per expanded node) –Solution depth (in levels from root, i.e., edge depth): d –Analysis b i nodes generated at level i At least this many nodes to test Total:  I b i = 1 + b + b 2 + … + b d =  (b d ) Worst-Case Space Complexity:  (b d ) Properties –Convergence: suppose b, d finite Complete: guaranteed to find a solution Optimal: guaranteed to find minimum-depth solution (discussion – why?) –Very poor worst-case time complexity (see Figure 3.12, R&N)

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Previously: Uninformed (Blind) Search –No heuristics: only g(n) used –Breadth-first search (BFS) and variants: uniform-cost, bidirectional –Depth-first search (DFS) and variants: depth-limited, iterative deepening Heuristic Search –Based on h(n) – estimated cost of path to goal (“remaining path cost”) h – heuristic function g: node  R; h: node  R; f: node  R –Using h h only: greedy (aka myopic) informed search f = g + h: (some) hill-climbing, A/A* Branch and Bound Search –Originates from operations research (OR) –Special case of heuristic search: treat as h(n) = 0, sort candidates by g(n) Informed (Heuristic) Search: Overview

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Best-First Search [1]: Characterization of Algorithm Family Evaluation Function –Recall: General-Search (Figure 3.9, 3.10 R&N) –Applying knowledge In problem representation (state space specification) At Insert(), aka Queueing-Fn() – determines node to expand next –Knowledge representation (KR): expressing knowledge symbolically/numerically Objective; initial state, state space (operators, successor function), goal test h(n) – part of (heuristic) evaluation function Best-First: Family of Algorithms –Justification: using only g doesn’t direct search toward goal –Nodes ordered so that node with best evaluation function (e.g., h) expanded first –Best-first: any algorithm with this property (NB: not just using h alone) Note on “Best” –Refers to “apparent best node based on eval function applied to current frontier” –Discussion: when is best-first not really best?

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Best-First Search [2]: Implementation function Best-First-Search (problem, Eval-Fn) returns solution sequence –inputs:problem, specification of problem (structure or class) Eval-Fn, an evaluation function –Queueing-Fn  function that orders nodes by Eval-Fn Compare: Sort with comparator function < Functional abstraction –return General-Search (problem, Queueing-Fn) Implementation –Recall: priority queue specification Eval-Fn: node  R Queueing-Fn  Sort-By: node list  node list –Rest of design follows General-Search Issues –General family of greedy (aka myopic, i.e., nearsighted) algorithms –Discussion: What guarantees do we want on h(n)? What preferences?

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Heuristic Search [1]: Terminology Heuristic Function –Definition: h(n) = estimated cost of cheapest path from state at node n to a goal state –Requirements for h In general, any magnitude (ordered measure, admits comparison) h(n) = 0 iff n is goal –For A/A*, iterative improvement: want h to have same type as g Return type to admit addition –Problem-specific (domain-specific) Typical Heuristics –Graph search in Euclidean space: h SLD (n) = straight-line distance to goal –Discussion (important): Why is this good?

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Today’s Reading: Sections 4.1 – 4.2, Russell and Norvig Reading for Next Lecture: Sections 4.3 – 4.5, Russell and Norvig Branch-and-Bound Search Heuristics for General-Search Function of Problem-Solving-Agent –Informed (heuristic) search: heuristic definition and development process –Best-First Search Greedy A/A* Admissibility property –Developing good heuristics Humans Intelligent systems (automatic derivation): case studies and principles Constraint Satisfaction Heuristics Thursday, Next Tuesday: More Search –Memory bounded, iterative improvement (gradient, Monte Carlo search) –Introduction to game tree search Next Class: More Informed (Heuristic) Search

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Terminology Search –Search problems –State space search –Approaches Blind (uninformed) Heuristic (informed) Applications –Problem solving Optimization Scheduling Design –Machine learning (hypothesis space search)

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Summary Points Search –Problem formulation: state space (initial / operator / goal test / cost), graph –State space search approaches Blind (uninformed) Heuristic (informed) Applications –Problem solving Optimization Scheduling Design –Machine learning (hypothesis space search) More Resources Online – –See also (“REFERENCES”) Course Project Guidelines to Be Posted Today (Mon 30 Aug 2004) –Write to cis730ta with your questions –Send your team selection!