Search Methods An Annotated Overview Edward Tsang.

Slides:



Advertisements
Similar presentations
Algorithm Design Methods (I) Fall 2003 CSE, POSTECH.
Advertisements

G5BAIM Artificial Intelligence Methods
Artificial Intelligence Presentation
Neural and Evolutionary Computing - Lecture 4 1 Random Search Algorithms. Simulated Annealing Motivation Simple Random Search Algorithms Simulated Annealing.
Multi-Objective Optimization NP-Hard Conflicting objectives – Flow shop with both minimum makespan and tardiness objective – TSP problem with minimum distance,
5-1 Chapter 5 Tree Searching Strategies. 5-2 Satisfiability problem Tree representation of 8 assignments. If there are n variables x 1, x 2, …,x n, then.
Modeling Rich Vehicle Routing Problems TIEJ601 Postgraduate Seminar Tuukka Puranen October 19 th 2009.
Heuristic Optimization Athens 2004 Department of Architecture and Technology Universidad Politécnica de Madrid Víctor Robles
Better Ants, Better Life? Hybridization of Constraint Propagation and Ant Colony Optimization Supervisors: Bernd Meyer, Andreas Ernst Martin Held Jun 2nd,
Unifying Local and Exhaustive Search John Hooker Carnegie Mellon University September 2005.
Models Physical: Scale, Analog Symbolic: Drawings Computer Programs Mathematical: Analytical (Deduction) Experimental (Induction)
Introduction to Artificial Intelligence Local Search (updated 4/30/2006) Henry Kautz.
Vehicle Routing & Scheduling
EAs for Combinatorial Optimization Problems BLG 602E.
Ant Colony Optimization Optimisation Methods. Overview.
Branch and Bound Algorithm for Solving Integer Linear Programming
Genetic Algorithms Nehaya Tayseer 1.Introduction What is a Genetic algorithm? A search technique used in computer science to find approximate solutions.
D Nagesh Kumar, IIScOptimization Methods: M1L4 1 Introduction and Basic Concepts Classical and Advanced Techniques for Optimization.
1 Integrality constraints Integrality constraints are often crucial when modeling optimizayion problems as linear programs. We have seen that if our linear.
1 IE 607 Heuristic Optimization Introduction to Optimization.
Ant Colony Optimization: an introduction
Metaheuristics The idea: search the solution space directly. No math models, only a set of algorithmic steps, iterative method. Find a feasible solution.
Software framework for metaheuristics Parallel Cooperative Optimization Research Group Laboratoire d’Informatique Fondamentale de Lille
Informed Search Methods
Lecture: 5 Optimization Methods & Heuristic Strategies Ajmal Muhammad, Robert Forchheimer Information Coding Group ISY Department.
Copyright R. Weber Search in Problem Solving Search in Problem Solving INFO 629 Dr. R. Weber.
Genetic Algorithms and Ant Colony Optimisation
Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti
15 July 2005Edward Tsang (Copyright)1 Constraint Satisfaction and Optimization Professor Edward Tsang University of Essex URL:
Local Search: walksat, ant colonies, and genetic algorithms.
CP Summer School Modelling for Constraint Programming Barbara Smith 2. Implied Constraints, Optimization, Dominance Rules.
Thursday, May 9 Heuristic Search: methods for solving difficult optimization problems Handouts: Lecture Notes See the introduction to the paper.
Mobile Agent Migration Problem Yingyue Xu. Energy efficiency requirement of sensor networks Mobile agent computing paradigm Data fusion, distributed processing.
Applications of Dynamic Programming and Heuristics to the Traveling Salesman Problem ERIC SALMON & JOSEPH SEWELL.
Exact and heuristics algorithms
CSE 589 Part VI. Reading Skiena, Sections 5.5 and 6.8 CLR, chapter 37.
1 Genetic Algorithms and Ant Colony Optimisation.
Genetic Algorithms Przemyslaw Pawluk CSE 6111 Advanced Algorithm Design and Analysis
Traveling Salesman Problem IEOR 4405 Production Scheduling Professor Stein Sally Kim James Tsai April 30, 2009.
Introduction to Optimization
Local Search and Optimization Presented by Collin Kanaley.
Optimization Problems
Thursday, 07 January 2016Edward Tsang (Copyright)1 Constraint Satisfaction for Decision Making Professor Edward Tsang University of Essex URL:
Computation: Potentials and Limitations Edward Tsang.
Biologically Inspired Computation Ant Colony Optimisation.
IE 312 Review 1. The Process 2 Problem Model Conclusions Problem Formulation Analysis.
Intro. ANN & Fuzzy Systems Lecture 37 Genetic and Random Search Algorithms (2)
Escaping Local Optima. Where are we? Optimization methods Complete solutions Partial solutions Exhaustive search Hill climbing Exhaustive search Hill.
Genetic Algorithms An Evolutionary Approach to Problem Solving.
School of Computer Science & Engineering
CSCI 4310 Lecture 10: Local Search Algorithms
Integer Programming An integer linear program (ILP) is defined exactly as a linear program except that values of variables in a feasible solution have.
Digital Optimization Martynas Vaidelys.
Local Search Algorithms
Computer Science cpsc322, Lecture 14
Artificial Intelligence (CS 370D)
Subject Name: Operation Research Subject Code: 10CS661 Prepared By:Mrs
Comparing Genetic Algorithm and Guided Local Search Methods
metaheuristic methods and their applications
Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Nov. 7, 2000
Metaheuristic methods and their applications. Optimization Problems Strategies for Solving NP-hard Optimization Problems What is a Metaheuristic Method?
Multi-Objective Optimization
Subset of Slides from Lei Li, HongRui Liu, Roberto Lu
Guided Local Search – CP Meets OR
Design & Analysis of Algorithms Combinatorial optimization
Artificial Intelligence
EMIS 8373: Integer Programming
Local Search Algorithms
Local Search Algorithms
Simulated Annealing & Boltzmann Machines
Presentation transcript:

Search Methods An Annotated Overview Edward Tsang

Search Methods – Summary Systematic Search  If solutions exist, they will be found  Specialized methods –E.g. Linear ProgrammingLinear Programming  Branch and Bound (originated from OR) Branch and Bound  A* (started in AI)  Constraint satisfaction Constraint satisfaction Stochastic Search  Sacrifice completeness  Generate and Test –Local Search, e.g. Hill Climbing Simulated Annealing Tabu Search Guided Local Search –Artificial Neural Networks –Evolutionary computationEvolutionary computation GA / GP / Ants Colony / … EDA – statistical basedEDA 16/10/2015All Rights Reserved, Edward Tsang Solution Space & Search Space

Linear Programming (LP)  To maximize/minimize an objective function  Subject to a set of linear inequalities  Example: to maximize x + y  Subject to: 2x + y ≤ 10 x + 2y ≤ 12 3x  y ≤ 9 16 October 2015All Rights Reserved, Edward Tsang

Linear Programming Example  Maximize x + y  Subject to: 2x + y ≤ 10 x + 2y ≤ 12 3x  y ≤ 9  Solution: x = y = x + y = October 2015All Rights Reserved, Edward Tsang 510 5

Simplex Method for LP  Start with any feasible solution  Craw along any edge to improve according to objective function  Repeat until no improvement is available 16 October 2015All Rights Reserved, Edward Tsang 510 5

Portfolio investment decision ProjectInvestmentReturn 1$5,000$8,000 2$7,000$11,000 3$4,000$6,000 4$3,000$4,000  Suppose you have $14K to invest  Make decision variables: x 1, x 2, x 3, x 4  {0, 1} x i =1 means invest in i; x i = 0 means don’t invest in i  Maximize R = 8x x 2 + 6x 3 + 4x 4  Subject to 5x 1 + 7x 2 + 4x 3 + 3x 4 ≤ October 2015All Rights Reserved, Edward Tsang Source: Michael Trick, Tutorial, 1998

Integer Programming  Maximize R = 8x x 2 + 6x 3 + 4x 4  Subject to 5x 1 + 7x 2 + 4x 3 + 3x 4 ≤ 14 –Where x 1, x 2, x 3, x 4  {0, 1}  Without integer constraint x 1 =1, x 2 =1, x 3 =0.5, x 4 =0, R = $22,000  If we round x 3 down to 0, R = $19,000  Better result: x 1 =0, x 2 =x 3 =x 4 =1, R = $21,000  Integer constraint leads to harder problem 16 October 2015All Rights Reserved, Edward Tsang

Integer Programming Example  Maximize x + y  Subject to: 2x + y ≤ 10 x + 2y ≤ 12 3x  y ≤ 9  Not a solution: x = 2.6, y = 4.7  Solutions x = 2, y = 5; x = 3, y = 4 16 October 2015All Rights Reserved, Edward Tsang 510 5

Friday, 16 October 2015Edward Tsang (Copyright)9 Travelling Salesman Problem (TSP)  Goal: to find shortest route through all cities  Optimization involved: minimization A (4,10) B (0,5) D (6,6) E (10,6) C (6,3)

Friday, 16 October 2015Edward Tsang (Copyright)10 Distance Table for an example TSP ABCDE A B C D E Heuristic:

Friday, 16 October 2015Edward Tsang (Copyright)11 Branch & Bound (1) A c=0 h=20 AB c=6 h=14 ABC c=12 h=11 ABCD c=15 h=8 ABCDEA c=26 h=0 ABCEDA c=25 h=0 ABCE c=17 h=7 ABD c=12 h=11 ABDC c=15 h=8ABDE c=16 h=7 ABDCEA c=27 h=0 ABDECA c=28 h=0 ABE c=16 h=10 AC, AD, AE to be searched Pruned c=cost h=lower bound Heuristics TSP

Friday, 16 October 2015Edward Tsang (Copyright)12 Branch & Bound (2) A c=0 h=20 AD c=4 h=17 ADC c=7 h=10 ADCE c=12 h=10 ADCEBA c=28 h=0 ADCBEA c=30 h=0 ADCB c=13 h=8 ADE c=8 h=13 ADEC c=13 h=10ADEB c=18 h=7 ADCBEA c=30 h=0 ADCBEA c=30 h=0 ADB c=10 h=11 AB, AC, AE to be searched …. c=cost h=lower bound Heuristics TSP

Friday, 16 October 2015Edward Tsang (Copyright)13 HC Example: 2-opting for TSP  Candidate tour: a round trip route  Neighbour: exchange two edges, change directions accordingly A B C E D A B C E D

Friday, 16 October 2015Edward Tsang (Copyright) 14 List reversing  2-Opting  List representation: –A list could represent cities in sequence  2-Opting can be seen as sub-list reversing –Easy to implement Breaking points

15 July 2005Edward Tsang (Copyright) 15 Local Search  Ingredients: –Cost function –Neighbourhood function –[Optional] Strategy for visiting neighbours e.g. steepest ascent  Problems: –local optimum –Plateau –When to stop? Ok with satisfiability But not optimization local max. global max. Cost function plateau neighbourhood better worse

15 July 2005Edward Tsang (Copyright)16 GLS: Augmented Cost Function  Identifying solution features, e.g. Edges used  associate costs and penalties to features  Given cost function g to minimize  Augmented Cost Function h(s) = g (s) +  ×  p i × I i (s)) is parameter to GLS I i (s) = 1 if s exhibits feature i; 0 otherwise p i is penalty for feature i, initialized to 0

15 July 2005Edward Tsang (Copyright)17 GLS on TSP  Local search: 2-opting  = a  g ( t* ) / n Features:  n 2 Features  cost = distance given  e.g. tour [1,5,3,4,6,2] t* = first local minimum produced by local search; g(t*) = cost of t* n = # of cities a = parameter to tune, within (0, 1]

16 October, 2015Copyright Edward Tsang18 Population-based Incremental Learning (PBIL)  Statistical approach –Related to ant-colonies, GA  General form is EDA (Estimation of Distribution Algorithms) –Build Bayesian Nets (probability dependency) Model M: x = v1 (0.5) x = v2 (0.5) y = v3 (0.5) y = v4 (0.5) Sample from M solution X, eg Evaluation X Modify the probabilities

Searching Artificial Intelligence  Knowledge representation + Search Search Space = the set of all possible solutions under the given representation 16 October 2015All Rights Reserved, Edward Tsang Complete Search Systematically explore every candidate solution in the search space Incomplete Search Use heuristics to search in promising areas for solutions

Stochastic Search  Incomplete search –i.e. even if solutions exist, they may not be found  Evolutionary computation –To evolve solutions thru maintaining a population  Hill Climbing –To heuristically improve on the current solution  Many more –Tabu search, guided local search, neural network, … 16 October 2015All Rights Reserved, Edward Tsang

Friday, 16 October 2015Edward Tsang (Copyright)21 Car Sequencing Problem Options ABS CD …   Total: 120 Production: ABS area:  3/5 CD area:  2/3

Friday, 16 October 2015Edward Tsang (Copyright)22 Combinatorial Explosion in Car Sequencing  Schedule 30 cars: –Search space: 30 factorial  leaf nodes  Generously allow: –Explore one in every leaf nodes! –Examine nodes per second!  Problem takes over 32 thousand years to solve!!! –10 32 ÷ ÷ ÷ 60 ÷ 60 ÷ 24 ÷ 365  31,710  How to contain combinatorial explosion?

Solution Space  Suppose you have 2 variables, x and y  Then you have a 2-dimensional solution space2-dimensional solution space  If the variables can take any real number value, then your solution space is continuous  If x and y can only take integer variables, then your solution space is discreteinteger variables  If you have n variables, then you have an n- dimensional space 16/10/2015All Rights Reserved, Edward Tsang

Search Space  Sometimes interchanged with “solution space”solution space  The search method defines the paths that one can take in the solution space  For example, the search space in branch and bound is a treeexample  Good knowledge of the search space can sometimes help solving the problem –E.g. TSP, “2-opting”, “basins of attraction”TSP2-optingbasins of attraction 16/10/2015All Rights Reserved, Edward Tsang

Conclusion  Problems are only hard if they have an exponential solution/search space  Use specialized methods when available!  Combinatorial Explosion haunts systematic search  Stochastic Search is more practical  There are many search methods  Given a problem, which method to use?  Knowing your search space helps! 16 October 2015All Rights Reserved, Edward Tsang