Nature’s Algorithms David C. Uhrig Tiffany Sharrard CS 477R – Fall 2007 Dr. George Bebis.

Slides:



Advertisements
Similar presentations
Decision Support Andry Pinto Hugo Alves Inês Domingues Luís Rocha Susana Cruz.
Advertisements

Reactive and Potential Field Planners
Local Search Algorithms
CS6800 Advanced Theory of Computation
Simulated Annealing Premchand Akella. Agenda Motivation The algorithm Its applications Examples Conclusion.
Reducibility Class of problems A can be reduced to the class of problems B Take any instance of problem A Show how you can construct an instance of problem.
CHAPTER 8 A NNEALING- T YPE A LGORITHMS Organization of chapter in ISSO –Introduction to simulated annealing –Simulated annealing algorithm Basic algorithm.
Introducing Natural Algorithms. A passionate affair Nature and Numbers – Friends for ever! I started it !
Algorithm Strategies Nelson Padua-Perez Chau-Wen Tseng Department of Computer Science University of Maryland, College Park.
MAE 552 – Heuristic Optimization Lecture 27 April 3, 2002
Iterative Improvement Algorithms
Imagine that I am in a good mood Imagine that I am going to give you some money ! In particular I am going to give you z dollars, after you tell me the.
Review Best-first search uses an evaluation function f(n) to select the next node for expansion. Greedy best-first search uses f(n) = h(n). Greedy best.
1 Simulated Annealing Terrance O ’ Regan. 2 Outline Motivation The algorithm Its applications Examples Conclusion.
Simulated Annealing Van Laarhoven, Aarts Version 1, October 2000.
Evolutionary Computation Application Peter Andras peter.andras/lectures.
D Nagesh Kumar, IIScOptimization Methods: M1L4 1 Introduction and Basic Concepts Classical and Advanced Techniques for Optimization.
Artificial Intelligence in Information Processing Genetic Algorithms by Theresa Kriese for Distributed Data Processing.
Optimization via Search CPSC 315 – Programming Studio Spring 2008 Project 2, Lecture 4 Adapted from slides of Yoonsuck Choe.
Simulated Annealing G.Anuradha. What is it? Simulated Annealing is a stochastic optimization method that derives its name from the annealing process used.
Ch. 11: Optimization and Search Stephen Marsland, Machine Learning: An Algorithmic Perspective. CRC 2009 some slides from Stephen Marsland, some images.
Ant Colony Optimization: an introduction
Metaheuristics The idea: search the solution space directly. No math models, only a set of algorithmic steps, iterative method. Find a feasible solution.
Optimization of thermal processes2007/2008 Optimization of thermal processes Maciej Marek Czestochowa University of Technology Institute of Thermal Machinery.
Genetic Algorithms and Ant Colony Optimisation
1 Local search and optimization Local search= use single current state and move to neighboring states. Advantages: –Use very little memory –Find often.
Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory Mixed Integer Problems Most optimization algorithms deal.
An Introduction to Artificial Life Lecture 4b: Informed Search and Exploration Ramin Halavati In which we see how information.
Boltzmann Machine (BM) (§6.4) Hopfield model + hidden nodes + simulated annealing BM Architecture –a set of visible nodes: nodes can be accessed from outside.
Researchers: Preet Bola Mike Earnest Kevin Varela-O’Hara Han Zou Advisor: Walter Rusin Data Storage Networks.
Dr.Abeer Mahmoud ARTIFICIAL INTELLIGENCE (CS 461D) Dr. Abeer Mahmoud Computer science Department Princess Nora University Faculty of Computer & Information.
Theory of Algorithms: Brute Force. Outline Examples Brute-Force String Matching Closest-Pair Convex-Hull Exhaustive Search brute-force strengths and weaknesses.
Simulated Annealing.
Thursday, May 9 Heuristic Search: methods for solving difficult optimization problems Handouts: Lecture Notes See the introduction to the paper.
Applications of Dynamic Programming and Heuristics to the Traveling Salesman Problem ERIC SALMON & JOSEPH SEWELL.
Exact and heuristics algorithms
Artificial Intelligence for Games Online and local search
1 Genetic Algorithms and Ant Colony Optimisation.
Genetic Algorithms Przemyslaw Pawluk CSE 6111 Advanced Algorithm Design and Analysis
Simulated Annealing G.Anuradha.
Introduction to Genetic Algorithms. Genetic Algorithms We’ve covered enough material that we can write programs that use genetic algorithms! –More advanced.
Genetic Algorithms. 2 Overview Introduction To Genetic Algorithms (GAs) GA Operators and Parameters Genetic Algorithms To Solve The Traveling Salesman.
Probabilistic Algorithms Evolutionary Algorithms Simulated Annealing.
Optimization Problems
Ramakrishna Lecture#2 CAD for VLSI Ramakrishna
Lecture 7 Jianjun Hu Department of Computer Science and Engineering University of South Carolina CSCE350 Algorithms and Data Structure.
Local Search. Systematic versus local search u Systematic search  Breadth-first, depth-first, IDDFS, A*, IDA*, etc  Keep one or more paths in memory.
Chapter 5. Advanced Search Fall 2011 Comp3710 Artificial Intelligence Computing Science Thompson Rivers University.
Genetic Algorithms. Underlying Concept  Charles Darwin outlined the principle of natural selection.  Natural Selection is the process by which evolution.
Intro. ANN & Fuzzy Systems Lecture 37 Genetic and Random Search Algorithms (2)
Chapter 3 Brute Force Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Genetic Algorithms An Evolutionary Approach to Problem Solving.
Genetic Algorithms And other approaches for similar applications Optimization Techniques.
Constraints Satisfaction Edmondo Trentin, DIISM. Constraint Satisfaction Problems: Local Search In many optimization problems, the path to the goal is.
Optimization Problems
Genetic Algorithms.
CSCI 4310 Lecture 10: Local Search Algorithms
Simulated Annealing Chapter
School of Computer Science & Engineering
Chapter 3 Brute Force Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Artificial Intelligence (CS 370D)
Subject Name: Operation Research Subject Code: 10CS661 Prepared By:Mrs
Optimization Problems
Metaheuristic methods and their applications. Optimization Problems Strategies for Solving NP-hard Optimization Problems What is a Metaheuristic Method?
3. Brute Force Selection sort Brute-Force string matching
More on Search: A* and Optimization
Xin-She Yang, Nature-Inspired Optimization Algorithms, Elsevier, 2014
3. Brute Force Selection sort Brute-Force string matching
Greg Knowles ECE Fall 2004 Professor Yu Hu Hen
3. Brute Force Selection sort Brute-Force string matching
Presentation transcript:

Nature’s Algorithms David C. Uhrig Tiffany Sharrard CS 477R – Fall 2007 Dr. George Bebis

What are “Nature’s Algorithms”? Combinatorial Optimization Problems Combinatorial Optimization Problems No easy way to examine all combinations No easy way to examine all combinations Use algorithms based on scientific principles, nature, and genetics Use algorithms based on scientific principles, nature, and genetics This does not give optimal solutions, but is more practical than an exhaustive search This does not give optimal solutions, but is more practical than an exhaustive search

Example: Traveling Salesman The Traveling Salesman problem is one of the most famous combinatorial optimization problems: The Traveling Salesman problem is one of the most famous combinatorial optimization problems: Given a set of cities (nodes), find the shortest path required to visit all cities exactly once. Given a set of cities (nodes), find the shortest path required to visit all cities exactly once.

Example: Traveling Salesman There are (n-1)!/2 possible tours for n cities. There are (n-1)!/2 possible tours for n cities tours for 8 cities 2520 tours for 8 cities 653,837,184,000 tours for 16 cities 653,837,184,000 tours for 16 cities Would require a computer nearly 8 days to compute them all! (for 1/1,000,000 sec per tour generation) Would require a computer nearly 8 days to compute them all! (for 1/1,000,000 sec per tour generation) Not practical to compute all tours… Not practical to compute all tours…

Example: Location Allocation Given a set of facilities, each of which can serve a certain number of nodes on a graph, place the facilities in such a way that the distance between each node and the facility is minimized. Given a set of facilities, each of which can serve a certain number of nodes on a graph, place the facilities in such a way that the distance between each node and the facility is minimized. Police stations and population densities Police stations and population densities Schools and student populations Schools and student populations

Example: Location Allocation

Issues with Combinatorial Optimization Problems Large set of discrete solutions Large set of discrete solutions Number of solutions grows exponentially Number of solutions grows exponentially In the Travelling Salesman problem, the solution size grows by n*(n-1)! for each city that is added. In the Travelling Salesman problem, the solution size grows by n*(n-1)! for each city that is added. In the Location Allocation problem, the solution size is affected by both the number of nodes but also the connections between nodes and the number of (and size of) facilities. In the Location Allocation problem, the solution size is affected by both the number of nodes but also the connections between nodes and the number of (and size of) facilities.

Evolution and the Genetic Algorithm Mimics the process of evolution in developing a solution. Mimics the process of evolution in developing a solution. Begins with a large set of solutions to the problem in a candidate pool. Begins with a large set of solutions to the problem in a candidate pool. Each of these algorithms is tested for ‘fitness’ Each of these algorithms is tested for ‘fitness’

Evolution and the Genetic Algorithm After the fitnesses are calculated, a new set of solutions is randomly picked from the ‘most fit’ algorithms. After the fitnesses are calculated, a new set of solutions is randomly picked from the ‘most fit’ algorithms. Mimics Natural Selection Mimics Natural Selection Next, two algorithms are paired together and swap pieces of each solutions Next, two algorithms are paired together and swap pieces of each solutions Then each algorithm changes (mutates) slightly Then each algorithm changes (mutates) slightly

Evolution and the Genetic Algorithm This pattern of evaluation, reproduction, swapping, and mutating mimics the way DNA is processed during sexual reproduction. This pattern of evaluation, reproduction, swapping, and mutating mimics the way DNA is processed during sexual reproduction. This process continues until a satisfactory solution is much better than the rest of the population of solutions. This process continues until a satisfactory solution is much better than the rest of the population of solutions.

Metallurgy and the Simulated Annealing Problem In metallurgy, when molten metal cools too quickly it can lose certain properties, such as strength or magnetism. In metallurgy, when molten metal cools too quickly it can lose certain properties, such as strength or magnetism. Annealing is the process of slowly cooling, which can help retain or amplify desired traits. Annealing is the process of slowly cooling, which can help retain or amplify desired traits. In Computer Science, Simulated Annealing Algorithms are usually an improvement on “Greedy Local Search” algorithms. In Computer Science, Simulated Annealing Algorithms are usually an improvement on “Greedy Local Search” algorithms.

Metallurgy and the Simulated Annealing Problem 1. Start with a proposed solution. 2. Change the solution slightly i.e. Swap two cities in the traveling salesman problem. i.e. Swap two cities in the traveling salesman problem. 3. Check to see if the new solution is better than the original. If not, undo the change. 4. Repeat steps 2 and 3 until it is apparent that any small change makes the solution worse.

Metallurgy and the Simulated Annealing Problem A greedy local search will only find Locally Optimal Solutions. A greedy local search will only find Locally Optimal Solutions. This also occurs with Genetic Algorithms when mutations stop producing positive changes. This also occurs with Genetic Algorithms when mutations stop producing positive changes.

Metallurgy and the Simulated Annealing Problem Try to improve on Greedy Local Searches by taking occasional risks by accepting a worse solution. Try to improve on Greedy Local Searches by taking occasional risks by accepting a worse solution. Compute ΔC, the difference between the cost of the current solution and the proposed solution. Then: for Random(r), 0 < r < 1, if r < e -ΔC/T Where T is the metaphoric “temperature” that is iteratively decreased until an acceptable temperature is reached. Compute ΔC, the difference between the cost of the current solution and the proposed solution. Then: for Random(r), 0 < r < 1, if r < e -ΔC/T Where T is the metaphoric “temperature” that is iteratively decreased until an acceptable temperature is reached.

Metallurgy and the Simulated Annealing Problem Theoretically, the temperature should be lowered proportionally to 1/log(n), but this rate is too slow. Theoretically, the temperature should be lowered proportionally to 1/log(n), but this rate is too slow. Realistically, the temperature is lowered at a linear or exponential rate. Realistically, the temperature is lowered at a linear or exponential rate. Even with this decreased cooling time, simulated annealing has proven to be very effective in finding a balance between exploring every solution and avoiding locally optimal results. Even with this decreased cooling time, simulated annealing has proven to be very effective in finding a balance between exploring every solution and avoiding locally optimal results.

Economics The idea of “commodities” and “consumers” competing for them The idea of “commodities” and “consumers” competing for them Prices are determined by competition Prices are determined by competition Consumers buy what they can afford Consumers buy what they can afford Prices are influenced by consumers Prices are influenced by consumers

Computer Networks Consumer = Process Consumer = Process Competing commodities: processor time, access to network printer, etc Competing commodities: processor time, access to network printer, etc

Example Two terminals want to share two database files Two terminals want to share two database files The files will run slower if they are accessed from the other location than from the local location The files will run slower if they are accessed from the other location than from the local location Decision should be made based on how much of each file should be stored at each terminal Decision should be made based on how much of each file should be stored at each terminal

Solution Give each file a starting price Give each file a starting price Give each terminal a fixed amount of money Give each terminal a fixed amount of money With constraints of the amount of money, terminals ask for a certain portion of each file With constraints of the amount of money, terminals ask for a certain portion of each file Price changes based on the proportion of how much of the files were requested Price changes based on the proportion of how much of the files were requested Done until equilibrium is reached Done until equilibrium is reached

Simulated N-Body Algorithm Study of moving particles in physics Study of moving particles in physics Particles are allowed to move around the graph Particles are allowed to move around the graph To minimize potential energy in the system, particles tend to spread out To minimize potential energy in the system, particles tend to spread out A “boundary” is formed to keep the particles from being pushed to the outer edge A “boundary” is formed to keep the particles from being pushed to the outer edge

Aggregation and Decomposition Every particle is grouped into a larger particle with combined mass Every particle is grouped into a larger particle with combined mass Smallest particle is removed Smallest particle is removed Small and the largest particle moved to where the potential energy is the lowest Small and the largest particle moved to where the potential energy is the lowest Take next smallest particle and do the same Take next smallest particle and do the same Result: reasonably good configuration Result: reasonably good configuration

Particle Physics Simulation used to solve traveling salesman problems Simulation used to solve traveling salesman problems Use of charged particles connected by electric bands Use of charged particles connected by electric bands

How Does It Work? Number of Cities = Number of negatively- charged particles = Number of positively-charged particles Number of Cities = Number of negatively- charged particles = Number of positively-charged particles Positive particles are placed in a circle Positive particles are placed in a circle Solution: Solution: Particles are set free Particles are set free When particle hits a city-stops When particle hits a city-stops When all have stopped- elastic string traces a path through each city When all have stopped- elastic string traces a path through each city