Evolution strategies Can programs learn?

Slides:



Advertisements
Similar presentations
G5BAIM Artificial Intelligence Methods Graham Kendall Evolutionary Algorithms.
Advertisements

Ali Husseinzadeh Kashan Spring 2010
Biologically Inspired Computing: Operators for Evolutionary Algorithms
Rene Plowden Joseph Libby. Improving the profit margin by optimizing the win ratio through the use of various strategies and algorithmic computations.
CHAPTER 9 E VOLUTIONARY C OMPUTATION I : G ENETIC A LGORITHMS Organization of chapter in ISSO –Introduction and history –Coding of  –Standard GA operations.
Intro to AI Genetic Algorithm Ruth Bergman Fall 2002.
Tutorial 1 Temi avanzati di Intelligenza Artificiale - Lecture 3 Prof. Vincenzo Cutello Department of Mathematics and Computer Science University of Catania.
Evolutionary Computational Intelligence Lecture 8: Memetic Algorithms Ferrante Neri University of Jyväskylä.
CS 447 Advanced Topics in Artificial Intelligence Fall 2002.
Intro to AI Genetic Algorithm Ruth Bergman Fall 2004.
Differential Evolution Hossein Talebi Hassan Nikoo 1.
Genetic Algorithms Overview Genetic Algorithms: a gentle introduction –What are GAs –How do they work/ Why? –Critical issues Use in Data Mining –GAs.
Prepared by Barış GÖKÇE 1.  Search Methods  Evolutionary Algorithms (EA)  Characteristics of EAs  Genetic Programming (GP)  Evolutionary Programming.
Introduction to AI Michael J. Watts
Introduction to Genetic Algorithms and Evolutionary Computation
Evolution Strategies Evolutionary Programming Genetic Programming Michael J. Watts
Intro. ANN & Fuzzy Systems Lecture 36 GENETIC ALGORITHM (1)
Schemata Theory Chapter 11. A.E. Eiben and J.E. Smith, Introduction to Evolutionary Computing Theory Why Bother with Theory? Might provide performance.
CS440 Computer Science Seminar Introduction to Evolutionary Computing.
Boltzmann Machine (BM) (§6.4) Hopfield model + hidden nodes + simulated annealing BM Architecture –a set of visible nodes: nodes can be accessed from outside.
Neural and Evolutionary Computing - Lecture 6
Computational Complexity Jang, HaYoung BioIntelligence Lab.
Genetic Algorithms Siddhartha K. Shakya School of Computing. The Robert Gordon University Aberdeen, UK
GENETIC ALGORITHM A biologically inspired model of intelligence and the principles of biological evolution are applied to find solutions to difficult problems.
EE459 I ntroduction to Artificial I ntelligence Genetic Algorithms Kasin Prakobwaitayakit Department of Electrical Engineering Chiangmai University.
G5BAIM Artificial Intelligence Methods Dr. Rong Qu Evolutionary Algorithms.
Artificial Intelligence Chapter 4. Machine Evolution.
Genetic Algorithms Czech Technical University in Prague, Faculty of Electrical Engineering Ondřej Vaněk, Agent Technology Center ZUI 2011.
Edge Assembly Crossover
Chapter 12 FUSION OF FUZZY SYSTEM AND GENETIC ALGORITHMS Chi-Yuan Yeh.
D Nagesh Kumar, IIScOptimization Methods: M8L5 1 Advanced Topics in Optimization Evolutionary Algorithms for Optimization and Search.
1 Autonomic Computer Systems Evolutionary Computation Pascal Paysan.
The Standard Genetic Algorithm Start with a “population” of “individuals” Rank these individuals according to their “fitness” Select pairs of individuals.
Selection and Recombination Temi avanzati di Intelligenza Artificiale - Lecture 4 Prof. Vincenzo Cutello Department of Mathematics and Computer Science.
Ch 20. Parameter Control Ch 21. Self-adaptation Evolutionary Computation vol. 2: Advanced Algorithms and Operators Summarized and presented by Jung-Woo.
Artificial Intelligence By Mr. Ejaz CIIT Sahiwal Evolutionary Computation.
CAP6938 Neuroevolution and Artificial Embryogeny Evolutionary Comptation Dr. Kenneth Stanley January 23, 2006.
Dr. Kenneth Stanley September 11, 2006
Evolutionary Programming
The 2st Chinese Workshop on Evolutionary Computation and Learning
Evolution Strategies Evolutionary Programming
Genetic Algorithms GAs are one of the most powerful and applicable search methods available GA originally developed by John Holland (1975) Inspired by.
Lecture 11.
Evolution strategies and genetic programming
School of Computer Science & Engineering
C.-S. Shieh, EC, KUAS, Taiwan
Lecture 10.
Example: Applying EC to the TSP Problem
Chapter 6: Genetic Algorithms
An evolutionary approach to solving complex problems
Genetic Algorithms, Search Algorithms
Example: Applying EC to the TSP Problem
Artificial Intelligence Chapter 4. Machine Evolution
Example: Applying EC to the TSP Problem
G5BAIM Artificial Intelligence Methods
Evolutionary Computation,
Introduction to Artificial Intelligence Lecture 11: Machine Evolution
Dr. Unnikrishnan P.C. Professor, EEE
Parameter control Chapter 8.
Artificial Intelligence Chapter 4. Machine Evolution
Evolutionary Programming
Boltzmann Machine (BM) (§6.4)
A Gentle introduction Richard P. Simpson
A Tutorial (Complete) Yiming
Biologically Inspired Computing: Operators for Evolutionary Algorithms
Parameter control Chapter 8.
Beyond Classical Search
Evolution Strategies Originally developed in Germany in the early 60s with Rechenberg and Schwefel being the main contributors. Main ideas include: floating.
Evolution Strategies Originally developed in Germany in the early 60s with Rechenberg and Schwefel being the main contributors. Main ideas include: floating.
Parameter control Chapter 8.
Presentation transcript:

Evolution strategies Can programs learn? This techniques can be used as an optimization strategy but we are going to look at a learning example

Evolution strategies Chapter 8 of Almost anything by David Fogel Michalewicz, Z. (1996). Genetic Algorithms + Data Structures = Evolution Programs, Springer-Verlag, ISBN 3-540-60676-9 Almost anything by David Fogel Fogel, D. B. (1994) IEEE Transactions on Neural Networks, Vol 5:1, pp3- 14 Fogel, D.B. (1998) Evolutionary Computation The Fossil Record, IEEE Press, ISBN 0-7803-3481-7, pp 3-14 Michalewicz, Z. and Fogel, D. (2000). How to Solve It : Modern Heuristics. Springer-Verlag, ISBN 3-540-66061-5

Evolution strategies There are other ways that we could design computer programs so that they “learn” For example, knowledge based on some suitable logic symbolism Use inference rules We should also be careful not confuse evolutionary strategies with evolutionary programming (EP). EP is about writing programs that write programs

Evolution strategies vs GA’s ES’s are an algorithm that only uses mutation and does not use crossover This is not a formal definition and there is no reason why we cannot incorporate crossover (as Michalewicz, 1996 shows)

Evolution strategies vs GA’s ES’s are normally applied to real numbers (continuous variables) rather than discrete values. Again, this is not a strict definition and work has been done on using ES’s for discrete problems (Bäck, 1991) and (Herdy, 1991)

Evolutionary Algorithms vs GA’s ES’s are a population based approach Originally only a single solution was maintained and this was improved upon.

Evolutionary Algorithms vs GA’s In summaryES’s are Like genetic algorithms but only use mutation and not crossover They operate on real numbers They are a population based approach But we can break any, or all, of these rules if we wish!

Evolution strategies - How They Work An individual in an ES is represented as a pair of real vectors, v = (x,σ) x, represents a point in the search space and consists of a number of real valued variables The second vector, σ, represents a vector of standard deviations

Evolution strategies- How They Work Mutation is performed by replacing x by xt+1 = xt + N(0, σ) N(0, σ) is a random Gaussian number with a mean of zero and standard deviations of σ

Evolution strategies - How They Work This mimics the evolutionary process that small changes occur more often than larger ones

Evolutionary Algorithms - How They Work In the earliest ES’s (where only a single solution was maintained), the new individual replaced its parent if it had a higher fitness In addition, these early ES’s, maintained the same value for σ throughout the duration of the algorithm It has been proven that if this vector remains constant throughout the run then the algorithm will converge to the optimal solution

Evolutionary Algorithms - How They Work Problem Although the global optimum can be proved to be found with a probability of one, it also states that the theorem holds for sufficiently long search time The theorem tells us nothing about how long that search time might be

Evolution strategies - How They Work To try and speed up convergence Rechenberg has proposed the “1/5 success rule.” It can be stated as follows The ratio, , of successful mutations to all mutations should be 1/5. Increase the variance of the mutation operator if  is greater than 1/5; otherwise, decrease it

Evolution strategies - How They Work Motivation behind 1/5 rule If we are finding lots of successful moves then we should try larger steps in order to try and improve the efficiency of the search If we not finding many successful moves then we should proceed in smaller steps

Evolution strategies - How They Work The 1/5 rule is applied as follows if (k) < 1/5 then σ = σcd if (k) > 1/5 then σ = σci if (k) = 1/5 then σ = σ

Evolution strategies - How They Work if (k) < 1/5 then σ = σcd if (k) > 1/5 then σ = σci if (k) = 1/5 then σ = σ k dictates how many generations should elapse before the rule is applied cd and ci determine the rate of increase or decrease for σ ci must be greater than one and cd must be less than one Schwefel used cd = 0.82 and ci = 1.22 (=1/0.82)

Evolution strategies - How They Work Problem with the applying the 1/5 rule It may lead to premature convergence for some problems Increase the population size, which now turns ES’s into a population based approach search mechanism

Evolution strategies - How They Work Increase population size The population size is now (obviously) > 1. All members of the population have an equal probability of mating We could now introduce the possibility of crossover As we have more than one individual we have the opportunity to alter σ independently for each member We have more options with regards to how we control the population (discussed next)

Evolution strategies - How They Work In evolution strategies there are two variations as to how we create the new generation

Evolution strategies - How They Work ( + ), uses  parents and creates  offspring After mutation, there will be  +  members in the population All these solutions compete for survival, with the  best selected as parents for the next generation

Evolution strategies - How They Work (, ), works by the  parents producing  offspring (where  >  ) Only the  compete for survival. Thus, the parents are completely replaced at each new generation Or, to put it another way, a single solution only has a life span of a single generation

Evolution strategies - How They Work The original work on evolution strategies (Schwefel, 1965) used a (1 + 1) strategy This took a single parent and produced a single offspring Both these solutions competed to survive to the next generation

Evolution strategies - Case Study Simplified Blackjack Blackjack is a two player game comprising of a dealer and a player. The dealer deals two cards (from a normal pack of 52 cards) to the player and one card to himself. All cards are dealt face up. All cards take their face value, except Jack, Queen and King which count as 10 and Aces which can count as one or eleven The aim for the player is to draw as many cards as he/she wishes (zero if they wish) in order to get as close as possible to 21 without exceeding 21

Evolution strategies - Your Go Simplified Blackjack How might we write an agent that learns how to play blackjack? Learn Probabilities?

Evolution strategies - Your Go Questions Do you think this would work? Should we use a single candidate for each probability or should we have a population greater than one? What sort of evolutionary scheme should we use; ( + ) or (, ); and what values should we give  and ? Can you come up with a better representation; other than trying to learn probabilities?

Evolution strategies - Finally Evolution strategies can be used as search methods as well as a learning mechanism It just needs saying!