D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.

Slides:



Advertisements
Similar presentations
Numbers Treasure Hunt Following each question, click on the answer. If correct, the next page will load with a graphic first – these can be used to check.
Advertisements

Scenario: EOT/EOT-R/COT Resident admitted March 10th Admitted for PT and OT following knee replacement for patient with CHF, COPD, shortness of breath.
Variations of the Turing Machine
Angstrom Care 培苗社 Quadratic Equation II
AP STUDY SESSION 2.
1
Select from the most commonly used minutes below.
Copyright © 2003 Pearson Education, Inc. Slide 1 Computer Systems Organization & Architecture Chapters 8-12 John D. Carpinelli.
Copyright © 2011, Elsevier Inc. All rights reserved. Chapter 6 Author: Julia Richards and R. Scott Hawley.
STATISTICS HYPOTHESES TEST (I)
Fixture Measurements Doug Rytting.
David Burdett May 11, 2004 Package Binding for WS CDL.
1 RA I Sub-Regional Training Seminar on CLIMAT&CLIMAT TEMP Reporting Casablanca, Morocco, 20 – 22 December 2005 Status of observing programmes in RA I.
Local Customization Chapter 2. Local Customization 2-2 Objectives Customization Considerations Types of Data Elements Location for Locally Defined Data.
CALENDAR.
1 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt BlendsDigraphsShort.
1 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt RhymesMapsMathInsects.
Chapter 7 Sampling and Sampling Distributions
1 Click here to End Presentation Software: Installation and Updates Internet Download CD release NACIS Updates.
Break Time Remaining 10:00.
Particle Swarm Optimization (PSO)
Turing Machines.
Table 12.1: Cash Flows to a Cash and Carry Trading Strategy.
PP Test Review Sections 6-1 to 6-6
BBA Part1_1 (Gajaseni, 2001)1 Man and Environment Asst. Dr. Nantana Gajaseni.
Bright Futures Guidelines Priorities and Screening Tables
EIS Bridge Tool and Staging Tables September 1, 2009 Instructor: Way Poteat Slide: 1.
Outline Minimum Spanning Tree Maximal Flow Algorithm LP formulation 1.
Bellwork Do the following problem on a ½ sheet of paper and turn in.
Exarte Bezoek aan de Mediacampus Bachelor in de grafische en digitale media April 2014.
TESOL International Convention Presentation- ESL Instruction: Developing Your Skills to Become a Master Conductor by Beth Clifton Crumpler by.
Copyright © 2013, 2009, 2006 Pearson Education, Inc. 1 Section 5.5 Dividing Polynomials Copyright © 2013, 2009, 2006 Pearson Education, Inc. 1.
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
1 RA III - Regional Training Seminar on CLIMAT&CLIMAT TEMP Reporting Buenos Aires, Argentina, 25 – 27 October 2006 Status of observing programmes in RA.
Basel-ICU-Journal Challenge18/20/ Basel-ICU-Journal Challenge8/20/2014.
1..
1 © 2004, Cisco Systems, Inc. All rights reserved. CCNA 1 v3.1 Module 10 Routing Fundamentals and Subnets.
Adding Up In Chunks.
SLP – Endless Possibilities What can SLP do for your school? Everything you need to know about SLP – past, present and future.
MaK_Full ahead loaded 1 Alarm Page Directory (F11)
1 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt 10 pt 15 pt 20 pt 25 pt 5 pt Synthetic.
Artificial Intelligence
: 3 00.
5 minutes.
Research Summary 08/2010 Dr. Andrej Mošat` Prof. A. Linninger, Laboratory for Product and Process Design, M/C 063 University of Illinois at Chicago 04.
1 hi at no doifpi me be go we of at be do go hi if me no of pi we Inorder Traversal Inorder traversal. n Visit the left subtree. n Visit the node. n Visit.
Analyzing Genes and Genomes
1 Let’s Recapitulate. 2 Regular Languages DFAs NFAs Regular Expressions Regular Grammars.
Speak Up for Safety Dr. Susan Strauss Harassment & Bullying Consultant November 9, 2012.
1 Titre de la diapositive SDMO Industries – Training Département MICS KERYS 09- MICS KERYS – WEBSITE.
Essential Cell Biology
Converting a Fraction to %
Numerical Analysis 1 EE, NCKU Tien-Hao Chang (Darby Chang)
Chapter 8 Estimation Understandable Statistics Ninth Edition
Clock will move after 1 minute
PSSA Preparation.
Essential Cell Biology
Immunobiology: The Immune System in Health & Disease Sixth Edition
Physics for Scientists & Engineers, 3rd Edition
Energy Generation in Mitochondria and Chlorplasts
Select a time to count down from the clock above
Copyright Tim Morris/St Stephen's School
9. Two Functions of Two Random Variables
Introduction Peter Dolog dolog [at] cs [dot] aau [dot] dk Intelligent Web and Information Systems September 9, 2010.
1 Dr. Scott Schaefer Least Squares Curves, Rational Representations, Splines and Continuity.
1 Decidability continued…. 2 Theorem: For a recursively enumerable language it is undecidable to determine whether is finite Proof: We will reduce the.
Differential Evolution Hossein Talebi Hassan nikoo 1.
Differential Evolution Hossein Talebi Hassan Nikoo 1.
Evolutionary Algorithms 3.Differential Evolution.
Presentation transcript:

D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December,

A GENDA Preface Basic Differential Evolution Difference Vectors Mutation Crossover Selection General Differential Evolution Algorithm Control Parameters Geometrical Illustration DE/ x/y/z 2

A GENDA Variations to Basic Differential Evolution Hybrid Differential Evolution Strategies Population-Based Differential Evolution Self-Adaptive Differential Evolution Differential Evolution for Discrete-Valued Problems Constraint Handling Approaches Comparison with other algorithms Applications 3

P REFACE Price and Storn in 1995 Chebychev Polynomial fitting Problem 3rd place at the First International Contest on evolutionary Computation (1stICEO) 1996, the best genetic type of algorithm for solving the real-valued test function suite. stochastic, population-based search strategy Main characteristics Guide search with distance and direction information from the current population original DE strategies for continuous-valued landscapes 4

B ASIC D IFFERENTIAL E VOLUTION mutation is applied first to generate a trial vector, which is then used within the crossover operator to produce one offspring, mutation step sizes are not sampled from a prior known probability distribution function. mutation step sizes are influenced by differences between individuals of the current population 5

D IFFERENCE V ECTORS Position of individuals and fitness Over time, as the search progresses, the distances between individuals become smaller The magnitude of the initial distances between individuals is influenced by the size of the population Distances between individuals are a very good indication of the diversity of the current population Use difference vector to determine the step size total number of differential perturbations n v is the number of differentials used n s is the population size 6

M UTATION 7

C ROSSOVER Binomial crossover Exponential crossover 8

S ELECTION Random Selection To select the individuals from which difference vectors are calculated. The target vector is either randomly selected or the best individual is selected Deterministic Selection To construct the population for the next generation, the offspring replaces the parent if the fitness of the offspring is better than its parent; otherwise the parent survives to the next generation. This ensures that the average fitness of the population does not deteriorate. 9

G ENERAL D IFFERENTIAL E VOLUTION A LGORITHM 10

11

C ONTROL P ARAMETERS population size, n s scale factor, β probability of recombination, Pr 12

P OPULATION S IZE The size of the population has a direct influence on the exploration ability of DE algorithms. The more individuals there are in the population, the more differential vectors are available, and the more directions can be explored The computational complexity per generation increases with the size of the population. Empirical studies provide the guideline that n s 10n x 13

S CALE F ACTOR The scaling factor, β (0,), controls the amplification of the differential variations, (x i 2 xi 3 ). The smaller the value of β, the smaller the mutation step sizes Smaller step sizes can be used to explore local areas. slower convergence Larger values for β facilitate exploration, but may cause the algorithm to overshoot optima As the population size increases, the scaling factor should decrease. 14

R ECOMBINATION P ROBABILITY This parameter controls the number of elements of the parent, x i (t), that will change. The higher the probability of recombination, the more variation is introduced in the new population, thereby increasing diversity and increasing exploration. Increasing p r often results in faster convergence, while decreasing pr increases search robustness 15

G EOMETRICAL I LLUSTRATION 16

G EOMETRICAL I LLUSTRATION 17

G EOMETRICAL I LLUSTRATION 18

E XAMPLE :P EAK FUNCTION 19

G EOMETRICAL I LLUSTRATION Generation 1: DEs population and difference vector distributions 20

G EOMETRICAL I LLUSTRATION Generation 6: The population coalesces around the two main minima 21

G EOMETRICAL I LLUSTRATION Generation 12: The difference vector distribution contains three main clouds – one for local searches and two for moving between the two main minima. 22

G EOMETRICAL I LLUSTRATION Generation 16: The population is concentrated on the main minimum 23

G EOMETRICAL I LLUSTRATION Generation 20: Convergence is imminent. The difference vectors automatically shorten for a fine-grained, local search. 24

G EOMETRICAL I LLUSTRATION Generation 26: The population has almost converged. 25

G EOMETRICAL I LLUSTRATION Generation 34: DE finds the global minimum. 26

DE/ X / Y / Z DE/best/1/ z DE/ x/n v /z DE/rand-to-best/ n v /z 27

DE/ X / Y / Z DE/current-to-best/1+ nv/z DE/rand/1/bin vs. DE/current-to-best/2/bin DE/rand/1/bin maintains good diversity DE/current-to-best/2/bin shows good convergence characteristics Dynamically switch between these two strategies 28

V ARIATIONS TO B ASIC D IFFERENTIAL E VOLUTION Hybrid Differential Evolution Strategies Gradient-Based Hybrid Differential Evolution Acceleration operator : to improve convergence speed Migration operator : to improve ability for escaping local optima Acceleration operator uses gradient descent to adjust the best individual toward obtaining a better position if the mutation and crossover operators failed to improve x( t), replaces the worst individual in the new population, C(t+1). 29

M IGRATION OPERATOR Gradient decent speed up but local minima Migration operator increase population diversity Generate new individual from best individuals Applied when diversity is too small i.e.: 30

H YBRID D IFFERENTIAL E VOLUTION WITH A CCELERATION AND M IGRATION 31

E VOLUTIONARY A LGORITHM -B ASED H YBRIDS DE reproduction process as a crossover operator in a simple GA Rank-Based Crossover Operator for Differential Evolution To select individuals to calculate difference vectors x i1 (t) precedes x i2 (t) if f(x i1 (t)) > f(x i2 (t)). 32

R ANK -B ASED M UTATION O PERATOR FOR D IFFERENTIAL E VOLUTION 33

O THER V ARIATIONS TO B ASIC DE Population-Based Differential Evolution Improve exploration by using 2 population set Initialize with n s pairs Rejected individual by selection put in auxiliary pop Self-Adaptive Differential Evolution Dynamic Parameters Self-Adaptive Parameters 34

D IFFERENTIAL E VOLUTION FOR D ISCRETE - V ALUED P ROBLEMS Angle Modulated DE where x is a single element from a set of evenly separated intervals determined by the required number of bits that need to be generated 35

A NGLE M ODULATED D IFFERENTIAL E VOLUTION 36

B INARY D IFFERENTIAL E VOLUTION 37

C ONSTRAINT H ANDLING A PPROACHES Penalty methods Converting the constrained problem to an unconstrained problem By changing the selection operator of DE, infeasible solutions can be rejected 38

C OMPARISON WITH GA AND PSO 39

C OMPARISON 40

41

42

43

44

A PPLICATIONS 1) General Optimization Framework "Mystic" by Mike McKerns, Caltech. 2) Multiprocessor synthesis. 3) Neural network learning. 4) Chrystallographic characterization. 5) Synthesis of modulators. 6) Heat transfer parameter estimation in a trickle bed reactor. 7) Scenario-Integrated Optimization of Dynamic Systems. 8) Optimal Design of Shell-and-Tube Heat Exchangers. 9) Optimization of an Alkylation Reaction. 10) Optimization of Thermal Cracker Operation. 11) Optimization of Non-Linear Chemical Processes. 12) Optimum planning of cropping patterns. 13) Optimization of Water Pumping System. 14) Optimal Design of Gas Transmission Network. 15) Differential Evolution for Multi-Objective Optimization 16) Physiochemistry of Carbon Materials. 17) Radio Network Design. 18) Reflectivity Curve Simulation. 45

C OMMERCIAL SOFT 1) Built in optimizer in MATHEMATICA's function Nminimize (since version 4.2). 2) MATLAB's GA toolbox contains a variant of DE. 3) Digital Filter Design. 4) Diffraction grating design. 5) Electricity market simulation. 6) Auto2Fit. 7) LMS Virtual Lab Optimization. 8) Optimization of optical systems. 9) Finite Element Design. 46

A PPLICATION : BUMP PROBLEM 47

A PPLICATION : BUMP PROBLEM 48

49

R EFERENCES [1] [2] Andries P. Engelbrecht,(2007),Computational Intelligence: An Introduction, 2nd Edition., ISBN: Andries P. Engelbrecht [3] Price, K.; Storn, R.M.; Lampinen, J.A. (2005). Differential Evolution: A Practical Approach to Global Optimization. Springer. ISBN ence/foundations+of+computations/book/ Differential Evolution: A Practical Approach to Global OptimizationISBN ence/foundations+of+computations/book/ [4] Feoktistov, V. (2006). Differential Evolution: In Search of Solutions. Springer. ISBN Differential Evolution: In Search of SolutionsISBN [5] J. Vesterstrom and R. Thomson, A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems, Proc. of IEEE Congress on Evolutionary Computation, 2004, pp. 1980–

يک سخن زندگي آن چيزي است که براي تو اتفاق ميافتد، در حالي که تو سرگرم برنامهريزيهاي ديگري هستي 51

T HANKS FOR YOUR ATTENTION 52