A Tutorial (Complete) Yiming Evolution Strategies A Tutorial (Complete) Yiming

Objectives For newly-coming students Get familiar with EC techniques For senior students Get inspired with some design principles in ES For myself Learn from all of audiences

Outline Where What How When/Why

Where All fields of optimisation including continuous, discrete, combinatorial, with or without constraints The objective function, i.e., f(y), can be represented in mathematical form, via simulations, or even in terms of measurements obtained from real objects.

Examples …and more….

When

What A search paradigm Inspired by biological evolution General EC algorithm

Mating Selection (fitness-independent Main principles of ES Mating Selection (fitness-independent or fitness-based) Recombination Mutation and Parameter control Survivor Selection

ES Main Features Representation Real-valued vectors Recombination Discrete, Intermediate, Weighted Mutation Gaussian Pertubation Mating Selection Fitness independent, Fitness-based Survivor Selection + or , Special Feature Parameter Control

Notations —Number of parents —Number of parents are used for recombination —Number of offspring —“plus” selection selects the best of individuals without considering their ages — “comma” selection selects the youngest individuals, parents die out In this case of using comma selection , must hold.

Two Algorithm Templates

Recombination Recombination combines information from several parents to generate a single new offspring Discrete (Dominant) Uniform select one parent to inherit values Intermediate Average values of values from all selected parents Weighted Weighted average of values from all selected parents Others One-point and two-point crossover, etc. Note: ES allows multi-combination ( rho > 2)

Mutation Mutation introduces (“small”) variations by adding a point symmetric perturbation to the result of recombination. The perturbation is drawn from a multivariate normal distribution, . We have Spherical/isotropic (Uncorrelated) with step size Axis-parallel (Uncorrelated) General (Correlated)

Mutation

Parameter Control The key of ES Goal: drive the endogenous strategy parameters (e.g., step-size) close to their optimal values.

(1+1)-ES with 1/5th Success Rule

(1+1)-ES with 1/5th Success Rule

Sigma is IMPORTANT Principle: Increasing it when the successful rate is high, decreasing it when the successful rate is low. Other ways to adapt it via evolution Self-adaptation Derandomized Self-adaptation Non-local derandomized step-size control

Address Dependencies CMA-ES

Natural Evolutionary Strategy

Examples 1+1 ES (1/5 rules) u+u ES NES

Rastrigin Function

u+u ES

CMA-ES

NES

Travelling Salesman Problem

How do you solve the TSP problem?

Some possible ways Mathematical Programming - Google OR-tools https://developers.google.com/optimization/routing/tsp Memetic algorithms Lin, S. (1965). Computer solutions of the traveling salesman problem. The Bell system technical journal, 44(10), 2245-2269. Ant colony algorithms https://github.com/ppoffice/ant-colony-tsp Simulated Annealing https://github.com/ildoonet/simulated-annealing-for-tsp

Genetic Algorithms Many can be found, I also implemented one. Neural Networks https://github.com/jpowie01/TSP-NN-Training http://jamestunnell.github.io/blog/2014/06/09/chaotic- tsp/ And many more

Can this optimisation problem be solved by learning?

What about Deep Learning?

Some possible ways Deep Supervised learning ? Deep Reinforcement learning https://github.com/pemami4911/neural-combinatorial-rl- pytorch https://multithreaded.stitchfix.com/blog/2016/07/21/skyn et-salesman/

Skynet Salesman

CNN plays a game

Skynet Salesman

Can we expand this idea to other combinatorial optimisation problems?