Omni-Optimizer A Procedure for Single and Multi-objective Optimization Prof. Kalyanmoy Deb and Santosh Tiwari

Motivation  Generic Programming Practices  Unified algorithm for all types of optimization problems  An efficient implementation of NSGA-II framework (procedure)  Developing an efficient and self-adaptive optimization paradigm

Literature Survey  CHC (Cross generation elitist selection, Heterogeneous recombination, Cataclysmic mutation) – Explicit Diversity  GENITOR (Steady state GA), more like (µ+1)-ES so far as selection mechanism is concerned. – High selection pressure  NPGA (Niched Pareto Genetic Algorithm), uses sharing parameter σ share – # of niches obtained depend on the sharing parameter

Literature Survey contd…  PESA (Pareto Envelope-based Selection Algorithm), Hyper- grid division of phenotypic space, selection based on crowding measure  NSGA-II (Non-dominated Sorting Genetic Algorithm)  SPEA2 (Strength Pareto Evolutionary Algorithm), Fine grained fitness assignment mechanism utilizing density information, Only archive members participate in mating – Excellent Diversity in phenotypic space  NCGA (Neighborhood Cultivation Genetic Algorithm), used neighborhood crossover, based on NSGA-II and SPEA2  RPSGAe (Reduced Pareto Set Genetic Algorithm with elitism)  ENORA (Evolutionary Algorithm of Non-dominated Sorting with Radial Slots)

Salient Features of the Algorithm  Based on NSGA-II framework  Based on the concept of Pareto dominance  Incorporates elitism  Explicit diversity preserving mechanism  Can be used for single-objective as well as multi- objective problems  Can be used for uni-global as well as multi- global problems  Independent of the number of niches that an optimization problems exhibits

Moving beyond NSGA-II  Restricted Selection Scheme  Tournament selection based on usual domination  Non-dominated sorting based on epsilon dominance  Crowding Distance Assignment  Genotypic as well as Phenotypic space niching  Choose best members from above average population  Remove worst members from below average population  More robust recombination and variation operators  Two point crossover for binary variables  Highly disruptive real variable mutation

Restricted Selection  Helps in preserving multi-modality  Experiments show that it gives faster overall convergence

Epsilon Domination Principle A finite percentage (based on function value) of solutions assigned a particular rank A finite percentage (based on function value) of solutions assigned a particular rank Allows somewhat inferior solutions to remain in the population Allows somewhat inferior solutions to remain in the population Provides guaranteed diversity Provides guaranteed diversity Helps to obtain multi-modal solutions in case of single objective problems Helps to obtain multi-modal solutions in case of single objective problems Epsilon is generally taken in the range ~ Epsilon is generally taken in the range ~ 10 -6

Modified Crowding Distance  Genotypic as well as Phenotypic space niching

Highly Disruptive Mutation Operator

Computational Complexity  Restricted selection O (nN 2 )  Ranking procedure O (MN 2 )  Crowding procedure max{ O (MN log N), O (nN log N) }  Overall iteration-wise complexity max {O (nN 2 ), O (MN 2 ), O (nN log N)}

Implementation Details  Code written in simple C and strictly conforms to ANSI/ISO standard  Data structure used is a custom doubly linked list (gives O(1) insertion and deletion)  Randomized quick sort used for sorting  Almost all the functions perform in-place operation (addresses are passed, significantly decreases stack overheads)

Simulation Results  GA parameters for all problems chosen as follows  η c = 20  η m = 20  P (crossover) = 0.8  P (mutation) = 1/n, where n = # of real variables  δ =  Population size and number of generations different for different problems

Simulation Results contd…  20 variable Rastrigin function  # of function evaluation  Least =  Median =  Worst =  20 variable Schwefel function  # of function evaluation  Least =  Median =  Worst =  Other single objective problems can be found in the paper  In all cases, better results are found in comparison to previous reported studies

Single objective multi-modal function f(x) = sin 2 (πx)x є [0,20]

Single objective multi-modal function Unconstrained Himmelblau’s function

Multi-objective Uni-Global Test Problems 30 variable ZDT2 (100×100)

Multi-objective Uni-Global Test Problems 10 variable ZDT4 (100×250)

Multi-objective Uni-Global Test Problems CTP4 (100×7000)

Multi-objective Uni-Global Test Problems CTP8 (100×100)

Multi-objective Uni-Global Test Problems DTLZ4 (300×100)

Multi-objective Multi-Global Test Problem F 1 (x) = summation (sin (πx i ) )x i є [0,6] F 2 (x) = summation (cos (πx i ) )x i є [0,6] Efficient points in phenotypic space

Multi-objective Multi-Global Test Problem Genotypic space plots

Few Sample Simulations  F(x) = sin 2 (10,000*pi*x)  Himmelblau’s Functions  ZDT Test Problems  CTP Test Problems  Test Problem TNK  Multi-global Multi-objective Test Problem

Further Ideas and Future Work  Incorporating PCX instead of SBX for crossover  Automatically fine-tuning mutation index so as to achieve arbitrary precision  Self-adaptation of parameter δ  Segregating population into niches without the introduction of DM  Dynamic population sizing  Using hierarchical NDS for the crowding distance assignment