10/01/2014 DMI - Università di Catania 1 Combinatorial Landscapes & Evolutionary Algorithms Prof. Giuseppe Nicosia University of Catania Department of Mathematics and Computer Science

10/01/2014 DMI - Università di Catania 2 Talk Outline 1.Combinatorial Landscapes 2.Evolutionary Computing

10/01/2014 DMI - Università di Catania 3 1. Combinatorial Landscapes the behaviour of search algorithms to characterize the difficulty The notion of landscape is among the rare existing concepts which help to understand the behaviour of search algorithms and heuristics and to characterize the difficulty of a combinatorial problem.

10/01/2014 DMI - Università di Catania 4 Search Space Psearch space P(S,f) Given a combinatorial problem P, a search space associated to a mathematical formulation of P is defined by a couple (S,f) S –where S is a finite set of configurations (or nodes or points) and –fcost function S –f a cost function which associates a real number to each configurations of S. the minimum and the maximum costs combinatorial optimization problems For this structure two most common measures are the minimum and the maximum costs.In this case we have the combinatorial optimization problems.

10/01/2014 DMI - Università di Catania 5 Example: K-SAT An instance of the K-SAT problem consists of a set V of variables, a collection C of clauses over V such that each clause c C has |c|= K. The problem is to find a satisfying truth assignment for C. The search space for the 2-SAT with |V|=2 is (S,f) where –S –S={ (T,T), (T,F), (F,T), (F,F) } and –the cost function –the cost function for 2-SAT computes only the number of satisfied clauses f sat (s)= #SatisfiedClauses(F,s), s S

10/01/2014 DMI - Università di Catania 6 An example of Search Space Let we consider F = (A B) ( A B) A B f sat (F,s) T 1 T F 2 F T 1 F 2

10/01/2014 DMI - Università di Catania 7 Search Landscape (S,f)search landscape (S,n,f)n neighborhood functionGiven a search space (S,f), a search landscape is defined by a triplet (S,n,f) where n is a neighborhood function which verifies n : S 2 S -{ 0} energy landscape neutralThis landscape, also called energy landscape, can be considered as a neutral one since no search process is involved. weighted graphIt can be conveniently viewed as weighted graph G=(S, n, F) where the weights are defined on the nodes, not on the edges.

10/01/2014 DMI - Università di Catania 8 Example and relevance of Landscape N dimensional hypercube The search Landscape for the K-SAT problem is a N dimensional hypercube with N = number of variables = |V|. hard to solve huge and complex search landscapeCombinatorial optimization problems are often hard to solve since such problems may have huge and complex search landscape.

10/01/2014 DMI - Università di Catania 9 Hypercubes

10/01/2014 DMI - Università di Catania 10 Solvable & Impossible Separating Insolvable and DifficultThe New York Times, July 13, 1999 Separating Insolvable and Difficult. B. Selman, R. Zecchina, et al.Determing computational complexity from characteristic phase transitions, Nature, Vol. 400, 8 July 1999,

10/01/2014 DMI - Università di Catania 11 =4.256 Phase Transition, =4.256

10/01/2014 DMI - Università di Catania 12 Characterization of the Landscape in terms of Connected Components #3-SATn=10 Number of solutions, number of connected components and CCs' cardinality versus for #3-SAT problem with n=10 variables.

10/01/2014 DMI - Università di Catania 13 (3)=4.256 CC's cardinality at phase transition (3)=4.256 (3)=4.256n #3-SAT problem Number of Solutions, number of connected components and CC's cardinality at phase transition (3)=4.256 versus number of variables n for #3-SAT problem.

10/01/2014 DMI - Università di Catania 14 Process Landscape process landscape(S, n, f, ) search process Given a search landscape (S, n, f), a process landscape is defined by a quadruplet (S, n, f, ) where is a search process. The process landscape represents a particular view of the neutral landscape (S, n, f) seen by a search algorithm. Examples of search algorithms: –Local Search Algorithms. –Complete Algorithms (e. g. Davis-Putnam algorithm). –Evolutionary Algorithms –Evolutionary Algorithms: Genetic Algorithms, Genetic Programming, Evolution Strategies, Evolution Programming, Immune Algorithms.

10/01/2014 DMI - Università di Catania Evolutionary Algorithms EAs are optimization methods based on an evolutionary metaphor that showed effective in solving difficult problems. Evolution is the natural way to program Evolution is the natural way to program Thomas Ray

10/01/2014 DMI - Università di Catania 16 Evolutionary Algorithms 1. Set of candidate solutions (individuals): Population. 2. Generating candidates by: –Reproduction –Reproduction: Copying an individual. –Crossover –Crossover: 2 parents 2 children. –Mutation –Mutation: 1 parent 1 child. Fitness function 3. Quality measure of individuals: Fitness function. Survival-of-the-fittest 4. Survival-of-the-fittest principle.

10/01/2014 DMI - Università di Catania 17 Main components of EAs Coding 1. Representation of individuals: Coding. Fitness 2. Evaluation method for individuals: Fitness. 1st generation 3. Initialization procedure for the 1st generation. mutationcrossover 4. Definition of variation operators (mutation and crossover). mating 5. Parent (mating) selection mechanism. environmental 6. Survivor (environmental) selection mechanism. Technical parameters 7. Technical parameters (e.g. mutation rates, population size). Experimental tests, Adaptation based on measured quality, Self-adaptation based on evolution.

10/01/2014 DMI - Università di Catania 18 Mutation and Crossover EAs manipulate partial solutions in their search for the overall optimal solution EAs manipulate partial solutions in their search for the overall optimal solution. These partial solutions or `building blocks' correspond to sub-strings of a trial solution - in our case local sub-structures within the overall conformation.

10/01/2014 DMI - Università di Catania 19 Algorithm Outline procedure EA; { t = 0; initialize population (P(t), d); evaluate P(t); until (done) { t = t + 1; parent_selection P(t); recombine (P(t), p cross ); mutate ( P(t), p mut ); evaluate P(t); survive P(t); }