1. Societies of Hill-Climbers Before presenting SoHCs let’s first talk about Hill-Climbing in general. For this lecture we will confine ourselves to binary- coded representations; however, it is possible for us to extend the concept of hill-climbing to real-coded representations as well.

2. Deterministic Hill-Climbing In our discussion on Hill-Climbing lets try to solve two function optimization problems (Maximization): –F1(x) = x 2, and –F2(x) = Σ x i We will apply two types of hill-climbing –Steepest Ascent and –Next Ascent.

3. Deterministic Hill-Climbing Hill-Climbers are local search techniques. Therefore it is important to define the neighbor of the local search. For our demonstration, all CSs that are a hamming distance of 1 away from the parent (indivdual being mutated) are considered neighbors.

4. Deterministic Steepest Ascent Hill-Climbing Consider the representation for CS for F1 and F2 to be composed of 5-bits. Deterministic Steepest Ascent Hill-Climbing works as follows: –Step1: Randomly generate and evaluate, b, the current best individual –Step2: Generate and evaluate ALL of the neighbors of b. Let Neighbors = the set of λ neighbors. –Step3: b = best-of(b  Neighbors) –Step4: if the the fitness of b does not improve either terminate the search or goto Step1; Else goto Step2.

5. Deterministic Steepest Ascent Hill-Climbing Let’s use DSAHC to solve F1 –Let b = (01001) –Neighbors = {(11001), (00001), (01101), 01011), (01000)} –The best of b  Neighbors is (11001) therefore, –Let b = (11001)

6. Deterministic Steepest Ascent Hill-Climbing Continuing (second cycle) –Let b = (11001) –Neighbors = {(01001), (10001), (11101), 11011), (11000)} –The best of b  Neighbors is (11101) therefore, –Let b = (11101)

7. Deterministic Steepest Ascent Hill-Climbing Continuing (third cycle) –Let b = (11101) –Neighbors = {(01101), (10101), (11001), 11111), (11100)} –The best of b  Neighbors is (11111) therefore, –Let b = (11111) On the fourth cycle, we cannot find a candidate solution that is better than b so we will terminate our search.

8. Deterministic Steepest Ascent Hill-Climbing For some problems the defined neighbor may be too large to exhaustively search. When this is the case, a Stochastic Steepest Ascent Hill-Climber may be preferred. In a Stochastic Steepest Ascent Hill-Climber, Neighbors is a set of a user-specified number of neighbors are generated from the neighborhood. b = best-of(b  Neighbors)

9. Deterministic Next Ascent Hill-Climbing Deterministic Next Ascent Hill-Climbing works as follows: –Step1: Randomly generate and evaluate, b, the current best individual For k = 1 to Number of Neighbors Do –Step2: Generate and evaluate the neighbors of b one at a time. –Generate and evaluate Neighbor k –Step3: b = best-of(b,Neighbor k ) End (For) –Step4: if the the fitness of b does not improve either terminate the search or goto Step1; Else goto Step2.

10. Deterministic Next Ascent Hill-Climbing Let’s use DNAHC to solve F1 –Let b = (01001) –Neighbor = (11001) –The best of b and Neighbor is (11001) therefore, –Let b = (11001)

11. Deterministic Next Ascent Hill-Climbing Continuing (second cycle) –Let b = (11001) –Neighbor = (01001) –The best of b and Neighbor is (11001) –Neighbor = (10001) –The best of b and Neighbor is (11001) –Neighbor = (11101) –The best of b and Neighbor is (11101) –Let b = (11101), goto third cycle

12. Deterministic Next Ascent Hill-Climbing Continuing (third cycle) –Let b = (11101) –Neighbor = (01101) –The best of b and Neighbor is (11101) –Neighbor = (10101) –The best of b and Neighbor is (11101) –Neighbor = (11001) –The best of b and Neighbor is (11101) –Neighbor = (11111) –The best of b and Neighbor is (11111) –Let b = (11111), goto fourth cycle

13. Deterministic Next Ascent Hill-Climbing Continuing (fourth cycle) –Let b = (11111) –Neighbor = (01111) –The best of b and Neighbor is (11111) –Neighbor = (10111) –The best of b and Neighbor is (11111) –Neighbor = (11101) –The best of b and Neighbor is (11111) –Neighbor = (11110) –The best of b and Neighbor is (11111) –Let b = (11111), Terminate Search

14. Stochastic Next Ascent Hill-Climbing In order to do deterministic hill-climbing one must also specify the order in which the neighbors are to be generated and evaluated. This order can dramatically affect the performance of hill-climbing. In a Stochastic Next Ascent Hill-Climber, neighbors are randomly selected from the neighborhood.

15. Some Hill-Climbing Questions Of the two type of hill-climbers which will perform better on F2? For what types of problems will Steepest Ascent Hill-Climbing outperform Next Ascent Hill-Climbing? For what types of problems will Next Ascent Hill-Climbing outperform Steepest Ascent Hill-Climbing?

16. Societies of Hill-Climbers A Society of Hill-Climbers (SoHCs) as proposed by Sebag & Schoenauer (1997) can be seen as being composed of: –A (μ+λ)-EC, and –A repoussoir (repeller) which is an adaptive external memory structure that is used to ‘push’ the search away from known sub- optimal solutions.

17. Societies of Hill-Climbers Societies of Hill-Climbers, since they evolve an adaptive external memory structure, can be consider a swarm (Swarm Intelligence Method) similar to: –Particle Swarm Optimization [P-Vectors], –Ant Colony Optimization [Pheromone Matrix], –Cultural Algorithms [Belief Space], –Tabu Search [Tabu List]

18. Societies of Hill-Climbers Sebag & Schoenaur describe two types of Hill-Climber Societies: –Natural Societies μ represents the number of hill-climbers (candidate solutions), λ represents the number of offspring created (λ/μ offspring for each parent), and M represents the number of bits to be mutated to create an offspring.

19. Societies of Hill-Climbers –Historical Societies R the percentage of the worst fit CSs that will be used to create the AvgWorst in order to update the repeller, α represents the decay rate for the components of R, and T is the tournament size for selecting a variable (gene) of the parent to mutate. The only sensitive parameter for historical societies is M.

20. Societies of Hill-Climbers Historical Societies Repeller is a real-valued vector of n where n is the number of variables. Each component of Repeller is initially assigned a value of 0.5. The Repeller is updated as follows: Repeller = (1-α) Repeller + αAvgWorst.

21. Societies of Hill-Climbers: Historical Societies An Example SoHC Procedure SoHC{ t = 0; Initialize Pop(t); Evaluate Pop(t); Intialize Repoussor(t); While (Not Done) { Offspring(t) = Procreate(Pop(t),Repoussor(t)); Evaluate(Offspring(t)); Pop(t+1) = Best_μ_of(Pop(t),Offspring(t)); AvgWorst = Worst_R%_of(Pop(t),Offspring(t)); Repoussoir(t+1) = (1-α)Repoussoir(t) + αAvgWorst; t = t + 1; }

22. Societies of Hill-Climbers: Historical Societies Let’s try an example by hand for F1! Pop(0): Person 1: 01010 Fit: 100 Person 2: 00011 Fit: 9 Offspring(0): Child 1: 00010 Fit: 4 Child 2: 01110 Fit: 196 Child 3: 10011 Fit: 361 Child 4: 00001 Fit: 1

23. Societies of Hill-Climbers: Historical Societies Person 2: 00011 Fit: 9 Child 1: 00010 Fit: 4 Child 4: 00001 Fit: 1 ---------------------------------- AvgWorst: Repoussoir(0): Repoussoir(1):

24. Societies of Hill-Climbers: Historical Societies Repoussoir(1): Pop(1): Person 1: 01110 Fit: 196 Person 2: 10011 Fit: 361 Offspring(1): Child 1: 11011 Fit: 729 Child 2: 10111 Fit: 529 Child 3: 11110 Fit: 900 Child 4: 01111 Fit: 225

25. Societies of Hill-Climbers: Historical Societies Person 1: 01110 Fit: 196 Person 2: 10011 Fit: 361 Child 4: 01111 Fit: 225 ------------------------------------ AvgWorst: Repoussoir(1): Repoussoir(2):

26. Societies of Hill-Climbers: Historical Societies Repoussoir(2): Pop(2): Person 1: 11011 Fit: 729 Person 2: 11110 Fit: 900 Offspring(1): Child 1: _____ Fit: ___ Child 2: _____ Fit: ___ Child 3: _____ Fit: ___ Child 4: _____ Fit: ___

27. Societies of Hill-Climbers: Historical Societies Will the Repoussoir work all of the time? How can we develop more efficient SoHCs? What have we learned from our brief introduction to Swarm Intelligence? –Some swarms are ‘pullers’ Ant Swarms, Ant Colony & Particle Swarm Optimizers, Cultural Algorithms –Some swarms are ‘pushers’ Societies of Hill-Climbers Tabu Search Should we really rely on pushing or pulling exclusively? During what parts of the evolutionary process would ‘pushing’ be more beneficial? During what parts of the evolutionary process would ‘pulling’ be more beneficial?