1. Regrouping Particle Swarm Optimization: A New Global Optimization Algorithm with Improved Performance Consistency Across Benchmarks
George I. Evers
Advisor: Dr. Mounir Ben Ghalia
Electrical Engineering Department
The University of Texas – Pan American

2. Outline From Physics to PSO Visual Illustration of Stagnation
& the Regrouping Method
III. RegPSO Formulation
IV. Graph of Solution Quality
V. Statistical Comparison with Basic PSO
VI. Summary
VII. Future Work

3. How PSO Derives from Standard Physics Equations
From Physics to PSO

4. From Physics to PSO Displacement Formula of Physics:
assuming constant acceleration over the time period

5. From Physics to PSO Iterative Version:
Using 1 time unit between iterations:
t = (k + 1) – k = 1 iteration per update
t2 = 1 iteration2 per update
For practical purposes, t drops out of the
equation.

6. From Physics to PSO Subscript “i” Used for Particle Index:
(All particles follow the same rule.)

7. From Physics to PSO
Particles are physical conceptualizations accelerating according to social and cognitive influences.

8. From Physics to PSO
Cognitive Acceleration The cognitive acceleration is proportional to (i) the distance, , of a particle from its personal best, and (ii) the cognitive acceleration coefficient, .

9. From Physics to PSO
Social Acceleration The social acceleration is proportional to (i) the distance, , of a particle from its global best, and (ii) the social acceleration coefficient, .

10. From Physics to PSO
Total Acceleration The overall acceleration can therefore be written as Substitution then leads from to

11. From Physics to PSO
Total Acceleration In place of constant , a pseudo-random number with an expected value of is generated per dimension to add an element of stochasm to the algorithm. In this manner becomes

12. From Physics to PSO Simulating Friction
To prevent velocities from growing out of control, only a fraction of the velocity is carried over to the next iteration. This is accomplished by introducing an inertia weight, , which is set less than 1.
In this manner
becomes

13. From Physics to PSO
Velocity and Position Updates The previous equation is separated into two more succinct equations, allowing velocities and positions to be recorded and analyzed separately.

14. The Main Obstacle: Premature Convergence/ Stagnation
II. Visual Example of Stagnation
& The Regrouping Method

15. Rastrigin Benchmark Used to Illustrate Stagnation

16. Swarm Initialization
Velocities are randomly initialized to lie between [-vmax, vmax] per dimension.
Particles 1 and 3 are selected to visually illustrate how velocities and positions are updated.

17. First Velocity Updates
The velocities of the previous iteration are reduced by the inertia weight to produce the inertial components in red.
First Velocity Updates
Particle 6 found the best function value, which it communicates to its friends. Social acceleration is shown in blue.

18. First Position Updates
Particles moved along the resultant velocity vectors to their new positions (Page Up, Page Down to see this).
First Position Updates
Particle 1 found a new personal best, but particle 3 did not.

19. Second Velocity Updates
Particle 3 is pulled cognitively toward its personal best and socially toward the global best while experiencing inertia.
Second Velocity Updates
Particle 1 is at its personal best, so it experiences only inertia and social acceleration.

20. Second Position Updates
Particles again moved along their resultant velocity vectors to new positions (Page Up, Page Down to see this).
Second Position Updates
Particle 3 found a new personal best, while particle 1 did not.

21. Swarm Snapshots
Having seen how particles iteratively update their positions, the following slides show the swarm state each 10 iterations to track the progression from initialization to eventual solution.

22. Swarm Initialization at Iteration 0
Rewinding to monitor collective behavior from the beginning….
Swarm Initialization at Iteration 0
Particles are randomly initialized within the original initialization space.

23. Swarm Collapsing at Iteration 10
Particles are converging to a local minimizer near [2,0] via their attraction to the global best in that vicinity.

24. Exploratory Momenta at Iteration 20
Momenta and cognitive accelerations keep particles searching prior to settling down.

25. Convergence in Progress at Iteration 30
Personal bests move closer to the global best and momenta wane as no better global best is found. Particles continue converging to the local minimizer near [2,0].

26. Momenta Waning at Iteration 40
Momenta continue to wane as particles are repeatedly pulled toward (a) the global best very near [2,0] and (b) their own personal bests in the same vicinity.

27. Mostly Converged at Iteration 50
Most particles are improving their approximation of the local minimizer found, while two particles still have some momenta.

28. Momenta Waning at Iteration 60
The final two particles are collapsing upon the global best while the remaining particles are refining the solution.

29. Momenta Waning at Iteration 70
All particles are in the same general vicinity.

30. Cognitive Acceleration at Iteration 80
At least one particle still has some exploratory momentum.

31. Premature Convergence Detected at Iteration 102
All particles have converged to within 0.011% of the diameter of the initialization space. It is important to allow particles to refine each solution before regrouping since they have no prior knowledge of which solution is the global minimizer.

32. Options for Dealing with Stagnation
Terminate the search rather than wasting computations while stagnated.
Allow the search to continue and hope for solution refinement.
Restart particles from new positions and look for a better solution.
Somehow flag solutions already found so that each restart finds new solutions, and continue restarting until no better solutions are found.
Reinvigorate the swarm with diversity to continue the current search for the global minimizer.

33. “Regrouping” Definition
Regroup: “to reorganize (as after a setback) for renewed activity” – Merriam Webster’s online dictionary

34. Regrouping at Iteration 103
Regrouping is more efficient than restarting on the original initialization space.

35. Exploration at Iteration 113
“Gbest” PSO continues as usual within the new regrouping space. Particles move toward the global best with new momenta, personal bests, and positions/perspectives.

36. Swarm Migration at Iteration 123
The swarm is migrating toward a better region discovered by an exploring particle near [1,0].

37. Differences of Opinion at Iteration 133
Some particles are refining a local minimizer near [1,0] while others continue exploring in the vicinity.

38. Solution Comparison at Iteration 143
Cognition pulls some particles back to the local well containing a local minimizer near [1, 0].

39. Solution Comparison at Iteration 153
Cognition and momenta keep particles moving as momenta wane.

40. Unconvinced of Optimality on Horizontal Dimension at Iteration 163
There is still some uncertainty on the horizontal dimension.

41. New Well Agreed Upon at Iteration 173
All particles agree that the new well is better than the previous.

42. Waning Momenta at Iteration 183
Momenta wane.

43. Premature Convergence Detected Again at Iteration 219
Regrouping improved the function value from approximately 4 to approximately 1, and premature convergence is detected again.

44. Swarm Regrouped Again at Iteration 220
The swarm is regrouped a second time.

45. Best Well Found at Iteration 230
The well containing the global minimizer is discovered.

46. Swarm Migration at Iteration 240
The swarm migrates to the newly found well.

47. Convergence at Iteration 250
Particles swarm to the newly found well due to its higher quality minimizer.

48. Cognition at Iteration 260
Momenta carry particles beyond the well.

49. Convergence at Iteration 270
Solution refinement of the global minimizer is in progress.

50. Regrouping PSO (RegPSO) Formulation
III. RegPSO Formula

51. Regrouping PSO (RegPSO) Detection of Premature Convergence

52. Regrouping PSO (RegPSO) Regrouping the Swarm

53. Regrouping PSO (RegPSO) High-Level Pseudo Code
Do Run Gbest PSO until premature convergence. Regroup the swarm. Re-calculate the velocity clamping value based on the range of the new initialization space. Re-initialize velocities. Re-initialize personal bests. Remember the global best. Until Search Termination

54. Effectiveness of RegPSO Demonstrated Graphically
IV. Graphical Comparison of Mean Function Values

56. Effectiveness of RegPSO Demonstrated Statistically
V. Statistical Comparison

57. Regrouping PSO (RegPSO) Compared to Gbest, Lbest PSO

58. Summary
By regrouping the swarm within an efficiently sized regrouping space when premature convergence is detected, RegPSO considerably improves performance consistency, as demonstrated with a suite of popular benchmarks.

59. Future Work Theoretical Improvements
Give the algorithm the ability to progress from regrouping to a solution refinement phase.
Testing
NP hard problems
Applications to real-world problems