2. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Particle Swarm optimisation: A mini tutorial

3. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com The “inventors” (1) Russell Eberhart eberhart@engr.iupui.edu

4. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com The “inventors” (2) James Kennedy Kennedy_Jim@bls.gov

5. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Part 1: United we stand

6. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Cooperation example

7. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Initialization. Positions and velocities

8. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Neighbourhoods geograph ical social

9. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com The circular neighbourhood Virtual circle 1 5 7 6 4 3 8 2 Particle 1’s 3- neighbourho od

10. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Psychosocial compromise Here I am! The best perf. of my neighbours My best perf. x pgpg pipi v i-proximity g-proximity

11. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com The historical algorithm for each particle update the velocity then move for each component d At each time step t Randomn ess inside the loop

12. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Random proximity x pgpg pipi v i-proximity g-proximity Hyperparallelepiped => Biased

13. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Animated illustration Global optimu m

14. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Part 2: How to choose parameters The right way This way Or this way

15. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Type 1” form with Usual values:  =1  =4.1 =>  =0.73 swarm size=20 hood size=3 Non divergence criterion Global constriction coefficient

16. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com 5D complex space } } Convergence Non diverge nce A 3D section  Re(y) Re(v)

17. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Move in a 2D section (attractor)

18. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Some functions... Rosenbrock Griewank Rastrigin

19. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com... and some results Optimum=0, dimension=30 Best result after 40 000 evaluations

20. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Beat the swarm! Your current position Your best perf. Best perf. of the swarm

21. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Part 3: Beyond real numbers 0123401234 01230123 0123401234 123456123456 012012 0123401234 8 8 Bingo!

22. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Minimun requirements Comparing positions in the search space H Algebraic operators

23. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com velocity = pos_minus_pos(position 1, position 2 ) velocity = linear_combin( ,velocity 1, ,velocity 2 ) position = pos_plus_vel(position, velocity) (position,velocity) = confinement(position t+1,position t ) Pseudo code form } algebra ic operat ors =>

24. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Fifty-fifty N=100, D=20. Search space: [1,N] D 105 evaluations: 63+90+16+54+71+20+23+60+38+15 = 12+48+13+51+36+42+86+26+57+79 (=450) granularity=1

25. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Knapsack N=100, D=10, S=100, 870 evaluations: run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12, 17) run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5) granularity=1

26. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Graph Coloring Problem 1 4 5 2 1 1 5 5 3 2 2 1 1 5 5 5 0 2 0 -1 4 -3 + = pos - plus - vel

27. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com The Tireless Traveller Example of position: X=(5,3,4,1,2,6) Example of velocity: v=((5,3),(2,5),(3,1))

28. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com n1n1 n3n3 n2n2 Apple trees Swarm size=3 Best position 7 7 6 3 11 6 6 4 10 3 0 17

29. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Part 4: Some variants

30. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Unbiased random proximity x pgpg pipi v i-proximity g-proximity Hyperparallelepiped => Biased Dimension Volume Hypersphere vs hypercube

31. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Clusters and queens Each particle is weighted by its perf. Dynamic clustering Centroids = queens = temporary new “particles”

32. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Think locally, act locally (Adaptive versions)

33. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Adaptive swarm size There has been enough improvement but there has been not enough improvement although I'm the worst I'm the best I try to kill myself I try to generate a new particle

34. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Adaptive coefficients The better I am, the more I follow my own way The better is my best neighbour, the more I tend to go towards him vv rand(0… b )(p-x)

35. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Energies: classical process Rosenbrock 2D. Swarm size=20, constant coefficients

36. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Energies: adaptive process Rosenbrock 2D. Adaptive swarm size, adaptive coefficients

37. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Part 5: Real applications (hybrid) Medical diagnosis Industrial mixer Electrical generatorElectrical vehicle

38. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Real applications (stand alone) Cockshott A. R., Hartman B. E., "Improving the fermentation medium for Echinocandin B production. Part II: Particle swarm optimization", Process biochemistry, vol. 36, 2001, p. 661-669. He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE International Conference on Evolutionary Computation, Anchorage, Alaska, USA, 1998. Secrest B. R., Traveling Salesman Problem for Surveillance Mission using Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of Technology, 2001. Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for Reactive Power and Voltage Control considering Voltage Security Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.

39. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com To know more Clerc M., Kennedy J., "The Particle Swarm-Explosion, Stability, and Convergence in a Multidimensional Somplex space", IEEE Transaction on Evolutionary Computation, 2002,vol. 6, p. 58-73. Clerc M., "L'optimisation par essaim particulaire. Principes et pratique", Hermès, Techniques et Science de l'Informatique, 2002. Particle Swarm Central, http://www.particleswarm.net THE site: Self advert

40. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Appendix

41. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Canonical form    M Eigen values e 1 and e 2

42. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Constriction Constriction coefficients

43. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Convergence criterion  654321 3.5 3 2.5 2 1.5 1 0.5

44. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Magic Square (1)

45. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Magic Square (2) D=3x3, N=100 10 runs 13430 evaluations 10 solutions 55 30 68 42 49 62 56 74 23 30 61 53 89 32 23 25 51 68 27 96 39 73 40 49 62 26 74 22 70 58 40 75 35 88 5 57 50 43 67 58 55 47 52 62 4 43 51 78 75 33 64 54 88 30 50 65 68 69 42 72 64 76 43 18 25 59 32 53 17 52 24 26 80 3 30 22 72 19 11 38 64 65 28 64 63 55 39 29 74 54

46. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com Non linear system Search space [0,1] 2 1 run 143 evaluations 1 solution 10 runs 1430 evaluations 3 solutions

47. Particle Swarm optimisation 2002-04- 24 Maurice.Clerc@WriteM e.com