1. Particle Swarm Optimization † Spencer Vogel † This presentation contains cheesy graphics and animations and they will be awesome

2. Origins

3. ✘ Inspired from the social behavior of bird flocks, fish swarms, and insect behavior Nature

4. ✘ Separation  Avoid crowding local flock mates ✘ Alignment  Move towards local average heading ✘ Cohesion  Move towards local position average Three Behaviors

5. Problem Definition

6. ✘ You’re given a search space Basic Concept

7. Basic Idea

8. ✘ You’re given a search space ✘ Within this space, you wish to find an optimum Basic Concept

9. Basic Idea

10. Particle Swarm Basics

11. ✘ Each particle is trying to find the global optimum Basic Idea

12. Initial Positions and Velocities

13. ✘ Each particle is trying to find the global optimum ✘ Each particle starts with an initial speed Basic Idea

14. Initial Positions and Velocities

15. ✘ Each particle is trying to find the global optimum ✘ Each particle is moving ✘ Each particle remembers where it’s local optima was Basic Idea

16. ✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it Neighborhoods

17. Social Geograp hical

18. Neighborhoods Global

19. Neighborhoods Virtual circle 1 5 7 6 4 3 8 2 Particle 1’s 3- neighbourhood Particle 4’s 5- neighbourhood

20. ✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it  Each particle knows the fitness of all other particles in it’s neighborhood ҂ The best position from it’s neighborhood is used to adjust the particle’s velocity Neighborhoods

21. ✘ As each particle has to move, it has to move to a new position at each time step  It does this by adjusting it’s velocity  It’s velocity is based off a random weight of: ҂ It’s current velocity ҂ A random portion in the direction of it’s personal optimal fitness ҂ A random portion of the direction of the neighborhood optimal fitness Particle Action

22. Swarm Dynamics Current Motion Influence Swarm Influence Particle Memory Influence Resulting Vector Projected Motion

23. Parameter Selection

24. ✘ Two common methods  Random  Pre-seeded  Even Distribution Initialization

25. ✘ Number of particles  Swarm size ✘ C1  Importance of personal best fitness ✘ C2  Importance of neighborhood best fitness ✘ V max  Limit on velocity Swarm Parameters

26. ✘ Number of Particles  10-50 is usually sufficient ✘ C1, C2  Traditionally, C1+C2=4 for empiricism ✘ V max  Often problem dependent  Too low and the program converges slowly  Too high and it becomes unstable Parameter Selection

27. Common Test Functions Griewank Rastrigin Rosenbrock Sinenvsin

28. Live Demo! http://www.macs.hw.ac.uk/~dwcorne/mypages/apps/pso.html

29. Demonstration of Large Swarm Dynamics

30. X

31. Common Variations

32. Unbiased Random Proximity Current Motion Influence Swarm Influence Particle Memory Influence Projected Motion

33. Adaptive Swarm Size In the case of a steady state or a declining state In the case of a steady rising graph, the program will seek out the worst fit particles, and choose to remove them to reduce computations the program will seek out the best fit particles and choose to make more particles nearby

34. ✘ The better an individual particle’s fitness is, the more it tends to follow it’s own parameters ✘ The better an individual neighbor, the more it tends towards the neighbor Adaptive Coefficients

35. Algorithm Characteristics

36. ✘ Insensitive to variable scaling ✘ Simple Implementation ✘ Easily Parallelized ✘ Derivative Free ✘ Very few customized parameters ✘ Very efficient global search Advantages

37. ✘ Tendency to converge prematurely in a mid-optimum point ✘ Slow convergence in a refined search stage ✘ Often plagued by high computational costs Disadvantages

38. ✘ Highly parallelizable  Each particle essentially preforms the same type of computation  Often extremely load balanced  Asynchronous implementations are valuable for rea-life problems Parallelization

39. Questions?