Particle Swarm Optimization † Spencer Vogel † This presentation contains cheesy graphics and animations and they will be awesome

Origins

✘ Inspired from the social behavior of bird flocks, fish swarms, and insect behavior Nature

✘ Separation  Avoid crowding local flock mates ✘ Alignment  Move towards local average heading ✘ Cohesion  Move towards local position average Three Behaviors

Problem Definition

✘ You’re given a search space Basic Concept

Basic Idea

✘ You’re given a search space ✘ Within this space, you wish to find an optimum Basic Concept

Basic Idea

Particle Swarm Basics

✘ Each particle is trying to find the global optimum Basic Idea

Initial Positions and Velocities

✘ Each particle is trying to find the global optimum ✘ Each particle starts with an initial speed Basic Idea

Initial Positions and Velocities

✘ Each particle is trying to find the global optimum ✘ Each particle is moving ✘ Each particle remembers where it’s local optima was Basic Idea

✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it Neighborhoods

Social Geograp hical

Neighborhoods Global

Neighborhoods Virtual circle Particle 1’s 3- neighbourhood Particle 4’s 5- neighbourhood

✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it  Each particle knows the fitness of all other particles in it’s neighborhood ҂ The best position from it’s neighborhood is used to adjust the particle’s velocity Neighborhoods

✘ As each particle has to move, it has to move to a new position at each time step  It does this by adjusting it’s velocity  It’s velocity is based off a random weight of: ҂ It’s current velocity ҂ A random portion in the direction of it’s personal optimal fitness ҂ A random portion of the direction of the neighborhood optimal fitness Particle Action

Swarm Dynamics Current Motion Influence Swarm Influence Particle Memory Influence Resulting Vector Projected Motion

Parameter Selection

✘ Two common methods  Random  Pre-seeded  Even Distribution Initialization

✘ Number of particles  Swarm size ✘ C1  Importance of personal best fitness ✘ C2  Importance of neighborhood best fitness ✘ V max  Limit on velocity Swarm Parameters

✘ Number of Particles  is usually sufficient ✘ C1, C2  Traditionally, C1+C2=4 for empiricism ✘ V max  Often problem dependent  Too low and the program converges slowly  Too high and it becomes unstable Parameter Selection

Common Test Functions Griewank Rastrigin Rosenbrock Sinenvsin

Live Demo!

Demonstration of Large Swarm Dynamics

X

Common Variations

Unbiased Random Proximity Current Motion Influence Swarm Influence Particle Memory Influence Projected Motion

Adaptive Swarm Size In the case of a steady state or a declining state In the case of a steady rising graph, the program will seek out the worst fit particles, and choose to remove them to reduce computations the program will seek out the best fit particles and choose to make more particles nearby

✘ The better an individual particle’s fitness is, the more it tends to follow it’s own parameters ✘ The better an individual neighbor, the more it tends towards the neighbor Adaptive Coefficients

Algorithm Characteristics

✘ Insensitive to variable scaling ✘ Simple Implementation ✘ Easily Parallelized ✘ Derivative Free ✘ Very few customized parameters ✘ Very efficient global search Advantages

✘ Tendency to converge prematurely in a mid-optimum point ✘ Slow convergence in a refined search stage ✘ Often plagued by high computational costs Disadvantages

✘ Highly parallelizable  Each particle essentially preforms the same type of computation  Often extremely load balanced  Asynchronous implementations are valuable for rea-life problems Parallelization

Questions?