Particle Swarm Optimization † Spencer Vogel † This presentation contains cheesy graphics and animations and they will be awesome

Particle Swarm Basics

✘ Each particle is trying to find the global optimum ✘ Each particle is moving ✘ Each particle remembers where it’s local optima was Basic Idea

✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it Neighborhoods

Social Geographical

✘ Each particle in the swarm cooperates with all of the other particles  Each particle has a neighborhood associated with it  Each particle knows the fitness of all other particles in it’s neighborhood ҂ The best position from it’s neighborhood is used to adjust the particle’s velocity Neighborhoods

✘ As each particle has to move, it has to move to a new position at each time step  It does this by adjusting it’s velocity  It’s velocity is based off a random weight of: ҂ It’s current velocity ҂ A random portion in the direction of it’s personal optimal fitness ҂ A random portion of the direction of the neighborhood optimal fitness Particle Action

Swarm Dynamics Current Motion Influence Swarm Influence Particle Memory Influence Resulting Vector Projected Motion

Common Test Functions Griewank Rastrigin Rosenbrock Sinenvsin

Parallelization

✘ Distributes all data to be processed amongst all processors ✘ Allows individual processors to complete all tasks for one set of data ✘ Applied widely to genetic algorithms Data Parallelism

✘ All particles are sent to the environment and the algorithm waits for all analysis to complete before continuing ✘ Usually results in poor efficiency as you end up waiting for some tasks to complete before updating the particles Synchronous

Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current State

✘ Separates updates actions for each point and the swarm ✘ Updates point actions after each point is analyzed, and swarm actions at the end of each iteration Asynchronous

Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current Fitness Check Convergence Update Particle Position Update Particle Velocity Report Results Evaluate Current State

✘ Master  Initialize all optimization parameters, positions, and velocities  Holds a queue of particles for slave processors to evaluate  Updates global positions and velocities  Sends the position of the next particle in the que to an available slave processor  Receives cost function values from slave processors  Checks convergence Asynchronous master/slave paradigm

✘ Slave  Receives particle position from master processor  Evaluates the fitness function at the given particle  Updates particle position  Updates particle velocity  Sends a cost function to the master processor Asynchronous master/slave paradigm

Asynchronous vs Synchronous (a) Asynchronous Implementation (b) Synchronous Implementation

✘ H(x) represent different fitness functions ✘ Values are the mean (standard deviation) of function evaluations required to reach the final solution Performance Comparison

Efficiency Comparison

Thread communication Loosely correlated parameters Strongly correlated parameters

Thread Communication Unknown parameter correlation

Performance Comparison Communication style 1 with Rosenbrok Communication style 1 with Rastrigin Communication style 2 with Griewank Communication style 3

My Project

✘ Written in C++ using OpenGL and boost  Uses a combination of glew and freeglut ✘ Allows variable initialization via a configuration file ✘ Particle’s “best fitness” value decreases over time ✘ Provides visual feedback of particles in the search space My Implementation

✘ Parameter tuning ✘ Getting social neighborhoods to set up correctly ✘ The swarm currently has moments of “convergence” followed by moments of “oscillation”  Bug not currently fixed Challenges

Things to still implement

✘ All points seek out the green dot (user controllable via mouse) ✘ 3 axis search ✘ Individual particle “depth” represented by red – purple – blue dots  Red represents + z axis  Blue represents – z-axis  Purple shades represent a neutral z axis Live Demo

Questions?

✘ Chu, Shu-Chuan, John F. Roddick, and Jeng-Shyang Pan. "A Parallel Particle Swarm Optimization Algorithm With Communication Strategies." Harbin Institute of Technology. 13 Sept Web. 15 Oct ✘ Venter, Gerhard, and Jaroslaw Sobieszczanski-Sobiesk. "A Parallel Particle Swarm Optimization Algorithm Accelerated by Asynchronous Evaluations." EngOpt. 30 May Web. 10 Nov ✘ Pinto, Andry, Hugo Alves, Inês Domingues, Luís Rocha, and Susana Cruz. "The Particle Swarm Optimization Algorithm." University of Porto Web. 15 Oct ✘ "Particle Swarm Optimization Demo." YouTube. YouTube, 8 Feb Web. 15 Oct References