Particle Swarm Optimization (PSO) Algorithm and Its Application in Engineering Design Optimization School of Information Technology Indian Institute of Technology Kharagpur September 9, 2005 Sushanta Kumar Mandal Research Scholar By

Outline  Introduction to Optimization  Optimization Procedure  Different Optimization Algorithms  Different Global Optimization Algorithms  Particle Swarm Optimization (PSO) Algorithm  Application of PSO in Design Optimization Problems

Optimization As ageless as time

Calculus Maximum and minimum of a smooth function is reached at a stationary point where its gradient vanishes.

Optimization is Everywhere  The more we know about something, the more we see where optimization can be applied.  Some personal decision making - Finding fastest route home or class - Optimal allocation of time for home work - Optimal budgeting

Goal of Optimization Find values of the variables that minimize or maximize the objective function while satisfying the constraints.

Component of Optimization Problem  Objective Function: An objective function which we want to minimize or maximize.  For example, in a manufacturing process, we might want to maximize the profit or minimize the cost.  In fitting experimental data to a user-defined model, we might minimize the total deviation of observed data from predictions based on the model.  In designing an inductor, we might want to maximize the Quality Factor and minimize the area.

Component of Optimization Problem  Design Variables: A set of unknowns or variables which affect the value of the objective function.  In the manufacturing problem, the variables might include the amounts of different resources used or the time spent on each activity.  In fitting-the-data problem, the unknowns are the parameters that define the model.  In the inductor design problem, the variables used define the layout geometry of the panel.

Component of Optimization Problem  Constraints : A set of constraints that allow the unknowns to take on certain values but exclude others.  For the manufacturing problem, it does not make sense to spend a negative amount of time on any activity, so we constrain all the "time" variables to be non-negative.  In the inductor design problem, we would probably want to limit the upper and lower value of layout parameters and to target an inductance value within the tolerance level.

Are All these ingredients necessary?  Almost all optimization problems have objective function.  No objective function. In some cases (for example, design of integrated circuit layouts), the goal is to find a set of variables that satisfies the constraints of the model. The user does not particularly want to optimize anything so there is no reason to define an objective function. This type of problems is usually called a feasibility problem.

Are All these ingredients necessary?  Variables are essential. If there are no variables, we cannot define the objective function and the problem constraints.  Constraints are not essential. In fact, the field of unconstrained optimization is a large and important one for which a lot of algorithms and software are available. It's been argued that almost all problems really do have constraints.

What We Need for Optimization  Models: Modeling is the process of identifying objective function, variables and constraints. The goal of models is “insight” not the numbers. A good mathematical model of the optimization problem is needed.  Algorithms: Typically, an interesting model is too complicated to be able to solve in with paper and pencil. An effective and reliable numerical algorithm is needed to solve the problem. There is no universal optimization algorithm. Algorithm should have robustness (good performance for a wide class of problems), efficiency (not too much computer time) and accuracy (can identify the error)

Flowchart of Optimal Design Procedure Need for optimization Choose design variables Formulate constraints Formulate objective function Set up variable bounds Select an optimization algorithm Obtain solution(s)

Mathematical Formulation of Optimization Problems

Constraints  Inequality constraints: x 1 2 – x 2 2  0  Equality constraints: x 1 = 2

Variable Bounds  Maximum and minimum bounds on each design variable.  Without variable bounds the constraints completely surround the feasible region.  Variable bounds are used to confine the search algorithm within these bounds.  Ex:

Classification of Optimization Methods  Single variable  Multi-variable  Constrained  Non-constrained  Single objective  Multi-objective  Linear  Non-linear

Classifications of Optimization Methods

Local and Global Optimizers  A local minimizer, x B *, of the region B, is defined so that: f(x B * )  f(x),  x  B.  Ex: Gradient based search methods, Newton-Rapson algorithms, Steepest Decent, Conjugate-Gradient algorithms, Levenberg-Marquardt algorithm etc.  Shortcomings: 1) One requires an initial guess to start with. 2) Convergence to an optimal solution depends on the chosen initial guess. 3) Most algorithms tend to get stuck to a sub-optimal solution. 4) An algorithm efficient in solving one optimization problem may not be efficient in solving another one. 5) These are useful over a relatively narrow range.

Local and Global Optimizers  The global optimizer, x *, is defined so that f(x * )  f(x),  x  S where S is the search space.  Ex. Simulated Annealing algorithm, Genetic Algorithm, Ant Colony, Geometric Programming, Particle Swarm Optimization etc.

Local and Global Optimizers

Particle Swarm Optimization  Evolutionary computational technique based on the movement and intelligence of swarms looking for the most fertile feeding location  It was developed in 1995 by James Kennedy and Russell Eberhart  Simple algorithm, easy to implement and few parameters to adjust mainly the velocity  A “swarm” is an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Continued…  It uses a number of agents (particles) that constitute a swarm moving around in the search space looking for the best solution.  Each particle is treated as a point in a D- dimensional space which adjusts its “flying” according to its own flying experience as well as the flying experience of other particles  Each particle keeps track of its coordinates in the problem space which are associated with the best solution (fitness) that has achieved so far. This value is called pbest.

Continued…  Another best value that is tracked by the PSO is the best value obtained so far by any particle in the neighbors of the particle. This value is called gbest.  The PSO concept consists of changing the velocity(or accelerating) of each particle toward its pbest and the gbest position at each time step.

Continued… Each particle tries to modify its current position and velocity according to the distance between its current position and pbest, and the distance between its current position and gbest. CurrentPosition[n+1] = CurrentPosition[n] + v[n+1] current position[n+1]: position of particle at n+1th iteration current position[n]: position of particle at nth iteration v[n+1] : particle velocity at n+1th iteration v n+1 : Velocity of particle at n+1 th iteration V n : Velocity of particle at nth iteration c 1 : acceleration factor related to gbest c 2 : acceleration factor related to lbest rand1( ): random number between 0 and 1 rand2( ): random number between 0 and 1 gbest: gbest position of swarm pbest: pbest position of particle

PSO Algorithm For each particle Initialize particle with feasible random number END Do For each particle Calculate the fitness value If the fitness value is better than the best fitness value (pbest) in history Set current value as the new pbest End Choose the particle with the best fitness value of all the particles as the gbest For each particle Calculate particle velocity according to velocity update equation Update particle position according to position update equation End While maximum iterations or minimum error criteria is not attained

gbest and lbest  global version: vx[ ][ ] = vx[ ][ ] + 2*rand( )*(pbest[ ][ ] – presentx[ ][ ]) +2*rand( )*(pbestx[ ][gbest] – presentx[ ][ ])  local version: vx[ ][ ] = vx[ ][ ] + 2*rand( )*(pbest[ ][ ] – presebtx[ ][ ]) + 2*rand( )*(pbestx[ ][lbest] – presentx[ ][ ])

Particle Swarm Optimization: Swarm Topology  In PSO, there have been two basic topologies used in the literature –Ring Topology (neighborhood of 3) –Star Topology (global neighborhood) I 4 I 0 I 1 I 2 I 3 I 4 I 0 I 1 I 2 I 3

PSO Parameters: Velocity  An important parameter in PSO; typically the only one adjusted  Calmps particles’ velocities on each dimenson  Determines “fineness” with which regions are searched  If too high, can fly past optimal solutions  If too low, can get stuck in local minima

Flow Chart for Extraction Procedure using PSO

Comparison of Genetic Algorithm and PSO Tested in a MATLAB Program, P4 1.7GHz CPU, 256M RAM. No. of particle/ population size = 100 No. of simulation runs: Crossover Prob.= 0.9 Mutation Prob. =.01

Model Fitting

Inductor Optimization

Constrained PSO Optimization

Thank You