1. Institute of Intelligent Power Electronics – IPE Page1 Introduction to Basics of Genetic Algorithms Docent Xiao-Zhi Gao Department of Electrical Engineering and Automation xiao-zhi.gao@aalto.fi

2. Institute of Intelligent Power Electronics – IPE Page2 Background of Genetic Algorithms Genetic Algorithms (GA) are optimization methods based on ideas of natural selection and evolutionary processes [Holland 75] GA’s unique characteristics –Derivative free –Stochastic/Random and Global optimization –Parallel search GA are applied in optimization of complex systems with little information

3. Institute of Intelligent Power Electronics – IPE Page3 Charles Darwin (1809-1882)

4. Institute of Intelligent Power Electronics – IPE Page4 ’The Origin of Species’ by Darwin [1859]

5. Institute of Intelligent Power Electronics – IPE Page5 Species Evolution and Natural Selection

6. Institute of Intelligent Power Electronics – IPE Page6 Evolution of Human Beings?

7. Institute of Intelligent Power Electronics – IPE Page7 Evolution of Human Beings?

8. Institute of Intelligent Power Electronics – IPE Page8 Darwinian Paradigm Reproduction Competition SelectionSurvive

9. Institute of Intelligent Power Electronics – IPE Page9 John Henry Holland: Father of Genetic Algorithms –John Henry Holland (born on 2 February 1929) is a Professor of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor –Book ‘Adaptation in Natural and Artificial Systems’ (1975) –http://en.wikipedia.org/wiki/Jo hn_Henry_Holland

10. Institute of Intelligent Power Electronics – IPE Page10 Conceptual Genetic Algorithms

11. Institute of Intelligent Power Electronics – IPE Page11 Procedure of Genetic Algorithms 1. Encode some possible/random solutions into bit strings (chromosomes), e.g, 1010011, and construct a population (group of chromosomes) –Float (real-valued) encoding is also possible 2. Evaluate these chromosomes, and calculate their fitnesses (i.e., how good they are) –Based on criteria function (fitness function ) 3. Create new chromosomes by using mutation and crossover operators on the existing chromosomes –Mutation and crossover probabilities

12. Institute of Intelligent Power Electronics – IPE Page12 Chromosome, Genes and Genomes

13. Institute of Intelligent Power Electronics – IPE Page13 Historical Terms in GA Binary strings: Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state machines: Evolutionary Programming LISP trees: Genetic Programming etc.

14. Institute of Intelligent Power Electronics – IPE Page14 Mutation Operator Mutation Probability

15. Institute of Intelligent Power Electronics – IPE Page15 Crossover Operator Crossover Probability Two-Point Crossover One-Point Crossover

16. Institute of Intelligent Power Electronics – IPE Page16 Procedure of Genetic Algorithms 4. Evaluate new chromosomes, and calculate their fitnesses 5. Select the mostly fitted chromosomes using roulette selection scheme –Retain size of population fixed –Better fitted chromosomes must have higher possibilities of survival –Chromosome with the best fitness is always kept 6. Repeat above steps until a pre-set criteria is met

17. Institute of Intelligent Power Electronics – IPE Page17 Roulette Selection Scheme

18. Institute of Intelligent Power Electronics – IPE Page18 Flow Chart of Genetic Algorithms

19. Institute of Intelligent Power Electronics – IPE Page19 Structure of Genetic Algorithms Best Chromosomes

20. Institute of Intelligent Power Electronics – IPE Page20 Typical Behavior of GA

21. Institute of Intelligent Power Electronics – IPE Page21 Typical Run: Progression of Fitness

22. Institute of Intelligent Power Electronics – IPE Page22 GA as Problem Solvers: Goldberg’s 1989 View

23. Institute of Intelligent Power Electronics – IPE Page23 GA as Problem Solvers: Michalewicz’ 1996 View

24. Institute of Intelligent Power Electronics – IPE Page24 An Example: Traveling Salesman Problem (TSP) The Traveling Salesman Problem: Find a tour of a given set of cities so that –each city is visited only once –the total distance traveled is minimized

25. Institute of Intelligent Power Electronics – IPE Page25 Representation

26. Institute of Intelligent Power Electronics – IPE Page26 Crossover

27. Institute of Intelligent Power Electronics – IPE Page27 Mutation

28. Institute of Intelligent Power Electronics – IPE Page28 TSP Example: 30 Cities

29. Institute of Intelligent Power Electronics – IPE Page29 Solution i (Distance = 941)

30. Institute of Intelligent Power Electronics – IPE Page30 Solution j (Distance = 800)

31. Institute of Intelligent Power Electronics – IPE Page31 Solution k (Distance = 652)

32. Institute of Intelligent Power Electronics – IPE Page32 Best Solution (Distance = 420)

33. Institute of Intelligent Power Electronics – IPE Page33 Overview of Performance

34. Institute of Intelligent Power Electronics – IPE Page34 An Example: Maximization of ’Peak’ Function Using GA The ’peak’ function is defined as: It has two inputs: x and y The ’peak’ function is a highly nonlinear function, and has multiple local minima Maximization of ’peak’ function is hard

35. Institute of Intelligent Power Electronics – IPE Page35 ’Peak’ Function

36. Institute of Intelligent Power Electronics – IPE Page36 Initial Populations

37. Institute of Intelligent Power Electronics – IPE Page37 Populations After Five Generations

38. Institute of Intelligent Power Electronics – IPE Page38 Populations After 10 Generations

39. Institute of Intelligent Power Electronics – IPE Page39 Performance of GA Optimization

40. Institute of Intelligent Power Electronics – IPE Page40 Some GA Applications

41. Institute of Intelligent Power Electronics – IPE Page41 Reference David Fogel (born in 1964) is a pioneer in evolutionary computation (Ph.D. from University of California, San Diego in 1992). He is currently the CEO of the Natural Selection, Inc.

42. Institute of Intelligent Power Electronics – IPE Page42 Reference Evolutionary Computation: Toward a New Philosophy of Machine Intelligence by David Fogel, 2005

43. Institute of Intelligent Power Electronics – IPE Page43 Fusion of GA with Neural Networks GA can be employed to optimize the parameters and structures of neural networks –Parameter optimization –Structure optimization (how to encode?) GA-based optimization is a universal solution –Independent of problems to be solved

44. Institute of Intelligent Power Electronics – IPE Page44 An Example: GA-based Optimization of Elman Neural Network [Gao 00] Initial outputs of context nodes in Elman neural network play an important role in network prediction accuracy Hybrid training of Elman neural network consists of two parts –Gradient descent algorithm for weights –Genetic algorithms for initial context nodes outputs

45. Institute of Intelligent Power Electronics – IPE Page45 Elman Neural Network

46. Institute of Intelligent Power Electronics – IPE Page46 Training Procedure of Elman Neural Network with BP Learning

47. Institute of Intelligent Power Electronics – IPE Page47 GA-involved Optimization Process of Initial Context Nodes Outputs PE: Prediction Error

48. Institute of Intelligent Power Electronics – IPE Page48 Motor Fault Diagnosis Using Elman Neural Network with GA-aided Training

49. Institute of Intelligent Power Electronics – IPE Page49 Conclusions Basic knowledge and typical applications of GA are introduced –Numerous variants of GA exist [Fogel 06] GA can be used to handle data mining GA-based optimization is a quite time consuming procedure –Suitable for simulations on high-speed parallel processing computers –Fast GA are being investigated