GENETIC ALGORITHMS & MACHINE LEARNING CS1111 GENETIC ALGORITHMS & MACHINE LEARNING

INTRODUCTION TO GENETIC ALGORITHMS UNIT-I   INTRODUCTION TO GENETIC ALGORITHMS

Introduction Genetic Algorithms (GAs) are adaptive heuristic search algorithm based on the evolutionary ideas of natural selection and genetics. As such they represent an intelligent exploitation of a random search used to solve optimization problems.

Introduction (cont..) The basic techniques of the GAs are designed to simulate processes in natural systems necessary for evolution, specially those follow the principles first laid down by Charles Darwin of "survival of the fittest". Since in nature, competition among individuals for scanty resources results in the fittest individuals dominating over the weaker ones.

Classes of Search Techniques Calculus Base Techniqes Guided random search techniqes Enumerative Techniques Fibonacci DFS Dynamic Programming BFS Sort Evolutionary Algorithms Tabu Search Simulated Anealing Hill Climbing Genetic Programming Genetic Algorithms

History of GAs In 1960,I.RECHENBERG ,introduce the idea of evolutionary strategy. As early as 1962, John Holland's work on adaptive systems laid the foundation for later developments. By the 1975, the publication of the book Adaptation in Natural and Artificial Systems, by Holland and his students and colleagues. Early to mid-1980s, genetic algorithms were being applied to a broad range of subjects. In 1992 John Koza has used genetic algorithm to evolve programs to perform certain tasks. He called his method "genetic programming" (GP).

WHAT IS GA? A genetic algorithm (or GA) is a search technique used in computing to find true or approximate solutions to optimization and search problems. (GA)s are categorized as global search heuristics. (GA)s are a particular class of evolutionary algorithms that use techniques inspired by evolutionary biology such as inheritance, mutation, selection, and crossover (also called recombination). Charles Darwin describes "survival of the fittest.".

The three most important aspects of using genetic algorithms are: (1) definition of the objective function, (2) definition and implementation of the genetic representation, and (3) definition and implementation of the genetic operators. Once these three have been defined, the generic genetic algorithm should work fairly well.

Why Genetic Algorithms? It is better than conventional AI in that it is more robust. Unlike older AI systems, they do not break easily even if the inputs changed slightly, or in the presence of reasonable noise. Also, in searching a large state-space, multi-modal state-space, or n-dimensional surface, a genetic algorithm may offer significant benefits over more typical search of optimization techniques. (linear programming, heuristic, depth-first, breath-first, and praxis.)

Biological Background “Chromosomes” All living organisms consist of cells. In each cell there is the same set of chromosomes. Chromosomes are strings of DNA and serves as a model for the whole organism. A chromosome consist of genes, blocks of DNA. Each gene encodes a particular protein. Basically can be said, that each gene encodes a trait, for example color of eyes. Possible settings for a trait (e.g. blue, brown) are called alleles(Allelomorph). Each gene has its own position in the chromosome. This position is called locus. Complete set of genetic material (all chromosomes) is called genome. Particular set of genes in genome is called genotype. The genotype is with later development after birth base for the organism's phenotype, its physical and mental characteristics, such as eye color, intelligence etc

Chromosome, Genes and Genomes

Genotype and Phenotype

Basic Genetic Algorithm Start with a large “population” of randomly generated “attempted solutions” to a problem Repeatedly do the following: Evaluate each of the attempted solutions Keep a subset of these solutions (the “best” ones) Use these solutions to generate a new population Quit when you have a satisfactory solution (or you run out of time)

CREATION OF OFFSPRING After an initial population is randomly generated, the algorithm evolves the through three operators: selection which equates to survival of the fittest; crossover which represents mating between individuals;(RECOMBINATION) mutation which introduces random modifications(MODIFIED DNA) FITNESS is measured by success of organism.

Each solution can marked by its value of fitness. Search Space A population of individuals are is maintained within search space for a GA, each representing a possible solution to a given problem. Each solution can marked by its value of fitness. GA Algorithm is started with a set of solutions called populations. Solutions for one population are taken to form a new one.

A fitness score is assigned to each solution representing the abilities of an individual to `compete‘.. The GA aims to use selective `breeding' of the solutions to produce `offspring' better than the parents by combining information from the chromosomes. The new population is better than the old one.

HOW…… Individuals in the population die and are replaced by the new solutions, eventually creating a new generation once all mating opportunities in the old population have been exhausted. least fit solutions die out. New generations of solutions are produced containing, on average, more good genes than a typical solution in a previous generation. Each successive generation will contain more good `partial solutions' than previous generations. algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population.

General Algorithm for GA Selection During each successive generation, a proportion of the existing population is selected to breed a new generation. Individual solutions are selected through a fitness-based process, where fitter solutions (as measured by a fitness function) are typically more likely to be selected. Certain selection methods rate the fitness of each solution and preferentially select the best solutions. Other methods rate only a random sample of the population, as this process may be very time-consuming. Most functions are stochastic and designed so that a small proportion of less fit solutions are selected. This helps keep the diversity of the population large, preventing premature convergence on poor solutions. Popular and well-studied selection methods include roulette wheel selection and tournament selection.

Roulette wheel selection In roulette wheel selection, individuals are given a probability of being selected that is directly proportionate to their fitness. Two individuals are then chosen randomly based on these probabilities and produce offspring.

Bolzmann Selection -Simulated Annealing Tournament Selection -provides selective pressure by holding tournament competition among Nu individuals Rank selection -First rank population is taken,which having high level of fitness Steady state Selection -Bigger part of chromosomes are selected to survive next generation.

General Algorithm for GA Reproduction(selction) The next step is to generate a second generation population of solutions from those selected through genetic operators: crossover (also called recombination), and/or mutation. For each new solution to be produced, a pair of "parent" solutions is selected for breeding from the pool selected previously. By producing a "child" solution using the above methods of crossover and mutation, a new solution is created which typically shares many of the characteristics of its "parents". New parents are selected for each child, and the process continues until a new population of solutions of appropriate size is generated.

Reproduction operators Crossover Mutation

Reproduction Crossover Two parents produce two offspring There is a chance that the chromosomes of the two parents are copied unmodified as offspring There is a chance that the chromosomes of the two parents are randomly recombined (crossover) to form offspring Generally the chance of crossover is between 0.6 and 1.0 Mutation There is a chance that a gene of a child is changed randomly Generally the chance of mutation is low (e.g. 0.001)

Reproduction Operators Crossover Generating offspring from two selected parents Single point crossover Two point crossover (Multi point crossover) Uniform crossover

One-point crossover 1 Parents: 1010001110 0011010010 Randomly one position in the chromosomes is chosen Child 1 is head of chromosome of parent 1 with tail of chromosome of parent 2 Child 2 is head of 2 with tail of 1 Randomly chosen position Parents: 1010001110 0011010010 Offspring: 0101010010 0011001110

Reproduction Operators comparison Single point crossover  Cross point Two point crossover (Multi point crossover) 