1. An Evolutionary Algorithm for Neural Network Learning using Direct Encoding Paul Batchis Department of Computer Science Rutgers University

2. Introduction ● Artificial Neural Networks (ANNs) provide a practical way of representing a function of complex or noisy data. But how to train? ● Usual training method is to create a fixed network architecture with random connection weights, then use a back propagation algorithm on the training data until the desired accuracy is achieved. ● This paper investigates an alternative training method, evolution.

3. Why Evolution for ANN Training? Evolution has several potential advantages over back propagation: ● Learns architecture along with connection weights. ● Learns ANNs with large number of layers. ● Global nature of evolutionary search means it is less likely to get stuck converging to local maxima, and more likely to find the global maximum.

4. The Algorithm ● The training algorithm works like a genetic algorithm, where the genetic code is an ANN. This is called direct encoding. ● For each generation, a population of ANNs are tested against each other for fitness in a given classification problem. ● The best half are carried into the next generation along with a mutated copy of each. ● Crossover is not used, due to the Competing Conventions Problem.

5. Encoding the ANN ● Feed-forward network of threshold units. ● Allows for random mutations: – Adjustments to connection weights. – Removal of connections. – Addition of new nodes and connections. ● Each unit is created with an immutable real- valued “level”, to allow for arbitrary feed-forward design.

6. Test Results

7. Conclusions ● In principle, evolution offers several (previously mentioned) advantages over back propagation for ANN learning. ● Experimental results indicate this evolutionary algorithm has superior performance over back propagation. ● Areas of future investigation are ways of further incorporating evolution into ANN learning: use of crossover, evolving ANN sub-components, evolving rules for network mutation.