The Evolution of Learning Algorithms for Artificial Neural Networks

Slides:

Advertisements

Similar presentations

Genetic Algorithms in Problem Solving EVOLVING COMPUTER PROGRAMS (1) t Evolving Lisp Programs Keplers Third Law: P 2 = cA 3 PROGRAM ORBITAL_PERIORD.

Advertisements

Pattern Association.

G53MLE | Machine Learning | Dr Guoping Qiu

Ch. Eick: More on Machine Learning & Neural Networks Different Forms of Learning: –Learning agent receives feedback with respect to its actions (e.g. using.

EA, neural networks & fuzzy systems Michael J. Watts

Institute of Intelligent Power Electronics – IPE Page1 Introduction to Basics of Genetic Algorithms Docent Xiao-Zhi Gao Department of Electrical Engineering.

Tetris AI 팀원 김유섭 ( ) 류동균 ( ) 임성훈 ( )

Searching for Single Top Using Decision Trees G. Watts (UW) For the DØ Collaboration 5/13/2005 – APSNW Particles I.

Parallelized Evolution System Onur Soysal, Erkin Bahçeci Erol Şahin Dept. of Computer Engineering Middle East Technical University.

Data Mining Techniques Outline

Genetic algorithms for neural networks An introduction.

1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.

Artificial Neural Networks Artificial Neural Networks are (among other things) another technique for supervised learning k-Nearest Neighbor Decision Tree.

Neural Networks. R & G Chapter Feed-Forward Neural Networks otherwise known as The Multi-layer Perceptron or The Back-Propagation Neural Network.

Machine Learning for Information Retrieval: Neural Networks, Symbolic Learning, and Genetic Algorithms Prerak Sanghvi Paper by: Hsinchun Chen Artificial.

Genetic Algorithms Learning Machines for knowledge discovery.

1 MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING By Kaan Tariman M.S. in Computer Science CSCI 8810 Course Project.

Classification of Music According to Genres Using Neural Networks, Genetic Algorithms and Fuzzy Systems.

COMP305. Part I. Artificial neural networks.. Topic 3. Learning Rules of the Artificial Neural Networks.

Genetic Algorithm What is a genetic algorithm? “Genetic Algorithms are defined as global optimization procedures that use an analogy of genetic evolution.

The Performance of Evolutionary Artificial Neural Networks in Ambiguous and Unambiguous Learning Situations Melissa K. Carroll October, 2004.

Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.

Khaled Rasheed Computer Science Dept. University of Georgia

Traffic Sign Recognition Using Artificial Neural Network Radi Bekker

Evolving a Sigma-Pi Network as a Network Simulator by Justin Basilico.

Slides are based on Negnevitsky, Pearson Education, Lecture 12 Hybrid intelligent systems: Evolutionary neural networks and fuzzy evolutionary systems.

Cristian Urs and Ben Riveira. Introduction The article we chose focuses on improving the performance of Genetic Algorithms by: Use of predictive models.

Soft Computing Lecture 18 Foundations of genetic algorithms (GA). Using of GA.

Neural NetworksNN 11 Neural netwoks thanks to: Basics of neural network theory and practice for supervised and unsupervised.

Explorations in Neural Networks Tianhui Cai Period 3.

Appendix B: An Example of Back-propagation algorithm

Lecture 8: 24/5/1435 Genetic Algorithms Lecturer/ Kawther Abas 363CS – Artificial Intelligence.

Conceptual Foundations © 2008 Pearson Education Australia Lecture slides for this course are based on teaching materials provided/referred by: (1) Statistics.

LOGO Ensemble Learning Lecturer: Dr. Bo Yuan

1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.

The Perceptron. Perceptron Pattern Classification One of the purposes that neural networks are used for is pattern classification. Once the neural network.

Neural and Evolutionary Computing - Lecture 9 1 Evolutionary Neural Networks Design  Motivation  Evolutionary training  Evolutionary design of the architecture.

Overview of Machine Learning RPI Robotics Lab Spring 2011 Kane Hadley.

1 Genetic Algorithms and Ant Colony Optimisation.

Genetic Algorithms Przemyslaw Pawluk CSE 6111 Advanced Algorithm Design and Analysis

 Based on observed functioning of human brain.  (Artificial Neural Networks (ANN)  Our view of neural networks is very simplistic.  We view a neural.

Genetic Algorithms Genetic algorithms provide an approach to learning that is based loosely on simulated evolution. Hypotheses are often described by bit.

CS 478 – Tools for Machine Learning and Data Mining Perceptron.

CITS7212: Computational Intelligence An Overview of Core CI Technologies Lyndon While.

Perceptrons Michael J. Watts

Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer.

Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.

“Principles of Soft Computing, 2 nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved. CHAPTER 2 ARTIFICIAL.

An Evolutionary Algorithm for Neural Network Learning using Direct Encoding Paul Batchis Department of Computer Science Rutgers University.

Evolving robot brains using vision Lisa Meeden Computer Science Department Swarthmore College.

A Presentation on Adaptive Neuro-Fuzzy Inference System using Particle Swarm Optimization and it’s Application By Sumanta Kundu (En.R.No.

Evolutionary Computation Evolving Neural Network Topologies.

Neural Networks: An Introduction and Overview

Neural Networks.

Fall 2004 Perceptron CS478 - Machine Learning.

ANN-based program for Tablet PC character recognition

Example, BP learning function XOR

Ranga Rodrigo February 8, 2014

EA, neural networks & fuzzy systems

Using Recurrent Neural Networks and Hebbian Learning

Hebb and Perceptron.

Chapter 3. Artificial Neural Networks - Introduction -

Artificial Intelligence Methods

Artificial Neural Networks

Lecture Notes for Chapter 4 Artificial Neural Networks

M. Gams Jozef Stefan Institute

Lecture 4. Niching and Speciation (1)

Training Feedforward Neural Networks Using Genetic Algorithms

Neural Networks: An Introduction and Overview

Learning Combinational Logic

Presentation transcript:

The Evolution of Learning Algorithms for Artificial Neural Networks Published 1992 in Complex Systems by Jonathan Baxter Michael Tauraso

Genetic Algorithm on NNs Start with a population of neural networks. Find the fitness of each for a particular task Weed out the low-fitness ones Breed the high-fitness ones to make a new population. Repeat.

Local Binary Neural Networks (LBNNs) All weights, inputs, and outputs are binary. Learning rule is a localized boolean function of two variables. This vastly simplifies everything. LBNNs are easy to encode into binary strings. LBNNs are easy to write into genetic algorithms

An LBNN

Rules for LBNNs Weights are +1, -1, or 0 Nodes: ai(t+1) =sign( ∑ aj(t)wji(t) ) Weights: wij(t+1) = f(ai(t), aj(t)) Weights are classified as fixed or learnable. 0 weights are fixed.

Training Rules Boolean functions of two variables 16 possible varieties Analog of Hebb’s rule given by: f(ai(t),aj(t)) = ai(t) aj(t)

Training Goal Learn the 4 boolean functions of one variable Identity, Inverse, Always 1, Always 0 Who wants to learn the boolean functions of one variable anyway?

Fitness Determination Start with an LBNN from the sample population Clamp the output node to train for a particular boolean function. Fitness is how well the network performs at calculating that boolean function after training.

A Successful LBNN

Findings Hebb’s rule is the most efficient learning rule. LBNNs can be thought of as state machines

LBNNs as State Machines Boolean functions are encoded as transitions between fixed points in the NN Other transitions seek to push the network toward the appropriate fixed point.

State Machine for an LBNN

Questions ?