Introduction to Neural Networks Neural Network Architecture and Learning

Neural Networks Notations

Lecture Overview The Neuron Model Neural Networks Architectures Learning and Learning Rules in Neural Networks.

Introduction Artificial neuron models and neural network architectures are generally quite simple. Why are artificial neural networks so powerful in solving computationally complex tasks, such as image/speech understanding, optimisation, and process control? This is because their ability to learn and to generalise.

Neuron Model

Activation Function

Neural Networks Architectures

Neural Networks Architecture

How to Design a Neural Network for a Given Task A neural network model consists of : (1)a neuron model (2)interconnection architecture (3)A learning (optimisation) algorithm. The design process includes: (1)Choosing a neuron model (2)Selecting a neural network architecture (3)Collecting training data and developing a learning algorithm (4)Training the network and testing its performance The neural network performance is evaluated by: –Convergence/stability –Accuracy –Storage capacity –Generalisation.

Learning in Neural Networks Learning is a process by which the free parameters (including hyper-parameters controlling the network structure) of a neural network are adapted through a process of stimulation by the environment where the network is embedded. The type of learning is determined by the manner in which the parameter changes take place.

Learning Tasks and Performance Measures

Hebbian Learning Rule-The oldest rule

Learning in Neural Networks

Hebbian Learning Rule

Competitive Learning Rule

Error Correction Learning Rule

Performance Improvement through Error Correction Learning

Error Gradient Descent Learning Rule