Back-propagation network (BPN) Student : Dah-Sheng Lee Professor: Hahn-Ming Lee Date:20 September 2003

Outline What is a Neural Network ? Artificial Neural Network (ANN) property Back-propagation network (BPN) Reference

What is a Neural Network ? Neural Networks are a different paradigm for computing: von Neumann machines are based on the processing/memory abstraction of human information processing. neural networks are based on the parallel architecture of animal brains. Neural networks are a form of multiprocessor computer system, with simple processing elements a high degree of interconnection simple scalar messages adaptive interaction between elements

What is a Neural Network ? (cont…) A biological neuron may have as many as 10,000 different inputs, and may send its output (the presence or absence of a short-duration spike) to many other neurons. Neurons are wired up in a 3-dimensional pattern. Real brains, however, are orders of magnitude more complex than any artificial neural network so far considered.

神經核（ soma ） 神經軸突（ axon ） 神經突觸（ synapses ） （ synapses ） 神經樹突（ dendrites ） (dendrites ）

What is a Neural Network ? (cont…) The dendrites are extensions of a neuron which connect to other neurons to form a neural network, while synapses are a gateway which connects to dendrites that come from other neurons. A biological neuron may thus be connected to other neurons as well as accepting connections from other neurons, and so we have the basis of a network. Through those connections, electrical pulses are transmitted, and information is carried in the timing and the frequency with which these pulses are emitted. So, our neuron receives information from other neurons, processes it and then relays this information to other neurons.

Artificial Neural Network (ANN) property ANN 運作所需的範例資料有 Training example Testing example 待推案例 ANN characteristics Input : training set,testing set output Processing Element(PE)

Artificial Neural Network (ANN) property (cont…) ANN Type Supervised learning :Perceptron, BPN, PNN,LVQ,CPN Unsupervised learning :SOM, ART Associate learning :Hopfield, Bidirectional Associative Memory(BAM), Hopfield-Tank Optimization application :HTN, ANN(Annealed Neural Network)

Artificial Neural Network (ANN) property (cont…) ANN structure One way feedforward Two way feedforward Feedback Y X1X1 X2X2 XnXn ‧‧‧‧‧‧ Y X1X1 X2X2 XnXn ‧‧‧‧‧‧ Y X1X1 X2X2 XnXn ‧‧‧‧‧‧

Artificial Neural Network (ANN) property (cont…) ANN Basic Model Processing Element(PE) : summation fc.,Activity fc.,transfer fc. Input layer : [x1.....xi](training set,testing set) Hidden layer : present PE's internal relationship Output layer : normalize output,competitive output,competitive learning Network : Learning,Recalling Weights : connecting between layers

Artificial Neural Network (ANN) property (cont…) Transfer Function Type Discrete Type Perceptron/step fc. Signum fc. Signum 0 fc. Hopfield-Tank fc. BAM fc. Linear Type threshold line fc. straight linear fc. Nonlinear Type sigmoid fc. Hyperbolic tangent fc.

Artificial Neural Network (ANN) property (cont…) (Single-layer Perceptrons)

Back-propagation network (BPN) The network model “BPN” is Supervised learning Feedforward Multilayer Perceptrons (Special case: no hidden layer)

(Multilayer Perceprons)

Back-propagation network (BPN) Training algorithm Step 1: Initialize the network synaptic weights to small random value. Step 2:Form the set of training input/output pairs, present an input pattern and calculate the network response. Step 3: The desire network response is compared with the actual output of the network, and by using 1* and 2* all the local errors can be computed Step 4:The weight of the network are update according to 3* Step 5:Until the network reaches a predetermined level of accuracy in producing the adequate response for all the training pattern, continue step 2 through 4

Back-propagation network (BPN) advantage algorithm Backpropagation Learning Algorithm with Momentum Updating Batch Updating Search-Then-Converge Method Batch Updating with Variable Learning Rate etc …

Reference “Principles of Neuroncomputing for Science and Engineer” Fredric M. Ham; Ivica Kostanic; McGRAW-HILL INTERNATIONAL EDITION, 2001 “ 類神經網路模式應用與實作 ” 8th 葉怡成 儒林出版社, 2003