Neural Network Tool Box Khaled A. Al-Utaibi

Outlines  Neuron Model  Transfer Functions  Network Architecture  Neural Network Models  Feed-forward Network  Training & Simulation  Example 1: Majority Function  Example 2: Handwritten Digits Recognition ICS583: Pattern Recoginition

Neuron Model ICS583: Pattern Recoginition

Neuron Model ICS583: Pattern Recoginition InputsWeights Bias Weighted Sum Transfer Function Output

Transfer Functions  Many transfer functions are included in the Neural Network Toolbox  Three of the most commonly used functions are Hard-Limit Transfer Function Linear Transfer Function Log-Sigmoid Transfer Function ICS583: Pattern Recoginition

Transfer Functions  Hard-Limit Transfer Function ICS583: Pattern Recoginition

Transfer Functions  Linear Transfer Function ICS583: Pattern Recoginition

Transfer Functions  Log-sigmoid Transfer Function ICS583: Pattern Recoginition

Network Architecture  Single Layer of Neurons ICS583: Pattern Recoginition

Network Architecture  Multiple Layers of Neurons ICS583: Pattern Recoginition

Neural Network Models  MATLAB contains several models of neural networks (general / special purpose): Feed-forward back-propagation network Elman back-propagation network Cascade-forward back-propagation network Pattern recognition network Fitting network SOM network (Self-Organizing Map) ICS583: Pattern Recoginition

Feed-Forward Network  Create feed-forward back-propagation network  Many neural network models in MATLAB are special cases of this model (e.g. pattern recognition, fitting, and SOM)  Syntax ICS583: Pattern Recoginition network_name = newff(arguments)

Feed-Forward Network  Arguments ICS583: Pattern Recoginition Argument(s)Description Pinput vectors TTarget vectors [S 1 S 2 … S N-1 ]Size of ith layer {TF 1, TF 2, …, TF N }Transfer function of ith layer BTFBp network training function BLFBp weight/bias learning function IPFInput processing functions. OPFOutput processing functions DDFData division function

Feed-Forward Network  Output layer size is determined from T  Input and output processing functions transform the inputs and outputs into a better form for network use: Re-encode unknown input/output values into numerical values Remove redundant inputs and outputs vectors. Normalizes input/output values. ICS583: Pattern Recoginition

Feed-Forward Network  MATLAB provides several data division functions that divide the input data into three sets (using different strategies and different percentage for each set: Training Set Validation Set Testing Set ICS583: Pattern Recoginition

Training the Network  Syntax  Arguments ICS583: Pattern Recoginition [ret_vals] = train(arguments) Argument(s)Description netNeural network to be trained PNetwork inputs TNetwork targets PiInitial input delay conditions AiInitial layer delay conditions

Training the Network  Returned Values ICS583: Pattern Recoginition Argument(s)Description netTrained neural network PTraining record (iter & performance) YNetwork outputs ENetwork errors PfFinal input delay conditions AfFinal layer delay conditions

Simulating the Network  Syntax  Arguments ICS583: Pattern Recoginition [ret_vals] = sim(arguments) Argument(s)Description netNeural network to be simulated PNetwork inputs PiInitial input delay conditions AiInitial layer delay conditions TNetwork targets

Simulating the Network  Returned Values ICS583: Pattern Recoginition Argument(s)Description YNetwork outputs PfFinal input delay conditions AfFinal layer delay conditions ENetwork errors PerfNetwork performance

Example 1: Majority Function P1P2P3T ICS583: Pattern Recoginition % initialize network inputs inputs = [ ; ; ]; % initialize network targets targets = [ ];

Example 1: Majority Function ICS583: Pattern Recoginition % initialize network inputs inputs = [ ; ; ]; % initialize network targets targets = [ ]; % create a feed-froward network with a hidden % layer of 3 neurons net = newff(inputs, targets, 3,... {'logsig', 'purelin'}); % train the network net = train(net,inputs,targets); % simulate the network outputs = sim(net,inputs);

Example 2: Handwritten Digits Recognition ICS583: Pattern Recoginition  Given a set of 1000 samples of different handwritten digits (0,1, …, 9),  Each digit is represented as a binary image of size 28x28 pixels

Example 2: Handwritten Digits Recognition ICS583: Pattern Recoginition  We would like to use MATLAB Neural Network Toolbox to design a neural network to recognize handwritten digits  Pattern recognition network (newpr) is suitable for this purpose

Example 2: Handwritten Digits Recognition ICS583: Pattern Recoginition % initialize network inputs inputs = [ ; ];

Example 2: Handwritten Digits Recognition ICS583: Pattern Recoginition % initialize network targets inputs = [ 0 1; ; ; ];

Questions ICS583: Pattern Recoginition