Neural Networks II By Jinhwa Kim

2 Neural Computing is a problem solving methodology that attempts to mimic how human brain function Artificial Neural Networks (ANN) Machine Learning Neural Computing: The Basics

3 Computing technology that mimic certain processing capabilities of the human brain Knowledge representations based on Massive parallel processing Fast retrieval of large amounts of information The ability to recognize patterns based on historical cases Neural Computing = Artificial Neural Networks (ANNs) Purpose of ANN is to simulate the thought process of human brain Inspired by the studies of human brain and the nervous system Neural Computing

4 The Biology Analogy Neurons: brain cells Nucleus (at the center) Dendrites provide inputs Axons send outputs Synapses increase or decrease connection strength and cause excitation or inhibition of subsequent neurons

5 A model that emulates a biological neural network Software simulations of the massively parallel processes that involve processing elements interconnected in a network architecture Originally proposed as a model of the human brain’s activities The human brain is much more complex Artificial Neural Networks (ANN)

6 Biological Artificial Soma Node Dendrites Input Axon Output Synapse Weight Slow speed Fast speed Many neurons Few neurons (Billions) (Dozens) Artificial Neural Networks (ANN) Three Interconnected Artificial Neurons

7 Components and Structure “A network is composed of a number of processing elements organized in different ways to form the network structure” Processing Elements (PEs) – Neurons Network Collection of neurons (PEs) grouped in layers Structure of the Network Topologies / architectures – different ways to interconnect PEs ANN Fundamentals Figure 15.3

8 ANN Fundamentals

9 Processing Information by the Network Inputs Outputs Weights Summation Function Figure 15.5 ANN Fundamentals

10 Transformation (Transfer) Function Computes the activation level of the neuron Based on this, the neuron may or may not produce an output Most common: Sigmoid (logical activation) function ANN Fundamentals

11 Learning in ANN 1. Compute outputs 2. Compare outputs with desired targets 3. Adjust the weights and repeat the process

12 Collect data and separate it into Training set (50%), Testing set (50%) Training set (60%), Testing set (40%) Training set (70%), Testing set (30%) Training set (80%), Testing set (20%) Training set (90%), Testing set (10%) Make sure that all three sets represent the population: true random sampling Use training and cross validation cases to adjust the weights Use test cases to validate the trained network Data Collection and Preparations

13 Neural Network Architecture There are several ANN architectures :feedforward, recurrent, Hopfield et al.

14 Neural Network Architecture Feed forward Neural Network Multi Layer Perceptron, - Two, Three, sometimes Four or Five Layers

15 Step function evaluates the summation of input values Calculating outputs Measure the error (delta) between outputs and desired values Update weights, reinforcing correct results At any step in the process for a neuron, j, we get Delta(Error) = Z j - Y j where Z and Y are the desired and actual outputs, respectively How a Network Learns

16 Continue Backpropagation Backpropagation (back-error propagation) Most widely used learning Relatively easy to implement Requires training data for conditioning the network before using it for processing other data Network includes one or more hidden layers Network is considered a feedforward approach

17 1. Initialize the weights 2. Read the input vector 3. Generate the output 4. Compute the error Error = Output – Desired output 5. Change the weights Drawbacks: A large network can take a very long time to train May not converge Backpropagation

18 Test the network after training Examine network performance: measure the network’s classification ability Black box testing Do the inputs produce the appropriate outputs? Testing

19 ANN Development Tools E-Miner Clementine NeuroSolutions Statistica Neural Network Toolkit Braincel (Excel Add-in) NeuralWorks Brainmaker PathFinder Trajan Neural Network Simulator NeuroShell Easy SPSS Neural Connector NeuroWare

20 Benefits of ANN Diverse Applications: Pattern recognition, learning, classification, generalization and abstraction, and interpretation of incomplete and noisy inputs Character, speech and visual recognition Advantages: Can provide some human problem-solving characteristics Can tackle new unseen kinds of problems Robust Fast decision making

21 Limitations of ANN Lack explanation capabilities Limitations and expense of hardware technology restrict most applications to software simulations Training time can be excessive and tedious Black box that is hardly understood by human

22 Business ANN Applications Accounting Identify tax fraud Enhance auditing by finding irregularities Finance Signatures and bank note verifications Foreign exchange rate forecasting Bankruptcy prediction Customer credit scoring Credit card approval and fraud detection* Stock and commodity selection and trading Forecasting economic turning points Pricing initial public offerings* Loan approvals …

23 Business ANN Applications Human Resources Predicting employees’ performance and behavior Determining personnel resource requirements Management Corporate merger prediction Country risk rating Marketing Consumer spending pattern classification Sales forecasts Targeted marketing, … Operations Vehicle routing Production/job scheduling, …