Artificial Neural Networks (ANNs) Week 5 Artificial Neural Networks (ANNs)

Case Scenario ABC Enterprise, as a multinational company has invested in many country sectors for revenues generation. One of the potential revenue for the company comes from the investment in the BSKL shares. The CEO of the company, Mr. Ridzuan intends to invest in selected potential shares available in the current market. He is aware that such investment is a risky business. Unfortunately, he does not have experienced staff to advice on the potential shares investment. Thus, Mr. Ridzuan is thinking whether there is any system that can predict short and long term future of current shares trading in the market with accurately. The system must be able to analyze history of the current shares trading and predicts when the best time to buy the shares is.

Human Brain 50 to 150 billion neurons in brain (estimation) Neurons grouped into networks Axons send outputs to cells Received by dendrites, across synapses

Artificial Neural Networks (ANNs) A model that emulates a biological neural network. Software simulations of the massively parallel processes that involve processing elements interconnected in a network architecture. Originally proposed as a model of the human brain’s activities. The human brain is much more complex.

Processing Information in an Artificial Neuron Inputs Weights x1 w1j Output Yj Neuron j Σ wij xi x2 w2j ƒ Summations Transfer function xi wij

Processing Processing elements are neurons Allows for parallel processing Each input is single attribute Connection weight Adjustable mathematical value of input Summation function Weighted sum of input elements Internal stimulation Transfer function Relation between internal activation and output Sigmoid/transfer function Threshold value Outputs are problem solution

Learning: Three Tasks The neurons are connected by links, and each link has a numerical weight associated with it. Weights are the basic means of long term memory in ANN. They express the strength or importance of each neuron input. ANN learns through repeated adjustments of these weights. In summary, learning in ANN involves three tasks: 1. Compute Outputs 2. Compare Outputs with Desired Targets 3. Adjust Weights and Repeat the Process

Learning Algorithms Learning is a fundamental characteristic of ANNs. Two Basic Learning Categories Supervised Learning Unsupervised Learning

Supervised Learning For a set of inputs with known (desired) outputs. Connection weights derived from known cases. Useful in pattern recognition (character, voice, object etc.) Examples Backpropagation network Hopfield network

Supervised Learning : Character Recognition Demonstration of a neural network used within an optical character recognition (OCR) application. The original document is scanned into the computer and saved as an image. The OCR software breaks the image into sub-images, each containing a single character. The sub-images are then translated from an image format into a binary format, where each 0 and 1 represents an individual pixel of the sub-image. The binary data is then fed into a neural network that has been trained to make the association between the character image data and a numeric value that corresponds to the character. The output from the neural network is then translated into ASCII text and saved as a file.

Unsupervised Learning Only input stimuli shown to the network. Humans assign meanings and determine usefulness. Useful in clustering (objects) and knowledge discovery. Examples Adaptive Resonance Theory (ART) Kohonen Self-organizing Feature Maps

Unsupervised Learning : Data Classification Classification of raw data into two subgroups.

Development of Systems Collect data The more, the better Separate data into training set to adjust weights Divide into test sets for network validation Select network topology Determine input, output, and hidden nodes, and hidden layers Select learning algorithm and connection weights Iterative training until network achieves preset error level Black box testing to verify inputs produce appropriate outputs Contains routine and problematic cases

Neural Network Software Program in: Programming language (C++, Java, VB) Neural network package or NN programming tool Tools (shells) incorporate: (MATLAB) Training algorithms Transfer and summation functions May still need to: Program the layout of the database Partition the data (test data, training data) Transfer the data to files suitable for input to an ANN tool

Advantages of ANNs Pattern recognition, classification, generalization, interpretation of incomplete and noisy inputs. Character, speech and visual recognition. Can tackle new kinds of problems. Robust, flexible and easy to maintain. Powerful hybrid systems.

Limitations of ANNs Do not do well at tasks that are not done well by people Lack explanation capabilities Limitations and expense of hardware technology restrict most applications to software simulations Training time can be excessive and tedious Usually requires large amounts of training and test data

ANN Examples NeuroXL Classifier (add-in for Ms Excell) http://www.neuroxl.com/index.htm N-OCR (character recognition) http://www10.brinkster.com/geniusportal/neural/nocr.html Neural Network Toolbox (MathWorks) http://www.mathworks.com/products/neuralnet/

Intelligence Density Dimension Accuracy Flexibility Embeddedability Independence from experts