Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and evolving N.N Basic Architectures and algorithms Applications Drawbacks and limitations

What are Neural Networks? information processing paradigm inspired by biological nervous systems, such as our brain Structure: large number of highly interconnected processing elements (neurons) working together Like people, they learn from experience (by example)

Historical background Neural network simulations appear to be a recent development. However, this field was established before the advent of computers, and has survived at least one major setback and several eras. These pioneers were able to develop convincing technology which surpassed the limitations identified by Minsky and Papert. Minsky and Papert, published a book (in 1969)

Human Brain Function

Neural Network Function

Terminology Input: Explanatory variables also referred to as “predictors”. Neuron: Individual units in the hidden layer(s) of a neural network. Output: Response variables also called “predictions”. Hidden Layers: Layers between input and output that an apply activation function.

Some Similarities

Why Neural Networks are desirable Human brain can generalize from abstract Recognize patterns in the presence of noise Recall memories Make decisions for current problems based on prior experience

When to use neural networks Use for huge data sets (i.e. 50 predictors and 15,000 observations) with unknown distributions Smaller data sets with outliers as neural networks are very resistant to outliers

Why Neural Networks in Statistics? The methodology is seen as a new paradigm for data analysis where models are not explicitly stated but rather implicitly defined by the network. Allows for analysis where traditional methods might be extremely tedious or nearly impossible to interpret.

Difference in Neural Networks The difference in the two approaches is that multiple linear regression has a closed form solution for the coefficients, while neural networks use an iterative process.

Applications Data Conceptualization –infer grouping relationships e.g. extract from a database the names of those most likely to buy a particular product. Data Filtering –e.g. take the noise out of a telephone signal, signal smoothing Planning –Unknown environments –Sensor data is noisy –Fairly new approach to planning

Applications Prediction: learning from past experience –pick the best stocks in the market –predict weather –identify people with cancer risk Classification –Image processing –Predict bankruptcy for credit card companies –Risk assessment

Strengths of a Neural Network Power: Model complex functions, nonlinearity built into the network Ease of use: –Learn by example –Very little user domain-specific expertise needed Intuitively appealing: based on model of biology, will it lead to genuinely intelligent computers/robots?

General Advantages Advantages –Adapt to unknown situations –Robustness: fault tolerance due to network redundancy –Autonomous learning and generalization Disadvantages –Not exact –Large complexity of the network structure For motion planning?

Status of Neural Networks Most of the reported applications are still in research stage No formal proofs, but they seem to have useful applications that work

Evolving networks Continuous process of: –Evaluate output –Adapt weights –Take new inputs ANN evolving causes stable state of the weights, but neurons continue working: network has ‘learned’ dealing with the problem

Where are NN used? Recognizing and matching complicated, vague, or incomplete patterns Data is unreliable Problems with noisy data –Prediction –Classification –Data association –Data conceptualization –Planning

Back propagation Desired output of the training examples Error = difference between actual & desired output Change weight relative to error size Calculate output layer error, then propagate back to previous layer Improved performance, very common!

Activation Function The only practical requirement for an activation function is that it be differentiable Sigmoid function is commonly used g(netinput) = 1/(1+ exp -(netinput) ) Or a simple binary threshold unit Ө(netinput) = {1,if netinput ≥ 0 ; 0, otherwise}

Training the Network Neural Networks must be first trained before being used to analyze new data Process entails running patterns through the network until the network has “learned” the model to apply to future data Can take a long time for noisy data

New Data Once the network is trained new data can be run through it The network will classify new data based on the previous data it trained with If an exact match can not be found it will match with the closest found in memory

Regression and Neural Networks Objective of regression problem is to find coefficients that minimize sum of errors To find coefficients we must have a dataset that includes the independent variable and associated values of the dependent variable. (very similar to training the network) Equivalent to a single layer feed forward network

Drawbacks and Limitations Neural Networks can be extremely hard to use The programs are filled with settings you must input and a small error will cause your predictions to have error also The results can be very hard to interpret as well