1. Introduction to Neural Networks Simon Durrant Quantitative Methods December 15th

2. A Typical Artificial Neural Network A neural network consists of neurons connected by weights. Inputs get multiplied by the weights and summed before entering neurons. Neurons have transfer functions which change the signals and then give outputs. Neurons may also have a threshold or bias input (not shown). But what are they for?

3. The Family of Neural Networks Many types of network; some common subdivisions. Supervised learning: – we have a set of exemplars for which we have known target outputs. – The network learns by adjusting weights to better achieve the target outputs. Unsupervised learning: – We aim to find groups and subdivisions within our data – Weights are adjusted such that neurons with similar weight patterns are made even more similar, while others are made more distinct. Each set of similar neurons comes to represent a particular subgroup in the data, and responds most strongly to inputs from the subgroup.

4. Classification with ANNs Two classes (red and blue), two-dimensional data (i.e. each data point is defined by two values, such as length and width). We want a model that can separate the two classes, and will be able to tell us which class a new data point belongs to. http://lcn.epfl.ch/tutorial/english/mlp/html/index.html

5. Classification with ANNs Two classes (red and blue), two-dimensional data (i.e. each data point is defined by two values, such as length and width). We want a model that can separate the two classes, and will be able to tell us which class a new data point belongs to. http://lcn.epfl.ch/tutorial/english/mlp/html/index.html

6. Regression with ANNs We want to predict Boston house prices. We have measured 13 different variables associated with 500+ houses in Boston for which we know the price. We want a model that will use the relevant information from our inputs in whatever complex combination gives the best outcome. We will use the Matlab Neural Network Toolbox for this demo. Our chosen network is a multi-layer perceptron.

7. Regression with ANNs Our network has learned to predict the correct price for houses that it was not trained on (the test set) – it has generalised. The strong performance (r ranges between 0 and 1, where 1 is a perfect score; we have r=0.948 for the unseen test data) is greater than the maximum that can be achieved with multiple linear regression.

8. Cluster Visualisation with ANNs We have taken four different measurements from different types of iris flowers: sepal length, sepal width, petal length, petal width. We want to know if there are subgroups of irises. A Self-Organising Map is the type of neural network we use here. It adjusts weights to group similar items using a Mexican hat. We will use the Matlab Neural Network Toolbox for this demo.

9. Cluster Visualisation with ANNs Weights have evolved to cover the input space. Looking at weight distances in the grid, we see clear subdivisions within the data. This is reflected in the number of hits for neighborouging neurons.

10. Cluster Visualisation with ANNs Another demo (from http://www.ai-junkie.com/ann/som/som5.html): self- organisation of small coloured blocks on the basis of their RGB colour values. It can be used for practical purposes in mapping world poverty, for example, when measured by a complex series of variables (e.g. health, nutrition, education, water supply etc.) All of these are forms of dimensionality reduction – take complex multivariate data and reduce it to two (or N) dimensions.

11. Advantages of Neural Networks Can handle many different statistical requirements (regression, classification, clustering, time series analysis, pattern analysis etc.). Can handle nonlinear data without any special measures. Are somewhat model-free, i.e. you do not need to know in advance whether to use a linear model, polynomial model etc.. Seamlessly provide generalisation, i.e. can be applied to novel inputs and give a useful and meaningful output. Provide graceful degradation; if you break part of the model, it does not fall apart entirely.

12. …and Disadvantages Can be something of a black box. Requires selection of particularly type of network. Requires choice of network architectural features (such as the number of neurons within a layer). Setting free parameter values in order to achieve good performance can sometimes be difficult. But if treated with care, artificial neural networks can offer a set of very powerful statistical techniques without requiring a large knowledge of statistics.

13. Applications of Neural Networks Sales forecasting. Industrial control systems. Robot navigation. Stock price prediction. Medical image analysis. Musical instrument classification. Modelling human cognition. Consumer behaviour data mining. …and 100s more.

14. Thanks for Listening! Any Questions?