Presentation is loading. Please wait.

Presentation is loading. Please wait.

December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 1 Example I: Predicting the Weather Since the.

Published byAnne Curtis Modified over 9 years ago

Similar presentations

Presentation on theme: "December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 1 Example I: Predicting the Weather Since the."— Presentation transcript:

1 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 1 Example I: Predicting the Weather Since the input does not only include our station, but also the eight surrounding ones, the input layer of the network looks like this: … our station … north … … northwest The network has 207 input neurons, which accept 207-component input vectors.

2 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 2 Example I: Predicting the Weather What should the output patterns look like? We want the network to produce a set of indicators that we can interpret as a prediction of the weather in 24 hours from now. In analogy to the weather forecast on the evening news, we decide to demand the following four indicators: a temperature prediction a temperature prediction a prediction of the chance of precipitation occurring a prediction of the chance of precipitation occurring an indication of the expected cloud cover an indication of the expected cloud cover a storm indicator (extreme conditions warning) a storm indicator (extreme conditions warning)

3 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 3 Example I: Predicting the Weather Each of these four indicators can be represented by one scaled output value: temperature (-10… 40 degrees Celsius) temperature (-10… 40 degrees Celsius) chance of precipitation (0%… 100%) chance of precipitation (0%… 100%) cloud cover (0… 9) cloud cover (0… 9) storm warning: two possibilities: storm warning: two possibilities: –0: no storm warning; 1: storm warning –probability of serious storm (0%… 100%) Of course, the actual network outputs range from 0 to 1, and after their computation, if necessary, they are scaled to match the ranges specified above.

4 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 4 Example I: Predicting the Weather We decide (or experimentally determine) to use a hidden layer with 42 sigmoidal neurons. In summary, our network has 207 input neurons 207 input neurons 42 hidden neurons 42 hidden neurons 4 output neurons 4 output neurons Because of the small output vectors, 42 hidden units may suffice for this application. Usually, more hidden units that input units are used; but as you know, no formula exists to compute the appropriate number of hidden neurons.

5 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 5 Example I: Predicting the Weather The next thing we need to do is collecting the training exemplars. First we have to specify what our network is supposed to do: In production mode, the network is fed with the current weather conditions, and its output will be interpreted as the weather forecast for tomorrow. Therefore, in training mode, we have to present the network with exemplars that associate known past weather conditions at a time t with the conditions at t – 24 hrs. So we have to collect a set of historical exemplars with known correct output for every input.

6 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 6 Example I: Predicting the Weather Obviously, if such data is unavailable, we have to start collecting them. The amount of data that we need depends on the amount of changes in weather at our location. For example, in Honolulu, Hawaii, we only need data for a few months, because there is little variation in the weather. In Boston, however, we would need to cover at least one year because of dramatic changes in weather across seasons. As we know, some winters in Boston are much harder than others, so it might be a good idea to collect data for several years.

7 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 7 Example I: Predicting the Weather And how about the granularity of our exemplar data, i.e., the frequency of measurement? Using one sample per day would be a natural choice, but it would neglect rapid changes in weather. If we use hourly instantaneous samples, however, we increase the likelihood of conflicts. Therefore, we decide to do the following: We will collect input data every hour, but the corresponding output pattern will be the average of the instantaneous patterns over a 12-hour period. This way we reduce the possibility of errors while increasing the amount of training data.

8 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 8 Example I: Predicting the Weather Now we have to train our network. If we use samples in one-hour intervals for one year, we have 8,760 exemplars (as a rule of thumb, though, we should ideally have about 5 to 10 times as many exemplars as there are weights in the network). Since with a large number of samples the hold-one-out training method is very time consuming, we decide to use partial-set training instead. The best way to do this would be to acquire a test set (control set), that is, another set of input-output pairs measured on random days and at random times. After training the network with the 8,760 exemplars, we could then use the test set to evaluate the performance of our network.

9 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 9 Example I: Predicting the Weather Neural network troubleshooting: Plot the global error as a function of the training epoch. The error should decrease after every epoch. If it oscillates, do the following tests.Plot the global error as a function of the training epoch. The error should decrease after every epoch. If it oscillates, do the following tests. Try reducing the size of the training set. If then the network converges, a conflict may exist in the exemplars.Try reducing the size of the training set. If then the network converges, a conflict may exist in the exemplars. If the network still does not converge, continue pruning the training set until it does converge. Then add exemplars back gradually, thereby detecting the ones that cause conflicts.If the network still does not converge, continue pruning the training set until it does converge. Then add exemplars back gradually, thereby detecting the ones that cause conflicts.

10 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 10 Example I: Predicting the Weather If this still does not work, look for saturated neurons (extreme weights) in the hidden layer. If you find those, add more hidden-layer neurons, possibly an extra 20%.If this still does not work, look for saturated neurons (extreme weights) in the hidden layer. If you find those, add more hidden-layer neurons, possibly an extra 20%. If there are no saturated units and the problems still exist, try lowering the learning parameter  and training longer.If there are no saturated units and the problems still exist, try lowering the learning parameter  and training longer. If the network converges but does not accurately learn the desired function, evaluate the coverage of the training set.If the network converges but does not accurately learn the desired function, evaluate the coverage of the training set. If the coverage is adequate and the network still does not learn the function precisely, you could refine the pattern representation. For example, you could include a season indicator to the input, helping the network to discriminate between similar inputs that produce very different outputs.If the coverage is adequate and the network still does not learn the function precisely, you could refine the pattern representation. For example, you could include a season indicator to the input, helping the network to discriminate between similar inputs that produce very different outputs. Then you can start predicting the weather!

11 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 11 A Note on Setting Desired Outputs Due to the sigmoid output function, the net input to the output units would have to be -  or  to generate outputs 0 or 1, respectively. Because of the shallow slope of the sigmoid function at extreme net inputs, even approaching these values would be very slow. To avoid this problem, it is advisable to use desired outputs  and (1 -  ) instead of 0 and 1, respectively. Typical values for  range between 0.01 and 0.1. For example, for  = 0.1, a desired output vector could look like this: y = (0.1, 0.1, 0.9, 0.1)

12 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 12 Example II: Face Recognition Now let us assume that we want to build a network for a computer vision application. More specifically, our network is supposed to recognize faces and face poses. This is an example that has actually been implemented. All information, program code, data, etc, can be found at: http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/faces.html

13 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 13 Example II: Face Recognition The goal is to classify camera images of faces of various people in various poses. Images of 20 different people were collected, with up to 32 images per person. The following variables were introduced: expression (happy, sad, angry, neutral) expression (happy, sad, angry, neutral) direction of looking (left, right, straight ahead, up) direction of looking (left, right, straight ahead, up) sunglasses (yes or no) sunglasses (yes or no) In total, 624 grayscale images were collected, each with a resolution of 30 by 32 pixels and intensity values between 0 and 255.

14 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 14 Example II: Face Recognition The network presented here only has the task of determining the face pose (left, right, up, straight) shown in an input image. It uses 960 input units (one for each pixel in the image), 960 input units (one for each pixel in the image), 3 hidden units 3 hidden units 4 output neurons (one for each pose) 4 output neurons (one for each pose) Each output unit receives an additional input, which is always 1. By varying the weight for this input, the backpropagation algorithm can adjust an offset for the net input signal.

15 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 15 Example II: Face Recognition The following diagram visualizes all network weights after 1 epoch and after 100 epochs. Their values are indicated by brightness (ranging from black = -1 to white = 1). Each 30 by 32 matrix represents the weights of one of the three hidden-layer units. Each row of four squares represents the weights of one output neuron (three weights for the signals from the hidden units, and one for the constant signal 1). After training, the network is able to classify 90% of new (non-trained) face images correctly.

16 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 16 Example II: Face Recognition

17 December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 17 Example III: Character Recognition http://sund.de/netze/applets/BPN/bpn2/ochre.html

Download ppt "December 5, 2012Introduction to Artificial Intelligence Lecture 20: Neural Network Application Design III 1 Example I: Predicting the Weather Since the."

Similar presentations

Artificial Neural Networks

Artificial Neural Networks
NEURAL NETWORKS Backpropagation Algorithm

NEURAL NETWORKS Backpropagation Algorithm
1 Machine Learning: Lecture 4 Artificial Neural Networks (Based on Chapter 4 of Mitchell T.., Machine Learning, 1997)

1 Machine Learning: Lecture 4 Artificial Neural Networks (Based on Chapter 4 of Mitchell T.., Machine Learning, 1997)
Artificial Intelligence 13. Multi-Layer ANNs Course V231 Department of Computing Imperial College © Simon Colton.

Artificial Intelligence 13. Multi-Layer ANNs Course V231 Department of Computing Imperial College © Simon Colton.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 7: Learning in recurrent networks Geoffrey Hinton.

CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 7: Learning in recurrent networks Geoffrey Hinton.
Introduction to Artificial Intelligence (G51IAI)

Introduction to Artificial Intelligence (G51IAI)
Reading for Next Week Textbook, Section 9, pp. 52-56 A User’s Guide to Support Vector Machines (linked from course website)

Reading for Next Week Textbook, Section 9, pp A User’s Guide to Support Vector Machines (linked from course website)
December 5, 2013Computer Vision Lecture 20: Hidden Markov Models/Depth 1 Stereo Vision Due to the limited resolution of images, increasing the baseline.

December 5, 2013Computer Vision Lecture 20: Hidden Markov Models/Depth 1 Stereo Vision Due to the limited resolution of images, increasing the baseline.
Simple Neural Nets For Pattern Classification

Simple Neural Nets For Pattern Classification
RBF Neural Networks x2 - + - 1 - + 2 - - - - x1 Examples inside circles 1 and 2 are of class +, examples outside both circles are of class – What NN does.

RBF Neural Networks x x1 Examples inside circles 1 and 2 are of class +, examples outside both circles are of class – What NN does.
Artificial Neural Networks ML 4.6-4.9 Paul Scheible.

Artificial Neural Networks ML Paul Scheible.
Radial Basis Functions

Radial Basis Functions
November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.

November 19, 2009Introduction to Cognitive Science Lecture 20: Artificial Neural Networks I 1 Artificial Neural Network (ANN) Paradigms Overview: The Backpropagation.
Neural Networks. R & G Chapter 8 8.1 Feed-Forward Neural Networks otherwise known as The Multi-layer Perceptron or The Back-Propagation Neural Network.

Neural Networks. R & G Chapter Feed-Forward Neural Networks otherwise known as The Multi-layer Perceptron or The Back-Propagation Neural Network.
MOVING AVERAGES AND EXPONENTIAL SMOOTHING

MOVING AVERAGES AND EXPONENTIAL SMOOTHING
September 21, 2010Neural Networks Lecture 5: The Perceptron 1 Supervised Function Approximation In supervised learning, we train an ANN with a set of vector.

September 21, 2010Neural Networks Lecture 5: The Perceptron 1 Supervised Function Approximation In supervised learning, we train an ANN with a set of vector.
November 30, 2010Neural Networks Lecture 20: Interpolative Associative Memory 1 Associative Networks Associative networks are able to store a set of patterns.

November 30, 2010Neural Networks Lecture 20: Interpolative Associative Memory 1 Associative Networks Associative networks are able to store a set of patterns.
October 5, 2010Neural Networks Lecture 9: Applying Backpropagation 1 K-Class Classification Problem Let us denote the k-th class by C k, with n k exemplars.

October 5, 2010Neural Networks Lecture 9: Applying Backpropagation 1 K-Class Classification Problem Let us denote the k-th class by C k, with n k exemplars.
October 14, 2010Neural Networks Lecture 12: Backpropagation Examples 1 Example I: Predicting the Weather We decide (or experimentally determine) to use.

October 14, 2010Neural Networks Lecture 12: Backpropagation Examples 1 Example I: Predicting the Weather We decide (or experimentally determine) to use.
October 7, 2010Neural Networks Lecture 10: Setting Backpropagation Parameters 1 Creating Data Representations On the other hand, sets of orthogonal vectors.

October 7, 2010Neural Networks Lecture 10: Setting Backpropagation Parameters 1 Creating Data Representations On the other hand, sets of orthogonal vectors.

Similar presentations

Ads by Google