Neural Networks Weka Lab Villanova University Machine Learning Project

Weka Practice We are going to explore using Weka for neural nets The most common NN classifier in Weka is the Multilayer Perceptron, which is located under functions. And we will us the weather.numeric arff file.

First let’s look at the data Open Weka and open weather.numeric.arff in the Preprocess tab. Click on the “Edit…” button to see the actual data. In the Viewer window you can reorder the data by clicking on a column head. And in the window on the bottom right you can click on Visualize All to see each attribute mapped against “Play” as a class

Edit window, reordered This has been modified to order by Play, the highlighted column. Reordering by various columns makes it easier to look at the effect of any one attribute.

Visualize All window Looking at these it’s pretty clear that none of the predictive attributes works well by itself. Humidity or outlook probably does better than the others. In fact, OneR uses outlook. But it’s not very good.

Question 1: How does it look? Just looking at it, can you predict Play easily? Which do you think will do better, ZeroR or OneR? Try the default for both, using cross- validation and the default settings. Which did better? Not easy. ZeroR does better, at 65%, because we have more “play” than “don’t play”. OneR only gives 42% accuracy.

Question 2: More classifiers Try J48 and IBk. What results did you get? For J48 the default is 2 instances/leaf. We have a very small dataset. Does letting it be a single leaf do better (set minNumObj to 1)? For IBk we the default number of neighbors is 1 (KNN). Will more neighbors do better? Try it. Why do you think you got what you did? J48 default is 65%. IBk is about 78%.

Question 3: Okay, on to Neural Nets First let’s look at the standard Weka Interface. In the Classify tab, under Functions choose Multilayer Preceptron as the classifier. Try it with the default options. What accuracy do you get? How many nodes are described? How many hidden layers does this NN have? Accuracy is 78.57. 6 nodes, zero through five. One hidden layer; Zero and One are the output nodes for yes and no, respectively.

Question 4:Applying the model Right-click on the result in the result list to visualize the classifier errors. Rectangles are errors, Xs are correct. You probably want to increase jitter so you can see instances more clearly Look at the accurate predictions (Xs). Chick on one to see the actual instance data. Where did most of the Xs fall? In this case, the currect predictions are largely in the “yes/yes” corner.

Question 5: Classifier Errors This shows instance 10. 1 and 9 are also errors. Classifier Errors are rectangles. Clicking on one opens that instance. Which three instances are the errors?

Question 6: Different Parameters In the GenericObjectEditor you can change the default parameters for the classifier. Try different number of hidden layers and nodes in each layer. learning rate and momentum training time Did you get your accuracy to change? . hiddenLayers -- This defines the hidden layers of the neural network. This is a list of positive whole numbers. 1 for each hidden layer. Comma separated. To have no hidden layers put a single 0 here. For hidden layers, 4,4,4 says three hidden layers with four nodes each. Note that for this small sample size there are not a lot of changes based on these options.

Question 7: A bigger data set Load the diabetes dataset, which has many more instances, and experiment again with the options. Did you see an effect? Can you improve on the default accuracy? Note: these can take a while. Initial results start showing up, but if the STOP button is not grayed out it’s still working. The default parameters give 75.4% accuracy. Students should be able to improve somewhat on this. For instance, 2,2,2 will give 77.3% accuracy.

Question 8.The NN GUI There is a neural net GUI, selected by the first option in the GenericObjectEditor. This editor can be used to change the NN on the fly, while it is being built. Section 11.4, pages 469-472, in the Weka text describes how to use this gui. Work through the examples on those pages. How good a network can you build?(measured by accuracy). Save and submit your final model. The Weka text is Data Mining: Practical Machine Learning Tools and Techniques, Third Edition. Ian H. Witten , Eibe Frank, Mark A. Hall, Morgan Kaufmann, 2011) ISBN-10: 0123748569