Primer on Neural networks Brooke aker 10-27-16

What are Models & Machine Learning ? All models are routines that minimize the error between observations and a fitted line. Models become equations that accept inbound data to form a prediction. Machine Learning is a model that automatically changes over time with more data.

Neural network (generally) use gradient decent methods

How to give neural networks a try in R Background: compare a linear regression model to Neural Network model We are going to use the Boston dataset in the MASS package. The Boston dataset is a collection of data about housing values in the suburbs of Boston. Our goal is to predict the median value of owner-occupied homes (medv) using all the other continuous variables

Start with a linear model for comparison We proceed by randomly splitting the data into a train and a test set, then we fit a linear regression model and test it on the test set. Note that I am using the gml() function instead of the lm() this will become useful later when cross validating the linear model.

Now prepare the Neural networks NN work best when data is scaled into 0-1 or -1 to 1 scales. To scale your data do this: find the max and mins maxs <- apply(data, 2, max) mins <- apply(data, 2, min) Now scale the data - e.g. in this case where the center point of each variable is the min# and where the max goes no further than the difference max less min. # note scale needs to be converted back to data frame scaled <- as.data.frame(scale(data, center = mins, scale = maxs - mins)) Set new training and testing data frames specific to NN train_ <- scaled[index,]test_ <- scaled[-index,]

Layers and Neurons !! neurons

Layers and Neurons !! Not a good understanding of how to formulate layers and neurons except following rule of thumb; 1-2 layers are sufficient for most Neural Networks Try neurons as 2/3rd the input layer size in first layer, 2nd layer neurons should be 2/3rd of first layer neuron size. For Boston housing data we have 13 data variables n <- names(train_) f <- as.formula(paste("medv ~", paste(n[!n %in% "medv"], collapse = " + "))) nn <- neuralnet(f,data=train_,hidden=c(8,5),linear.output = T) plot(nn)

Plot(nn) The black lines show the connections between each layer and the weights on each connection while the blue lines show the bias term added in each step. The bias can be thought as the intercept of a linear model. The net is essentially a black box so we cannot say that much about the fitting, the weights and the model. But the training algorithm has converged and therefore the model is ready to be used.

Then calc a mean squared error for the nn Do the NN prediction with the test_ data pr.nn <- compute(nn,test_[,1:13]) Rescale the data back to its original form so the MSE comparison is valid pr.nn_ <- pr.nn$net.result*(max(data$medv)-min(data$medv))+min(data$medv) test.r <- (test_$medv)*(max(data$medv)-min(data$medv))+min(data$medv) Get the MSE of the NN MSE.nn <- sum((test.r - pr.nn_)^2)/nrow(test_)

Finally compare the linear to nn model mse Linear Model MSE Neural Network MSE 21.6 7.1

Compare errors graphically plot(test$medv,pr.nn_,col='red',main='Real vs predicted NN',pch=18,cex=0.7)abline(0,1,lwd=2)legend('bottomright',legend='NN',pch=18,col='red', bty='n') plot(test$medv,pr.lm,col='blue',main='Real vs predicted lm',pch=18, cex=0.7)abline(0,1,lwd=2)legend('bottomright',legend='LM',pch=18,col='blue', bty='n', cex=.95)

conclusions Questions Neural Networks are (one of) the basis for Artificial Intelligence and can produce superior outcomes compared to more traditional methods. Subject to experimentation Are black boxes How to do feature selection / engineering? Does the neural network experiment with which variables to include / exclude? What conclusion to draw if MSE is larger than traditional methods?