1. Primer on Neural networks
Brooke aker

2. What are Models & Machine Learning ?
All models are routines that minimize the error between observations and a fitted line.
Models become equations that accept inbound data to form a prediction.
Machine Learning is a model that automatically changes over time with more data.

3. Neural network (generally) use gradient decent methods

4. How to give neural networks a try in R
Background: compare a linear regression model to Neural Network model
We are going to use the Boston dataset in the MASS package. The Boston dataset is a collection of data about housing values in the suburbs of Boston. Our goal is to predict the median value of owner-occupied homes (medv) using all the other continuous variables

5. Start with a linear model for comparison
We proceed by randomly splitting the data into a train and a test set, then we fit a linear regression model and test it on the test set. Note that I am using the gml() function instead of the lm() this will become useful later when cross validating the linear model.

6. Now prepare the Neural networks
NN work best when data is scaled into 0-1 or -1 to 1 scales. To scale your data do this: find the max and mins
maxs <- apply(data, 2, max)
mins <- apply(data, 2, min)
Now scale the data - e.g. in this case where the center point of each variable is the min# and where the max goes no further than the difference max less min. # note scale needs to be converted back to data frame
scaled <- as.data.frame(scale(data, center = mins, scale = maxs - mins))
Set new training and testing data frames specific to NN
train_ <- scaled[index,]test_ <- scaled[-index,]

7. Layers and Neurons !!
neurons

8. Layers and Neurons !!
Not a good understanding of how to formulate layers and neurons except following rule of thumb;
1-2 layers are sufficient for most Neural Networks
Try neurons as 2/3rd the input layer size in first layer, 2nd layer neurons should be 2/3rd of first layer neuron size. For Boston housing data we have 13 data variables
n <- names(train_)
f <- as.formula(paste("medv ~", paste(n[!n %in% "medv"], collapse = " + ")))
nn <- neuralnet(f,data=train_,hidden=c(8,5),linear.output = T)
plot(nn)

9. Plot(nn)
The black lines show the connections between each layer and the weights on each connection while the blue lines show the bias term added in each step. The bias can be thought as the intercept of a linear model.
The net is essentially a black box so we cannot say that much about the fitting, the weights and the model. But the training algorithm has converged and therefore the model is ready to be used.

10. Then calc a mean squared error for the nn
Do the NN prediction with the test_ data
pr.nn <- compute(nn,test_[,1:13])
Rescale the data back to its original form so the MSE comparison is valid
pr.nn_ <- pr.nn$net.result*(max(data$medv)-min(data$medv))+min(data$medv)
test.r <- (test_$medv)*(max(data$medv)-min(data$medv))+min(data$medv)
Get the MSE of the NN
MSE.nn <- sum((test.r - pr.nn_)^2)/nrow(test_)

11. Finally compare the linear to nn model mse
Linear Model MSE
Neural Network MSE
21.6
7.1

12. Compare errors graphically
plot(test$medv,pr.nn_,col='red',main='Real vs predicted NN',pch=18,cex=0.7)abline(0,1,lwd=2)legend('bottomright',legend='NN',pch=18,col='red', bty='n')
plot(test$medv,pr.lm,col='blue',main='Real vs predicted lm',pch=18, cex=0.7)abline(0,1,lwd=2)legend('bottomright',legend='LM',pch=18,col='blue', bty='n', cex=.95)

13. conclusions Questions
Neural Networks are (one of) the basis for Artificial Intelligence and can produce superior outcomes compared to more traditional methods.
Subject to experimentation
Are black boxes
How to do feature selection / engineering?
Does the neural network experiment with which variables to include / exclude?
What conclusion to draw if MSE is larger than traditional methods?