1. MNIST Dataset Training with Tensorflow
Avinash More

2. Tensorflow
Example of numpy : expensive operations such as matrix multiplication outside Python, using highly efficient code implemented in another language
still be a lot of overhead from switching back to Python every operation
especially bad if you want to run computations on GPUs or in a distributed manner, where there can be a high cost to transferring data
Instead of running a single expensive operation independently from Python, TensorFlow lets us describe a graph of interacting operations that run entirely outside Python

3. Dataset MNIST data set is available of Yann LeCun’s website.
It has images
3 Parts:
55,000 data points of training data (mnist.train)
10,000 points of test data (mnist.test)
5,000 points of validation data (mnist.validation)
This split is important because we want to make sure that model we generate can be generalized.

4. Data point Each MNIST data point has 2 parts
Image of a handwritten digit (mnist.train.images)
Corresponding label (mnist.train.label)

5. Mnist data – image 28 * 28 Big array of numbers
We can flatten this array into a vector of 28 * 28 = 784 pixels
Flattening will result into loss of 2 D structure but for softmax function it won’t matter.
So the result would be a tensor of dimension [55000, 784]

6. Mnist data - Image Label
a number between 0 and 9 representing the digit drawn in the image
“one-hot-vector” - a vector which is 0 in most dimensions, and 1 in a single dimension
For example, 3 would be [0,0,0,1,0,0,0,0,0,0]
Dimension of mnist.train.labels is a [55000, 10]

7. Softmax Regression
Goal: to look at an image and give the probabilities for it being each digit
look at a picture of a nine and be 80% sure it's a nine, but give a 5% chance to it being an eight (because of the top loop) and a bit of probability to all the others because it isn't 100% sure
If you want to assign probabilities to an object being one of several different things, softmax is the thing to do, because softmax gives us a list of values between 0 and 1 that add up to 1

8. Tensorflow terminologies
Placeholder: A placeholder is simply a variable that we will assign data to at a later date. It allows us to create our operations and build our computation graph, without needing the data. In TensorFlow terminology, we then feed data into the graph through these placeholders.
A Variable is a modifiable tensor that lives in TensorFlow's graph of interacting operations. It can be used and even modified by the computation.

9. Tensoflow graph and session
TensorFlow separates definition of computations from their execution
tf.Graph = A graph defines the computation. It doesn’t compute anything, it doesn’t hold any values, it just defines the operations that you specified in your code.
Tf.session: A session allows to execute graphs or part of graphs. It allocates resources (on one or more machines) for that and holds the actual values of intermediate results and variables.