Coding in Python and Basics of Tensorflow Dr. Hien NGUYEN Ilker GURCAN (Teaching Assistant)

Coding in Python-Hello World Commenting and Print Function # For a single line comment ‘’’ If you want to have multiple lines of comment, then you may use triple quotes ‘’’ print(‘You may print out using %s function for printing %d number of lines, or any number of lines ’, %(‘print()’, 10)) print(‘\n’ * 3) # You may also duplicate strings print(”I don’t like ”, end=“”) print(“new lines”)

Coding in Python-Hello World List groceryList = [‘Juice’, ‘Tomatoes’, ‘Potatoes’] otherEvents = [‘Wash car’, ‘Pick up kids’, ‘Cash check’] toDoList = [otherEvents, groceryList] print(‘What I should do’, toDoList[0][1]) # Will print out “Tomatoes” groceryList.append(item) groceryList.insert(index, item) groceryList.remove(…) del otherEvents[2] And many others such as sort, reverse, etc. You have other methods which could be applied to container-typed classes such as : len(list1), max(list2), min(listN), etc.

Coding in Python-Hello World Tuples piTuple = {3, 1, 4, 1, 5, 9} # Unmodifiable list newList = list(piTuple) # Convert to list newTuple = tuple(newList) Map superHeroes = {‘Fiddler’ : ‘Isaac Chaney’, ‘iliTheBlack’ : ‘Ilker GURCAN’, ’Tulu Girl’ : ‘Tulun ERGIN’} print(superHereos[‘iliTheBlack’]) # Will print out “Ilker GURCAN” print(superVillains.keys()) print(superVillains.values()) superVillains['Tulu Girl'] = ’Michael Jordan’ del superVillains[‘Fiddler’]

Coding in Python-Hello World Loops and Conditions: Looping through an iterable object: for x in range(0, 10): print(x, ' ', end="") for y in groceryList: print(y, ' ', end="") for x in [1,2,4,7,9]: print(x) An example for both conditionals and while loop: i=0 while(i<20): if(i%2 == 0): print(i) elif(i == 9): break else: i += 1 # i = i + 1 continue i += 1 # This will be ignored when else block is executed

Coding Python-Methods Method Signature: def methodName(method_args=default_args): doSomething [return return_val] Calling a function: methodName(args) No notion of call by value or pass by reference as in C/C++. Every immutable object is passed by its value; while all mutable objects are passed by so called “object”. Check this link for more info.

Coding Python-Classes class Animal: __name = "" # Double underscore means that your field is private __height = 0 __weight = 0 __sound = 0 def __init__(self, name, height, weight, sound): # Constructor self.__name = name self.__height = height self.__weight = weight self.__sound = sound def setName(self, name): self.__name = name def getName(self): return self.__name def getType(self): print("Animal") # For the sake of polymorphism def getSound(self): return self.__sound def toString(self): return "{} is {} cm tall and {} kilograms and say {}” .format(self.__name, self.__height, self.__weight, self.__sound)

Coding Python-Inheretance class Dog(Animal): # Inheritance __owner = "" def __init__(self, name, height, weight, sound, owner): self.__owner = owner super(Dog, self).__init__(name, height, weight, sound) def setOwner(self, owner): self.__owner = owner def getType(self): # Overridden print("Dog") # Remember it for polymorphism def toString(self): # Overridden parStr = super(Dog, self).toString() return (parStr + ". His owner is {}").format(self.__owner) def multipleSounds(self, howMany=None): # New Method if howMany is None: print(self.getSound()) else: print(self.getSound() * howMany)

Coding Python-Polymorphism class AnimalTesting: def getType(self, animal): animal.getType() Create two Animals: cat = Animal('Whiskers', 33, 10, 'Meow') print(cat.toString()) dog = Dog("Newton", 110, 50, "Hoowww", "iliTheBlack") print(dog.toString()) print(dog.multipleSounds(4)) Now where polymorphism comes into play: testAnimal = AnimalTesting() testAnimal.getType(cat) # Will work regardless of its type and print out “Animal” testAnimal.getType(dog) # Will work regardless of its type and print out “Dog”

Tensorflow An open source software library for numerical computation using data flow graphs. Works on both CPU and GPU Multi platform support: Desktop, Server or Mobile device Purpose initially was to conduct research on machine learning and neural networks. Now applicable to many domains. You may even build your graph to solve Dijkstra’s shortest path algorithm. A substitution for Torch library

Tensorflow-Data Structures Constants tf.zeros(…), tf.ones(…), and tf.constant(…) const_val = tf.constant(9, dtype=tf.float32, shape=[2, 2]) # [[9, 9], [9, 9]] Placeholders Placeholders are used to feed graphs with the input data. As its name suggests, they are placeholders for actual tensors. x = tf.placeholder(tf.float32, shape=(1024, 1024)) y = tf.matmul(x, x) with tf.Session() as sess: rand_array = np.random.rand(1024, 1024) sess.run(y, feed_dict={x: rand_array})

Tensorflow-Data Structures Variables To train a model you should use variables (weights, biases, counters, etc.). They can be saved to disk. They can be restored from disk. They are used to hold and update parameters of your model. There are some important concepts specific to variables: Creation Device placement Initialization Saving/Restoring Sharing them across separate model executions

Tensorflow-Data Structures Variable Creation: tf.Variable(aTensor, name=“weights”) You may use either built in functions to create a tensor: aTensor = tf.random_normal(…) / tf.zeros(…) / tf.range(…),… etc. OR a python array to create a tensor aTensor = tf.constant(aPythonArray) Calling function Variable(…) adds several ops to the graph object: variable op holding variable’s value Initializer op that sets the variable to its initial value. In fact it is tf.assign operation.

Tensorflow-Data Structures Device placement of Variables: with tf.device(“/cpu:0”) # You may use any enumerated device v = tf.Variable(…) Note that however; parameter update operations and optimizer must be run on the same device as on where these variables have been created. Initialization: Variable initializaters must be run before everything else. Since it is tedious operation to initialize all variables; tensorflow has a global initializer function used as the following: init_op = tf.global_variable_initializer() with tf.Session() as sess: sess.run(init_op)

Graph Building Process Tensorflow is composed of several mechanics. In fact any meaningful neural network would have these following three components: Inference: Building the model, defining all operations carried out by the graph. Including: Weights Biases Activation functions Net functions Loss: How we define the loss w.r.t. the actual output and the one gathered from network. The purpose is to minimize this loss function using an optimizer. There various built-in cost functions in tensorflow: Softmax loss Cross entropy loss MSE … Train: It is mainly composed of (but not restricted to) following operations: Defining summary writers Creating optimizer: Gradient Descent, SGD, Adam, etc. Associating loss with the optimizer and returning optimizer to be run on the session

Visualizing Learning Tensorboard A tool to visualize the output of your graph. You may attach multiple summary collectors to a You may visualize the outcome of a particular layer by monitoring variables: def variable_summaries(var):   """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""   with tf.name_scope('summaries'):     mean = tf.reduce_mean(var)     tf.summary.scalar('mean', mean)     with tf.name_scope('stddev'):       stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))     tf.summary.scalar('stddev', stddev)     tf.summary.scalar('max', tf.reduce_max(var))     tf.summary.scalar('min', tf.reduce_min(var))     tf.summary.histogram('histogram', var)

Visualizing Learning You may also attach summary writer to your loss function; so that you may visualize how your cost function behaves. with tf.name_scope('cross_entropy'):   # The raw formulation of cross-entropy,   #   # tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(tf.softmax(y)),   #                               reduction_indices=[1]))   #   # can be numerically unstable.   #   # So here we use tf.nn.softmax_cross_entropy_with_logits on the   # raw outputs of the nn_layer above, and then average across   # the batch.   diff = tf.nn.softmax_cross_entropy_with_logits(targets=y_, logits=y)   with tf.name_scope('total'):     cross_entropy = tf.reduce_mean(diff) tf.summary.scalar('cross_entropy', cross_entropy)

Visualizing Learning Writing into a log file which is read and parsed by tensorboard: summary = tf.summary_merge_all() with tf.Session() as sess: summary_writer = tf.summary.FileWriter(FLAGS.log_dir, sess.graph) … summary_str = sess.run(summary, feed_dict=feed_dict) summary_writer.add_summary(summary_str, step) summary_writer.flush() After training is complete, run the following command to visualize results on a web browser (ensure that there is no space on both sides of the assignment operator): tensorboard --logdir=path/to/logFolder

Useful Links Python Tutorial – Language basics Numpy Tutorial – All tensorflow array operations are inspired by numpy Python Collections – Lists, Queues, Maps, etc. Installing Tensorflow on Ubuntu – You may also find installation for different platforms on the link