Intro2CS Tirgul 11

What will we cover today? 2 Generator expression. Anonymous functions. Functional programming Map, Reduce, Filter

List Comprehensions – motivation Say you would like to get a list of squares, from some list/tuple/iterable. You would have probably written: But this seems like a lot of code for such a simple operation. In these cases you should remember your zen! Simple is better, but how can we simplify this? def get_squares(seq): lst = [] for x in seq: lst.append(x**2) return lst

List Comprehensions - 1 Is a way to create a list from another iterable/sequence, where each element in the created list: Satisfies a certain condition. Is a function of a member in the original sequence/iterabel. It allows for the creation of lists in an understandable syntax. You just write your expression inside the ‘[]’!

List Comprehensions - 2 Returning to our previous example: Could be simplified to: Much easier right? def get_squares(seq): lst = [] for x in seq: lst.append(x**2) return lst def get_squares(seq): return [x**2 for x in seq]

Advanced Comprehensions We saw some basic use cases of list comprehensions, lets try and look at some more advanced ones! We will use multiple modules available in the python3 library, you are more then welcome to look into them!

Pythagorean triads - listcomp A Pythagorean triad are 3 numbers x,y,z such that Lets say we want all Pythagorean triads existing in the range of a number. Meaning, all 3-tuples from the range One solution will be to use 3 for loops (or so) but another solution could be: from itertools import combinations as comb def pythagorean_in_range(n): return [(x,y,z) for x,y,z in comb(range(1,n+1),3) \ if x**2 + y**2 == z**2] iterator

Generator expression - 1 A generator expression is created by changing the list comprehensions ‘[]’ to ‘()’ In both cases 0,3,6,9 will be printed Actually if x is a generator expression, using list(x) will result the same as using a list comp. res = [i for i in range(10) if not i%3] for t in res: print t res2 = (i for i in range(10) if not i%3) for t in res2: print t [i for i in range(10) if not i%3] == list(i for i in range(10) if not i%3)

Perfect integers - listcomp A number, x, is a factor of another number, n, if We say that an integer is “perfect” if it equals the sum of all its factors (excluding itself). Given a number N we would like to find all perfect integers in the range: genexp def find_perfects(N): return [num for num in range(1,N+1) \ if sum( \ (fac for fac in range(1,num) if num%fac==0) \ ) == num]

Generator expression - 2 10 So is a genexp the same as a listcomp? Well the short answer is, no. The longer answer: Genexp could be used only once. You cannot access a genexp using the [], or any other list method for that manner: You cannot store any of the resulted information. Like generator, a genexp requires less memory. So pick the one most suitable for you! res = [i for i in range(10) if not i%3] res2 = (i for i in range(10) if not i%3) res[2] #works res2[2] #fails http://stackoverflow.com/questions/11964130/list-comprehension-vs-generator-expressions-weird-timeit-results http://stackoverflow.com/questions/23297702/tuple-on-genexp-vs-listcomp http://stackoverflow.com/questions/2173845/why-is-this-genexp-performing-worse-than-a-list-comprehension 10

One liners: Reversing a dictionary, say mydict: 11 Reversing a dictionary, say mydict: dict( [ (val,key) for key,val in mydict.items() ] ) Getting a list of N inputs from the user: [ input(“Enter input: “) for inpt in range(inpt_num) ] Reverse all words in a string: " ".join(text.split()[::-1]) (or) "".join(reversed(text.split()))

Lazy loading - Generator 12 def read_in_chunks(file_object, chunk_size=1024): """Lazy function (generator) to read a file piece by piece. Default chunk size: 1k.""" while True: data = file_object.read(chunk_size) if not data: break yield data f = open('really_big_file.dat') for piece in read_in_chunks(f): process_data(piece)

Generetors of files 13 log_lines = (line for line in huge_log_file.readlines() if complex_condition(line)) def get_log_lines(log_file): line = log_file.readline() while True: try: if complex_condition(line): yield line line = log_file.readline() except StopIteration: raise log_lines = get_log_lines(log_file)

Lambda Notation Python’s lambda creates anonymous functions >>> f1= lambda x: 2*x + 1 >>> f1(10) 21 >>> f2 = lambda x,y: x if x>y else y >>> f2(3, 5) 5 >>> f3= lambda x,y,z: x in z and y in z >>> f3('a','d',['b','a','c']) False >>> f3('a',‘c',['b','a','c']) True

Lambda Notation Limitations Note: only one expression in the lambda body; Its value is always returned The lambda expression must fit on one line!

Functions are first-class objects A higher-order function is a function that takes another function as a parameter They are “higher-order” because it’s a function of a function h Python has functions as first-class citizens, so you can simply pass the functions by name These are often used with lambda

Higher-Order Functions Functions are first-class objects Functions can be used as any other datatype, eg: Arguments to function Return values of functions Assigned to variables Parts of tuples, lists, etc Examples Map Reduce Filter

Example: minmax def minmax(test, *args): res = args[0] for arg in args[1:]: if test(arg, res): res = arg return res def lessthan(x, y): return x < y def grtrthan(x, y): return x > y print(minmax(lessthan, 4, 2, 1, 5, 6, 3) ) 1 print(minmax(grtrthan, 4, 2, 1, 5, 6, 3) ) 6

Lazy loading - Iterator 19 def read_in_chunks(file_object, chunk_size=1024): def read1k(): return file_object.read(chunk_size) for piece in iter(read1k, ''): process_data(piece) f = open('really_big_file.dat') read_in_chunks(f)

Newton-Raphson method Figure 1 Geometrical illustration of the Newton-Raphson method. http://numericalmethods.eng.usf.edu

Newton-Raphson method Figure 2 Derivation of the Newton-Raphson method. http://numericalmethods.eng.usf.edu

Newton's method def derivative(f, x, h): deriv = (f(x+h)-f(x-h))/(2*h) return deriv def solve(f, x0, h): xn = x0 prev = x0+10 while (abs(xn - prev) > h): prev = xn print(xn) xn = xn - (f(xn))/derivative(f, xn, h) return xn print(solve(lambda x:x**2+x-1,0,0.01)) 0.9999999999999991 0.666666666666667 0.6190476190476192 0.6180344478216818

Map map map(function, sequence) Map applies function to each element of sequence and creates a iterable object of the results You can optionally provide more iterables as parameters to map and it will place tuples in the result list Map returns an iterator which can be cast to list

Map Problem map Goal: given a list of three dimensional points in the form of tuples, create a new list consisting of the distances of each point from the origin e.g., points = [(2, 1, 3), (5, 7, -3), (2, 4, 0), (9, 6, 8)] - distance(x, y, z) = sqrt(x**2 + y**2 + z**2) - loop through the list and add results to a new list from math import sqrt def distance(point) : x, y, z = point return sqrt(x**2 + y**2 + z**2) points = [(2, 1, 3), (5, 7, -3), (2, 4, 0), (9, 6, 8)] distances = list(map(distance, points))

Generator of linked list class Node: def __init__(self, data=None, next=None): self.__data = data self.__next = next def set_next(self,next): self.__next = next def get_next(self): return self.__next def get_data(self): return self.__data class LinkedList: def __init__(self, head): self.__head = head def get_head(self): return self.__head

Generator of a linked list def list_iter(linked_list): head = linked_list.get_head() cur = head while cur!= None: try: yield cur.get_data() cur = cur.get_next() except StopIteration: return

Map of linked list A = Node(1) B = Node(3) C = Node(5) D = Node(7) A.set_next(B) B.set_next(C) C.set_next(D) l = LinkedList(A) same_list = LinkedList(Node(1,Node(3,Node(5,Node(7))))) lst = list(map(lambda x:x**2, list_iter(same_list))) print(lst) [1, 9, 25, 49]

Filter filter filter(function, seq) The filter runs through each element of sequence (any iterable object such as a List or another collection) It applies function to each element of iterable If function returns True for that element then the element is put into a List In python 3, filter returns an iterator which must be cast to type list with list()

Filter Problem filter Goal: given a list of lists containing answers to an algebra exam, filter out those that did not submit a response for one of the questions, denoted by “nan” scores = [["nan", 12, .5, 78,], [2, 13, .5, .7], [2, "nan", .5, 78], [2, 14, .5, 39]]

filter Filter Problem Import math scores = [["nan", 12, .5, 78,], [2, 13, .5, .7], [2, "nan", .5, 78], [2, 14, .5, 39]] #solution 1 - intuitive def has_NaN(answers) : for num in answers : if math.isnan(float(num)) : return False return True valid = list(filter(has_NaN, scores)) print(valid) #Solution 2 – sick python solution valid = list(filter(lambda x : "nan" not in x, scores)) [[2, 13, 0.5, 0.7], [2, 14, 0.5, 39]]

Tree Generator Filter Problem class TreeNode: def __init__(self, value, left=None, right=None): self.value = value self.left = left self.right = right def in_order(self): if self.left is not None: for v in self.left.in_order(): yield v yield self.value if self.right is not None: for v in self.right.in_order():

Filter - Tree Generator t=TreeNode(1,TreeNode(2,TreeNode(3),TreeNode(4)), TreeNode(5,TreeNode(6))) for v in t.in_order(): print(v) l = filter(lambda x: x%3==0, t.in_order()) print(list(l))

reduce Reduce reduce(function, seq) The function reduce(func, seq) continually applies the function func() to the sequence seq. It returns a single value.  function must take two parameters In python 3 reduce() requires an import statement from functools import reduce

Reduce reduce If seq = [ s1, s2, s3, ... , sn ], calling reduce(func, seq) works like this: At first the first two elements of seq will be applied to func, i.e. func(s1,s2) The list on which reduce() works looks now like this: [ func(s1, s2), s3, ... , sn ] In the next step func will be applied on the previous result and the third element of the list, i.e. func(func(s1, s2),s3) The list looks like this now: [ func(func(s1, s2),s3), ... , sn ] Continue like this until just one element is left and return this element as the result of reduce()

reduce Reduce Example Example: what does this reduce function returns? nums = [1, 2, 3, 4, 5, 6, 7, 8] nums = list(reduce(lambda x, y : (x, y), nums)) Print(nums) (((((((1, 2), 3), 4), 5), 6), 7), 8)