Python Foundations by Chris Gahan (( In Stereo where Available ))

TRICK #1 Dont Repeat Yourself Dont Repeat Yourself If everything is defined once, then radically changing the behaviour of your program is trivial. If everything is defined once, then radically changing the behaviour of your program is trivial.

TRICK #2 Get your program running on real data as soon as possible Get your program running on real data as soon as possible

TRICK #3 Test it! Test it! Even if you dont have fancy test cases, make some little bit of code at the end of the program to test its functionality as youre developing it. Even if you dont have fancy test cases, make some little bit of code at the end of the program to test its functionality as youre developing it. I like to break my programs into a bunch of modules with tests in the if __name__ == __main__: block I like to break my programs into a bunch of modules with tests in the if __name__ == __main__: block when you execute the file, the tests run when you execute the file, the tests run when you import it as a module, the tests are ignored when you import it as a module, the tests are ignored

TRICK #4 Use as much of Pythons libraries and built-in functionality as you can Use as much of Pythons libraries and built-in functionality as you can Pythons built-in stuff is a solid foundation Pythons built-in stuff is a solid foundation There are solutions to many many problems in the standard library There are solutions to many many problems in the standard library Use lists and dicts for as much as you can Use lists and dicts for as much as you can writing custom classes when you can use a list or dict will overcomplicate your code writing custom classes when you can use a list or dict will overcomplicate your code

TRICK #5 The Zen of Python The Zen of Python Execute import this in the interpreter Execute import this in the interpreter

Things you should know Exceptions are awesome Exceptions are awesome But dont over-use them But dont over-use them Decorators are cool Decorators are cool But you rarely need them But you rarely need them List comprehensions and Generators are wicked List comprehensions and Generators are wicked Use them as much as possible! Use them as much as possible! Rethinking your problems as a series of generators or list transformations is an excellent habit to get into Rethinking your problems as a series of generators or list transformations is an excellent habit to get into

More things you should know (about performance) Appending strings is slow Appending strings is slow Use lists and.join(list) them! Use lists and.join(list) them! Searching lists is slow Searching lists is slow Use sets or dicts! Use sets or dicts! Comparing lists is slow Comparing lists is slow Use sets! Use sets! List comprehensions are fast! List comprehensions are fast! Use them as much as possible Use them as much as possible If your program still isnt fast enough, use Psyco or Pyrex! If your program still isnt fast enough, use Psyco or Pyrex! Psyco optimizes code automatically (faster) Psyco optimizes code automatically (faster) Pyrex generates C code which can be compiled (fastest) Pyrex generates C code which can be compiled (fastest)

Everything is an object Lists are objects Lists are objects Functions are objects Functions are objects Classes are objects Classes are objects Types are objects (IntType, ClassType, TypeType, etc.) Types are objects (IntType, ClassType, TypeType, etc.) Dicts are objects Dicts are objects Objects are objects Objects are objects Internally, all Objects are secretly Dicts! Internally, all Objects are secretly Dicts!

Lists This is your bread and butter in Python. Lists are everything: This is your bread and butter in Python. Lists are everything: list = [1,2,3,4,Ninja(),rumplestiltskin] list = [1,2,3,4,Ninja(),rumplestiltskin] Handy methods: Handy methods: list.append(5) list.append(5) Stick 5 on the end of the list Stick 5 on the end of the list list.extend([5,6,7]) list.extend([5,6,7]) Append the contents of [5,6,7] to the list Append the contents of [5,6,7] to the list list.pop(position) list.pop(position) Yank off an element from the list and hand it to you. If you dont specify the position, it pops off the last element. Yank off an element from the list and hand it to you. If you dont specify the position, it pops off the last element. list.reverse() list.reverse() Reverse the list Reverse the list list.sort() list.sort() Sort the list Sort the list list.index(item) list.index(item) Return the position of item in the list Return the position of item in the list

Slicing Lets you grab chunks from a list or a string. Lets you grab chunks from a list or a string. list[5:9] list[5:9] Grabs elements 5,6,7,8 Grabs elements 5,6,7,8 list[-3:-1] list[-3:-1] Negative indexes mean from the end, -1 being first from the end, -2 being second from the end, etc.. So this would grab two elements: the 3 rd and 2 nd from the end. Negative indexes mean from the end, -1 being first from the end, -2 being second from the end, etc.. So this would grab two elements: the 3 rd and 2 nd from the end. list[:4] list[:4] Grabs elements 0,1,2,3 Grabs elements 0,1,2,3 list[6:] list[6:] Start at element 6 and grab all the rest of the elements until the end Start at element 6 and grab all the rest of the elements until the end list[:] list[:] Grab all elements (make a copy of the entire thing) Grab all elements (make a copy of the entire thing) Note: The off-by-one thing (where it ignores the last element in the slice) is there for a good reason – it makes slice math super easy! (Its the same reason that range(a,b) ignores the last element b)

Slicing Strings Since strings are also technically lists of characters, they can also be sliced Since strings are also technically lists of characters, they can also be sliced To copy a string, take a slice thats the entire string: To copy a string, take a slice thats the entire string: newString = oldString[:] newString = oldString[:]

Trivia Did you know that slices are actually OBJECTS? Its TRUE! Did you know that slices are actually OBJECTS? Its TRUE! You can make an object that returns its own custom slice objects which return anything you want You can make an object that returns its own custom slice objects which return anything you want SQLObject does this when you slice a database query result – it uses LIMIT and OFFSET to get the chunk of the slice, resulting in mad speed! SQLObject does this when you slice a database query result – it uses LIMIT and OFFSET to get the chunk of the slice, resulting in mad speed!

Stupid Tuple Tricks Tuples are like lists, but static Tuples are like lists, but static once you define a tuple, you cant modify it once you define a tuple, you cant modify it Whenever you use a comma in Python, a tuple is secretly created. For example: Whenever you use a comma in Python, a tuple is secretly created. For example: a = 1, 2, 3 a = 1, 2, 3 open(file.txt, r) open(file.txt, r) person1, person2 = Muhammed, Shareef person1, person2 = Muhammed, Shareef a, b = b, a a, b = b, a Internally, function parameters are tuples. A clever way to swap two variables Assigning two values at once

Kinds of Strings Different kinds of strings: Different kinds of strings: Raw strings ignore escape codes (\n, \x, etc) Raw strings ignore escape codes (\n, \x, etc) Great for regexes – you dont have to go: \\t, \\w, \\s, etc.. Great for regexes – you dont have to go: \\t, \\w, \\s, etc..\\t\\w\\s\\t\\w\\s

String operations s.replace(victim, replacement) s.replace(victim, replacement) Replace all occurrences of victim Replace all occurrences of victim s.strip([chars]) s.strip([chars]) Remove leading/trailing whitespace and newlines (or, remove leading/trailing [chars]) Remove leading/trailing whitespace and newlines (or, remove leading/trailing [chars]) s.split([delimiter]) s.split([delimiter]) Break a string up into pieces and return them as a list of strings. The delimiter is whitespace by default. Break a string up into pieces and return them as a list of strings. The delimiter is whitespace by default. You want more? Hit TAB in ipython! You want more? Hit TAB in ipython!

Formatting Text for Output Want to format text for output? Want to format text for output? If you have a bunch of elements you want to print nicely, use,.join(list) If you have a bunch of elements you want to print nicely, use,.join(list) If you want to print strings, instead of: If you want to print strings, instead of: hi + name +, how are you? hi + name +, how are you? …you can do: …you can do: hi %s, how are you? % name hi %s, how are you? % name Or Or hi %(name)s, how are you? % locals() hi %(name)s, how are you? % locals()

String Interpolation String interpolation lets you substitute placeholders (%s, %d, etc.) in a string (just like printf in C) String interpolation lets you substitute placeholders (%s, %d, etc.) in a string (just like printf in C)

Why string interpolation? Because when you have lots of variables in the output, it becomes annoying to keep track of where your quotes end. (Also, typing ++ requires hitting shift a lot – its easy to mess it up). Because when you have lots of variables in the output, it becomes annoying to keep track of where your quotes end. (Also, typing ++ requires hitting shift a lot – its easy to mess it up). name: +name+, age: +age+, city: +city+, province: +province+, IP: +ip+, favorite colour: +favorite_colour name: +name+, age: +age+, city: +city+, province: +province+, IP: +ip+, favorite colour: +favorite_colour …is harder to read than: …is harder to read than: name: %s, age: %s, city: %s, province: %s, IP: %s, favorite colour: %s % (name, age, city, province, ip, favorite_colour) name: %s, age: %s, city: %s, province: %s, IP: %s, favorite colour: %s % (name, age, city, province, ip, favorite_colour) …which is harder to read than: …which is harder to read than: name: %(name)s, age: %(age)s, city: %(city)s, province: %(province)s, IP: %(ip)s, favorite colour: %(favorite_colour)s % locals() name: %(name)s, age: %(age)s, city: %(city)s, province: %(province)s, IP: %(ip)s, favorite colour: %(favorite_colour)s % locals() Of course, Ruby has the prettiest string interpolation: Of course, Ruby has the prettiest string interpolation: name: #{name}, age: #{age}, city: #{city}, etc.. name: #{name}, age: #{age}, city: #{city}, etc..

Files Files are great in python Files are great in python f = open(filename.txt) f = open(filename.txt)

Making lists pretty Even though Python usually cares about indentation, lists dont… so get creative! Even though Python usually cares about indentation, lists dont… so get creative! Note: You can have an extra comma on the last item to make cutting and pasting and shifting lines around easier!

Making dicts pretty Dicts let you use that extra comma too…

Making function calls pretty Sometimes you need a huge number of function parameters. Theres a couple ways to make this readable: Sometimes you need a huge number of function parameters. Theres a couple ways to make this readable: Theres that extra comma again!

Functions Standard function Standard function You can set the default value for a parameter (in this case, None). You can set the default value for a parameter (in this case, None). These are called keyword arguments These are called keyword arguments

if statements You can do some neat tricks in if statements. You can do some neat tricks in if statements. Compare identity with the is and is not operators: Compare identity with the is and is not operators: if theSocket is not None: if theSocket is not None: if hand.contents() is None: if hand.contents() is None: Checking if stuff is empty: Checking if stuff is empty: if []:=> false if []:=> false if :=> false if :=> false if None: => false if None: => false if 0:=> false if 0:=> false

if statements Theres also the handy elif construct: Theres also the handy elif construct: (Thats right, Python has no switch statement!)

Functions Inline functions are kinda gross and very limited (they can only execute one statement, and its returned by default), but theyre sometimes handy. Inline functions are kinda gross and very limited (they can only execute one statement, and its returned by default), but theyre sometimes handy. Its usually better to just def a function. Its usually better to just def a function. (List comprehensions are better for this, by the way…)

Capturing Function Parameters You can capture all the parameters passed to a function in a list by naming one parameter *args: You can capture all the parameters passed to a function in a list by naming one parameter *args:

Capturing Function Parameters You can also get a dictionary of all the keyword arguments You can also get a dictionary of all the keyword arguments

Using the Apply method Apply lets you call a function and pass the parameters in as a list. For example, using the cmp (compare) function (which returns -1 for less than, 0 for equal, 1 for greater than): Apply lets you call a function and pass the parameters in as a list. For example, using the cmp (compare) function (which returns -1 for less than, 0 for equal, 1 for greater than):

But you dont actually need apply… You can actually do the same thing by putting the parameters in a list and dereferencing it with *list: You can actually do the same thing by putting the parameters in a list and dereferencing it with *list:

And it works for dicts too! Dictionaries can be dereferenced by doing **dict: Dictionaries can be dereferenced by doing **dict:

Regexes Regexes can be executed on a string directly: Regexes can be executed on a string directly: match = re.match(expression, string) match = re.match(expression, string) OR, they can be compiled first for extra speed: OR, they can be compiled first for extra speed: compiled_regex = re.compile(expression) compiled_regex = re.compile(expression) match = compiled_regex.match(string) match = compiled_regex.match(string) When you do a.match() or a.search(): When you do a.match() or a.search(): No match returns None No match returns None A match returns a re match object, which contains: A match returns a re match object, which contains: groups() groups() group([group number]) group([group number]) groupdict() (if you used named groups: (? expression)) groupdict() (if you used named groups: (? expression))

Decorators Decorators let you add wrapper code around methods and classes Decorators let you add wrapper code around methods and classes The Old Way The New Way

Require Integer Decorator

Pre/Postconditions Decorator

Automatic Thread Locking Decorator

Static Methods / Class Methods Oh yeah, I forgot to mention… Oh yeah, I forgot to mention… Static methods are methods that you can call on an instance or the class itself Static methods are methods that you can call on an instance or the class itself They dont take self as a parameter, so they cant operate on the object. Theyre just helper methods. They dont take self as a parameter, so they cant operate on the object. Theyre just helper methods. Class methods are called on classes only and receive the class as the first argument (instead of self) Class methods are called on classes only and receive the class as the first argument (instead of self) Useful when you have a big class hierarchy and you want the parent class to do things to the child classes. Useful when you have a big class hierarchy and you want the parent class to do things to the child classes.

Docstrings, PyDoc, iPython Docstrings rule. Docstrings rule. Its why everything in ipython has help for it Its why everything in ipython has help for it Whoever invented them was a genius, because theyre elegant and powerful: Whoever invented them was a genius, because theyre elegant and powerful: (Its common to use triple-quotes because they let you write multi-line docstrings, and they kinda look nicer, but regular strings work too.)

List Comprehensions List comprehensions make it simpler to do a really common task: computing something for every element in a list and putting the result in another list. List comprehensions make it simpler to do a really common task: computing something for every element in a list and putting the result in another list. Check out that SAVINGS! Check out that SAVINGS!

List Comprehensions They can also work as filters. They can also work as filters. Heres how you filter a list of numbers and return only the even numbers: Heres how you filter a list of numbers and return only the even numbers:

Generators Generators are special objects that spit out values one at a time as requested by the caller. Generators are special objects that spit out values one at a time as requested by the caller. They can result in huge memory savings when processing large batches of data They can result in huge memory savings when processing large batches of data Theyre also great for implementing things like pipes, or procedurally generated sequences Theyre also great for implementing things like pipes, or procedurally generated sequences

Generators To make your function a generator, just use yield instead of return To make your function a generator, just use yield instead of return Calling a generator returns a generator object Calling a generator returns a generator object Generator objects have a.next() method that returns the next value in the sequence Generator objects have a.next() method that returns the next value in the sequence Generators maintain their state between calls to.next() Generators maintain their state between calls to.next()

Generators Example: Example:

Generators Another example: Another example:

Generator Expressions Identical to list comprehensions, except that the result is a generator instead of a list. Identical to list comprehensions, except that the result is a generator instead of a list. for square in ( num**2 for num in numbers ): for square in ( num**2 for num in numbers ): print you got yourself a square! =>, square print you got yourself a square! =>, square

More tips…

Making programs flexible It makes it much easier to write code when you can change it easily It makes it much easier to write code when you can change it easily I constantly modify my programs as Im going, and as such, Ive developed some good tricks to make that easier. I constantly modify my programs as Im going, and as such, Ive developed some good tricks to make that easier. Writing your program as if its an API that you can call from the interpreter is an easy way of testing it Writing your program as if its an API that you can call from the interpreter is an easy way of testing it see altavista.py see altavista.py