1. Python – May 11 Briefing Course overview Introduction to the language Lab

2. Python Why computer programming? –Computer is a general-purpose machine, born to be programmed –We want to solve many types of problems Our goals –Become multi-lingual –Practice with design & implementation –Learn by doing: start simple Help available –Textbook, python.org, tutorials

3. Background First released, 1991 –Guido van Rossum, CWI, Netherlands –Dissatisfaction with existing languages High-level, multi-purpose language –Relatively easy to learn & use (we hope!) –Procedural, object oriented or functional –Interpreted –Interface to OS, file system, Web –Some data structures built into language –Standard library –(More details in sections 1.3 and 1.5)

4. Initial To-Do Essential skills; special features of language Routine stuff –Source file name ends in.py –How to run interpreter (command line or IDE) –Program structure I/O –Interactive, file; formatting Variables, identifiers Doing math Comments, tokens, whitespace, style conventions

5. continued Control structures –Asking questions, repeating stuff –nesting –How to make comparisons –Boolean operations: and/or/not Functions/procedures: parameters, returns Data structures –String, dynamically-sized list, etc. Standard (i.e. run-time) library –Random numbers, etc.

6. In the end… How can you tell if you’ve mastered a language? –Write a game –Given a spec or algorithm, can you implement it? –You may prefer to do something in Python! –You never need to know everything. Know where to find stuff quickly This course lies between first 2 prog. courses –You’ll see more operations & applications than in the 1 st course. Not as theoretical as 2 nd course.

7. Beginning Versions 2.x and 3.x Lab: let’s use IDE such as ActivePython –Free, simple, includes debugger Code can by entered: –directly in shell (good for experimenting!) –in a file, which we can save and reuse later

8. Initial nuts & bolts Comments begin with #, go to end of line Statements don’t end in semicolon Don’t arbitrarily indent code Put \ at end of line to continue statement Variables are not declared Interactive output –Version 2.6: print statement print name –Version 3.0: print function print (name)

9. continued Examples print “Hello, world” print 7+4*3 Interactive input –input( ) for getting a number –raw_input( ) for getting a string name = raw_input(“What is your name”)

10. What’s next? Lab Please read chapter 2 Practice with –Basic I/O –Mathematical operations –Strings –Simple if statement –Simple loop

11. Python – May 12 Recap lab Chapter 2 –operators –Strings –Lists –Control structures

12. Review Some things we saw in lab –String slice is like Java substring, e.g. s[2:7] –Range is a function that returns a list, and we can iterate over this list: for j in range(5, 10): … –Formatting real numbers print “Answer is {0:4.1f} feet.”.format(ans) –Syntax error: cursor takes you location –Initialize your variables: Interpreter remembers value from previous run! (Even if you earlier ran a different program)

13. Chapter 2 ** and // operators We have += but not ++ Boolean operators are words List : dynamic array. –Commonly used functions are: append, insert, remove –Can use [ ] to access/change single element. –See handout

14. Strings Some operations –Length: use len(stringName)  integer –indexOf: use stringName.find(pattern)  integer –charAt: just use brackets –Substring: just use brackets Note that you can leave one of the arguments empty: s[4:] means to go to the end –String compare: can use ==, > and <.

15. File I/O File output outFile = open(“output.txt”, “w”) outFile.write(“help me\n”) outFile.close() File input (see CountLines.py) inFile = open(“input.txt”, “r”) for line in inFile:... inFile.close() # Good idea to close file, as interpreter may # remember where you left off next time!

16. Python – May 13 Recap lab More about loops / repetition Functions!

17. Review Examples –Useful list functions: index, count –elif –Range with negative stride: range(6,0,-2) = [6,4,2] –Printing without automatic newline for i in range(1,6): print i*5, –Yes, you can insert anywhere in a list: L = [1,2,3] L.insert(1,7)  L becomes [1,7,2,3]

18. cont’d Can convert type: int(), float(), str() –type() tells you what you currently have Typical response: val = input(“Enter a number”) if str(type(val)) == “ ”: # handle float case else: # handle int case

19. List comprehension Special case of a “loop” inside an expression. Often we want to do something to every item in a list. L = [ 1, 2, 3, 4 ] M = [-x for x in L] # brackets required Combining 2 lists! A = [ 1, 2, 3] B = [ 7, 8, 9] [x+y for x in A for y in B] Produces this: [8, 9, 10, 9, 10, 11, 10, 11, 12] A = [ "parl", "jou", "pens" ] B = [ "e", "es", "ons" "ez", "ent" ] The same formula now gives 15 verb conjugations

20. More on loops while True is allowed break & continue statements Important skill: error checking of input needInput = True while needInput: value = input("Please enter positive number") if value > 0: needInput = False else: print("Sorry, bad input")

21. Functions Declare early –A function must be declared before it is called. The interpreter only reads your program once. Syntax def functionName(parameters): code optional return statement Example: fun.py

22. Python – May 16 Recap lab Simple string tokenizing Random numbers Tomorrow: –multidimensional array (list of list) –Exceptions

23. Review To see both code & I/O: Window  Tile If your program has functions, how does the interpreter know where your program starts? When you write a loop with range: for i in range (1, 10) –i retains its last value after the loop (9) Look up something? Documentation: Python Standard Library, see “index” Link at top right of page.

24. Tokenizing Important skill in reading text input –Input file may be in specific format Two techniques –String’s split( ) function –Python’s regular expression library  more powerful

25. String split( ) Simple to use Good when there is a certain pattern of characters you want as a delimiter s = “moo--goo---gai--pan“ s.split(“-”) [‘moo’,’’,’goo’,’’,’’,’gai’,’’,’pan’] s.split(“--“) [‘moo’,’goo’,’-gai’,’pan’] s.split(“---”) [‘moo--goo‘,’gai--pan’]

26. Using re More often, we want to handle multiple possible delimiters. –Like Java’s StringTokenizer or C’s strtok( ) –Extracting words by ignoring any and all punctuation See handout: tokenize.py –tok = re.compile( )  takes your set of delimiters and creates a “regular expression object” i.e. tokenizer Don’t forget the brackets around delimiters. –tok.split(my_string)  gives list of tokens

27. Random #s Python has a built-in library (module) called random At top of program, say: import random Two most important functions –random.uniform(a, b) returns a random real number between a and b inclusive –random.randint(a, b) does the same for int

28. Python – May 17 Recap lab Multidimensional list Exceptions Commitment: quiz tomorrow

29. Lab notes Put delimiters inside (“[ ]”) When you sort a list, there is no return value L.sort()# rather than L = L.sort() In lottery problem, how did you make sure all winning numbers distinct? –Technique also helpful when “dealing” cards. Can check syntax before running: next to run button there is a “check”. Status bar shows result, and cursor taken to error, if any

30. Multi-D Intuitive, just like other languages. List inside a list [ [ 1, 2 ], [ 8, 6 ] ] Access element with multiple indices: list[3][2] Very often associated with nested loops Applications? –List of lines; list of words on the line –Board game

31. Examples Sudoku and Reversi 175294836 6231745 89563271 51973246 3485129 86941753 9342567 41379582 7521839

32. Exceptions Chapter 10 covers in great detail Similar to Java Find out what could go wrong with your code try-except block try : possibly exceptional code except Name_of_exception: what to do in case of problem

33. Exceptions Common types –ValueError –IOError –IndexError You can raise an exception if you discover something wrong: if value < 0: raise ValueError Another use of “raise”: raise SystemExit is a way out of your program.

34. Example while True: try: value = input("Enter a positive number") if value <= 0: raise ValueError else: break except ValueError: print "Please follow directions" continue print value

35. Python – May 18 Quiz Sorting Relatives of the list: –Tuple –Dictionary –Set –We’ll do more with dictionaries later

36. Lab notes Interactive run – hard to tell when program’s output begins and ends – what can you do? Review Battleship –How would you change to handle real game?

37. Sorting Often we want to sort data in a list, but our list does not contain atoms like single numbers. Need to tell Python how to compare elements Analogous to comparators in Java. Steps: –Create function, taking 2 arbitrary elements from your list. Return positive / negative / zero. –Call: list.sort(comparatorFunction) –See example handout (compare.py)

38. Tuple Similar to list: use ( ) instead of [ ] Good for identifying a point in some space, like an ordered pair, triple Often an anonymous object Immutable – not meant to be updated, just throw away –We just have count & index functions Syntax is straightforward: multiple assignment –Can re-use variables later (a, b, c) = (10, 8, 2)

39. Dictionary A nice array Index can be anything that Python can easily evaluate, such as a single int, float or string. Typical procedure: –Initialize as { } –Add element by assignment, e.g. d[“USA”] = 308 –Can traverse dictionary elegantly for i in d: print i, d[ i ]

40. Notes In a dictionary, like a set, the order of the data is irrelevant. The “key” is already an index. Example: { “Finland”: 5, “India”: 1150, “USA”, 308, “France” : 61 } Don’t rely on Finland being at the “beginning” of the dictionary. The value 5 is obtained by d[“Finland”], not d[0] ! Python can quickly find your data in a dictionary

41. Illustration d = { } d[“Tina”] = 3 d[“Kevin”] = 2 “Kevin” in d  returns True “James” in d  returns False d [“James”] gives a KeyError, so when in doubt, check to see that key actually exists!

42. Applications Good for sparse array: only store the values you actually use. Here, the value at each key can be a list: f [1920] = [90, 230] f [1920][0] = 90 192019511960198620002010 9015065066561876837728 230700750317651449572

43. Applications (2) Excellent for maintaining data –E.g. Stock portfolio: reading a list of buy/sell transactions. At any point in time we may want to know total holdings –Can remove an element from a dictionary using del: del portfolio[“IBM”] del enemy[“Soviet Union”]

44. Set Essentially, a set is a list in which the elements do not repeat. (See chapter 7) –E.g. Useful when you need a lot of boolean values Can convert a list to a set by using set( ). s = set([1, 3, 5, 7, 9]) If you call set( ) on a string, you get a set of its characters! Operations in not in & | - ^ Note that you can’t use ~

45. Python – May 19 Review –What is the difference between: list, tuple, set, dictionary? –When is it appropriate to use each? Creating our own data types: classes/objects Reminder: “meeting” program due tomorrow

46. Paradigms Note that there are 3 ways to approach programming Procedural – follow definition of program: list of operations to perform: verb focus Object oriented – noun focus: define a type with its attributes and operations Functional – everything is a function call; loops written as recursion

47. OO A different way to approach problem solving – think about the nouns first. Python supports OO design. –We can define attributes & operations that belong to instances of the class. Everything is public – no information hiding.

48. Example class Rectangle: length = 4# Static & default values width = 3 def area(self): return self.length * self.width # ------------------------------------------ r = Rectangle() print r.length print r.area()

49. Example (2) More realistic to allow instances to be different! To initialize attributes, use special function called __init__ class Triangle: def __init__(self, side1, side2, side3): self.a = side1 self.b = side2 self.c = side3 def perimeter(self): return self.a + self.b + self.c t = Triangle(3,4,5) print t.perimeter()

50. Notes Python not the best language for true OO. Unfortunately, can’t have more than 1 “constructor” When you call an instance method, you literally pass it with 1 fewer parameter than in the declaration – leave off the “self”. definition: def giveRaise(self, percent): … call: bob.giveRaise(4)

51. Careful! class Rectangle: length = 4 width = 3 def __init__(self, L, W): self.length = L self.width = W # --------------------------------- r1 = Rectangle(10, 8) print Rectangle.length# equals 4 print r1.length# equals 10 Moral – if you have some static values you want to share, don’t confuse yourself by using same name as attribute. Now you see why we always use “self”.

52. Python – May 20 Reading data from the Web –Example: web.py Practice with large amount of I/O –See lab sheet

53. Web input Just like Java, Python has an easy way to automatically read a Web page import urllib2 page = urllib2.urlopen(“http://--------”) Can continue as if “page” is an input file for line in page: Example: web.py reads a stock quote.

54. Large input Algorithm needs to be practical, in addition to correct. Beware of nested loops Comments about the data; “density” #3 - County population, area, density –Assume that all blocks in county are listed together. #5 – Creating tract-level data –Need to look up tract in dictionary search for tract, because blocks in input file are not sorted by tract –Once you are finished with a county, you can output all its tracts and empty the dictionary.

55. Python – May 25 Quiz Python’s imaging library Lab –Creating images –Pretty parenthesizer problem Questions about homework?

56. PIL Python imaging library –First, download if necessary –Defines “Image” and many useful functions –We’ll create images! Practical note: –Need to add PIL to Python’s search path –“import sys” and print sys.path to see what is ordinarily included –Since sys.path is a list, you can simply append the PIL folder to it…. if necessary

57. 3 steps First, create image object image = Image.new(“RGB”, (100,100)) 2 parameters are mode and dimensions Next, paint your pixels usually in a nested loop of x & y values image.putpixel((x,y),(red,green,blue)) Finally, write image to file image.save(“myimage.jpg”) Does not have to be jpg. See handout

58. Reference http://www.pythonware.com/library/pil/handbook These sections are especially helpful –Concepts –Image: Easy to use –ImageDraw A “draw” object can create geometric shapes like circles, rectangles and polygons. Can also be used to read or modify existing image image.getpixel((x, y))  returns RGB tuple

59. Python – May 26 Persistent objects Experimenting with time

60. Persistence You may find it helpful to write a large object to a binary file. import cPickle file = open(“outputfile”, “wb”) cPickle.dump(aLotOfData, file) file.close() file2 = open(“outputfile”, “rb”) newlist = cPickle.load(file2) file2.close()

61. Time module import time time.localtime() returns a list containing: –Year, month, day, hour, min, sec, day#, Julian day time.clock() –Returns 0 the first time you call it –Otherwise returns a float # of seconds since first call. –Good to subtract 2 consecutive calls to time a segment of code. time.sleep(# of seconds)

62. Timing quick event How long does it take to add an element to a list? t1 = time.clock() # loop to add 100 elements t2 = time.clock() print (t2-t1)/100 What’s wrong with this approach?

63. Python – May 27 How to use modules in Python Examples –Printing large digits one at a time –Defining a class in another file –Careful if module has a function with same name Lab Last homework: map of population change

64. Modules Typical design: put functions in a separate file, freeing up “main” program to do the important action. Not required, but convenient if same functions used in multiple programs! At top of source file: import module_name # required reload (module_name)# convenient!

65. Examples Suppose other file is called bank.py import bank x = bank.computeInterest(75, 3, 2) # Saying “bank.” is not required unless we have a # name conflict. If you want to use a class defined in another source file, you have to name both the module and the class import shapes r1 = shapes.rectangle(10, 8) print r1.area()

66. Python – May 31 Multi-threaded applications Outline for final Lab & surveys

67. Thread intro Also called “lightweight process” One process may have multiple threads of execution Allows a process to do 2+ things concurrently –Games –Simulations Even better: if you have 2+ CPU’s, you can execute in parallel Multicore architecture  demand for multithreaded applications for speedup More efficient than using several concurrent processes

68. Thread steps Import the “threading” module Create a class –Extends the existing threading.Thread class –Constructor: initialize each thread with some unique information –Implement run( ) function: this is the work you want the thread to do Main program: –Create threads 1 at a time, put them in list –“start” them: this invokes their run( ) functions –“join” them: i.e. wait for them to finish

69. Final Two parts Written part, up to 1 hour, 50% –12-15 questions Programming part, remaining 2 hours, 50% –5 questions –Once you start, you cannot go back to written part –Open book, open note –You may use ActivePython’s copious documentation –Do not use Internet

71. Images import Image Create √ –Initialize mode and dimensions of image –image.putpixel( ) –Save Read –image.getpixel((x, y))  returns RGB tuple

72. Python – May 19 Discuss lab Bitwise operators

73. Lab notes pass – the statement that does nothing copy.deepcopy() – make a true copy of an object, avoid aliasing error import copy list = [4,6] list2 = copy.deepcopy(list) list[0] = 9 print list2

74. Bitwise Python includes bitwise operations –Extremely fast… efficient in time, energy, space. –Applications: lots of boolean values, saving space if you know you have a value requiring few bits. –Two kinds of operators: logical and shift. Logical Operators &means “and” |means “or” ^means “exclusive or” ~means “not” – this is a unary operator

75. How they work Truth table (for each bit position) Most important facts: –Anything “and-ed” with 0 becomes 0 –Anything “or-ed” with 1 becomes 1 –Anything “xor-ed” with 1 becomes inverted XY& (and)| (or)^ (xor) 11110 10011 01011 00000

76. Example Integers usually hold 32 bits. For simplicity, let’s just show 8. x = 1010 1101 y = 1111 0000 -------------------- x & y = x | y = x ^ y =

77. Answer Integers usually hold 32 bits. For simplicity, let’s just show 8. x = 1010 1101 y = 1111 0000 -------------------- x & y = 1010 0000 x | y = 1111 1101 x ^ y = 0101 1101

78. Binary number Based on powers of two: This number equals 128+32+8+4+1 = 173 Interesting formula for ~ ~n = – (n + 1) …2561286432168421 010101101

79. Shorthand Occasionally we need to write a large number in binary. Better to do so as “hexadecimal” – a shorthand that compresses 4 bits in one symbol. Notation: begin with “0x” followed by, usually, 8 digits. Each digit is: 0-9, a-f a = ten = “1010”d = thirteen = “1101” b = eleven = “1011”e = fourteen = “1110” c = twelve = “1100”f = fifteen = “1111” Example: What does 0x7ffc0 look like in binary?

80. Shift operations Two directions –Shift left (<<) basically means to multiply by a power of 2. –Shift right (>>) essentially divides by a power of 2 and eliminates the remainder. For example 1 << 4 = 2**4 = 16 5 << 3 = 5 * 2**3 = 40 13 >> 1 = 13 / 2 = 6

81. Practical use Let’s suppose we want to use an integer to hold 31 boolean values, representing the days of a month. We’ll use bit positions 1-31 only. Start with may = 0, clearing all bits. To turn on a certain bit, we need to “or with 1” may = may | (1 << 19) To turn off a bit, we need to “and with 0” may = may & ~(1 << 19) Multiple bits?

82. Python – May 20 Recap lab: –Answers to question #3? Finish bitwise operators Set data type

83. Bitwise Note – Python integers can have > 32 bits Operators Masks –Used to determine exactly which bits we want to turn on, turn off or invert Applications –Bit vectors: use integer as “array” of booleans –Bit fields: pack more than one value

84. How to… Useful operations for a bit vector: Turn on a bit Turn off a bit Invert a bit Determine if a bit is 1 Determine if a bit is 0

85. Mask Often we want to modify more than 1 bit, so we need to create an operand that looks like: –Ones in middle: 0000000000111110000000 –Zeros in middle: 1111111000001111111111 How? Two ways –Can subtract powers of 2 –Use only bitwise operations

86. Examples ~0 … 11111111 11111111 ~0 << 5 … 11111111 11100000 ~(~0 << 5)… 00000000 00011111 ~(~0 << 5) << 4… 00000001 11110000 And you can invert one more time if needed. How could we write this as a difference of powers of 2 ?

87. Bit field Store more than one value in a variable, to save space. Example: date as month-day-year –How many bits do we need for each? date = 0 date |= month << 16 date |= day << 11 date |= year MonthDayYear 4511

88. Set The big problem with bitwise operators is that they are low level. Rely too much on data representation, which may distract from our algorithm. We can achieve similar functionality with the set data type Create, add/delete elements, operations (Ch. 7)

89. Python – May 21 Recap lab OOP

90. Review Sometimes we check a bit to see if it’s set. result = myValue & (1 << 21) if result != 0: … Don’t ask if result == 1. What is the algorithm to look for longest sequence of something? Default parameter Set (useful functions p.283)

91. Python – May 22 Recap lab Dictionary data type

92. Review How do you like OO in Python? Very long string literal: delimited by 3 single quotes and can extend for multiple lines without the \

93. Python – May 26 Tuple data type Image processing

94. Python – May 27 Homework questions? “Persistent” data Time

95. Python – May 28 Recap lab Modules Notes about final

96. Lab review How were you able to distinguish the images of dice? Speed of – –Sorting –Accessing elements of list vs. dictionary