1. Python Mini-Course University of Oklahoma Department of Psychology Lesson 17 Reading and Writing Files 5/10/09 Python Mini-Course: Lesson 17 1

2. Lesson objectives 1. Open files for reading, writing, or appending data 2. Write data to a text file 3. Use the os module to manipulate paths and pathnames 4. Use the pickle module to store complex data types 5/10/09 Python Mini-Course: Lesson 17 2

3. Files in Python Files are objects http://www.python.org/doc/2.5.2/lib /bltin-file-objects.html http://www.python.org/doc/2.5.2/lib /bltin-file-objects.html Python file methods are wrappers for the standard C stdio package 5/10/09 Python Mini-Course: Lesson 17 3

4. File types Text file Contains ACSII or Unicode characters Can be created and read by most applications Text editors (Notepad, SimpleText, etc.) IDEs (IDLE, SPE, Eclipse, etc.) Word processors (MS Word, etc.) Spreadsheet programs (Excel, etc.) Other apps (SAS, SPSS, R, Mathmatica, etc.) 5/10/09 Python Mini-Course: Lesson 17 4

5. File types Binary file Contain data coded in other formats Examples: JPEG images Audio or video clips Packed binary data from FORTRAN Matlab data files (.m files) 5/10/09 Python Mini-Course: Lesson 17 5

6. The open statement Returns a file object for access with file methods Syntax fid = open(filename, mode) where fid is the name of the file object 5/10/09 Python Mini-Course: Lesson 17 6

7. The filename argument Should be a string containing the complete name of the file, including the file extension NB: In MS Windows, most file extensions are hidden in Windows Explorer Can include a partial or complete path Default path is the folder containing the main script (.py file) 5/10/09 Python Mini-Course: Lesson 17 7

8. File modes: reading a file 'r' read (text file) 'rb' read (binary file) Can read file contents but cannot change file If file does not exist, raises exception 5/10/09 Python Mini-Course: Lesson 17 8

9. File modes: writing 'w' write (text file) 'wb' write (binary file) Create a new file Overwrites existing file if there is one 5/10/09 Python Mini-Course: Lesson 17 9

10. File modes: append 'a' append (text file) 'ab' append (binary file) Append data to (the end of) a file If file does not exist, creates a new file 5/10/09 Python Mini-Course: Lesson 17 10

11. File modes: mixed modes 'r+' read and write existing file If file does not exist, raises exception 'a+' read and write existing file Creates new file if one does not exist 'w+' read and write a new file Overwrites file if it already exists 5/10/09 Python Mini-Course: Lesson 17 11

12. Note Data transferred between files and your programs is represented as Python strings, even if it is binary data. String objects can contain character bytes of any value 5/10/09 Python Mini-Course: Lesson 17 12

13. End-of-line translations Unix and Linux (and Mac OS X) Use newline: \n DOS and Windows Use return + newline: \r\n Old Mac OSs Use return: \r 5/10/09 Python Mini-Course: Lesson 17 13

14. End-of-line translations Python automatically translates Windows EOLs when reading and writing files on Windows platforms When in text mode Not in binary mode 5/10/09 Python Mini-Course: Lesson 17 14

15. Example: eol.py, win.txt, mac.txt text_mode = [open('win.txt','r').read(), open('mac.txt','r').read()] print 'Text mode:' print text_mode binary_mode = [open('win.txt','rb').read(), open('mac.txt','rb').read()] print '\nBinary mode:' print binary_mode 5/10/09 Python Mini-Course: Lesson 17 15

16. File read methods file.read() Read all data until EOF is reached and return as a string object file.readline() Read one entire line from the file (keeps the trailing newline character) and return as a string object file.readlines() Read until EOF using readline() and return a list containing the lines thus read 5/10/09 Python Mini-Course: Lesson 17 16

17. Example: read.py fin = open('win.txt', 'r') print fin.read() fin.seek(0) print fin.readline() fin.seek(0) print fin.readlines() fin.close() 5/10/09 Python Mini-Course: Lesson 17 17

18. File write methods file.write(str) Write a string to the file NB: Due to buffering, the string may not actually show up in the file until the flush() or close() method is called file.writelines(sequence) Write a sequence of strings to the file NB: Does not add line separators, but this can be done using the string join operator 5/10/09 Python Mini-Course: Lesson 17 18

19. Example: randnums.py import random fout = open('rand.txt', 'w') fout.write('Number\n') seq = [] for i in range(10): s = '%2.4f' % (random.random()) seq.append(s) fout.writelines('\n'.join(seq)) fout.close() 5/10/09 Python Mini-Course: Lesson 17 19

20. Example: randnums2.py import random fout = open('rand.txt', 'w') fout.write('Index\tNumber\n') seq = [] for i in range(10): s = '%d\t%2.4f' % (i, random.random()) seq.append(s) fout.write('\n'.join(seq)) fout.close() 5/10/09 Python Mini-Course: Lesson 17 20

21. The os module Provides generic operating system (OS) support and a standard, platform-independent OS interface Includes tools for environments, processes, files, shell commands, and much more http://www.python.org/doc/2.5.4/lib/ module-os.html http://www.python.org/doc/2.5.4/lib/ module-os.html 5/10/09 Python Mini-Course: Lesson 17 21

22. File and directory commands os.getcwd() Returns the name of the current wording directory as a string os.chdir(path) Changes the current working directory for this process to path, a directory name string 5/10/09 Python Mini-Course: Lesson 17 22

23. File and directory commands os.listdir(path) Returns a list of names of all the entries in the directory path 5/10/09 Python Mini-Course: Lesson 17 23

24. Portability constants os.curdir() String for the current directory os.pardir() String for the parent directory os.sep() String used to separate directories os.linesep() String used to terminate lines 5/10/09 Python Mini-Course: Lesson 17 24

25. The pickle module Used to serialize and de-serialize a Python object structure http://www.python.org/doc/2.5.4 /lib/module-pickle.html 5/10/09 Python Mini-Course: Lesson 17 25

26. The pickle module Pickling the process whereby a Python object hierarchy is converted into a byte stream Unpickling the inverse operation, whereby a byte stream is converted back into an object hierarchy 5/10/09 Python Mini-Course: Lesson 17 26

27. The pickle module pickle.dump(obj, file) Write a pickled representation of obj to the open file object file pickle.load(file) Read a string from the open file object file and interpret it as a pickle data stream, reconstructing and returning the original object hierarchy 5/10/09 Python Mini-Course: Lesson 17 27

28. Example: pickling.py import random, pickle seq = [] for i in range(10): s = '%d\t%2.4f' % (i, random.random()) seq.append(s) print seq f = open('temp.pk', 'w') pickle.dump(seq, f) f.close() 5/10/09 Python Mini-Course: Lesson 17 28

29. Example: pickling.py seq = [] print seq f = open('temp.pk', 'r') seq = pickle.load(f) print seq 5/10/09 Python Mini-Course: Lesson 17 29