CISC101 Reminders Assn 3 sample solution is posted. Winter 2019 CISC101 4/9/2019 CISC101 Reminders Assn 3 sample solution is posted. Assignment 4 due next Friday. Use lists, sets, dictionaries and design and write functions to solve some data mining exercises. If anyone would volunteer to administer the USATS – please contact Scott Reed at coursefeedback@compsa.queensu.ca Winter 2019 CISC101 - Prof. McLeod Prof. Alan McLeod

Today os, os.path and sys Modules. Reading remote files by using a URL. Start Dictionaries. Winter 2019 CISC101 - Prof. McLeod

os Module This module is very helpful if you have to do quite a bit of work with files and folders. Basically the module provides an interface to your OS. Winter 2019 CISC101 - Prof. McLeod

CISC101 os Module Behind the scenes, this module loads a module for your particular operating system. So regardless of your OS, you just use import os… See section 16.1 in the Python Library Reference. Lots of goodies, particularly file system utilities. Here’s a selection of file-related functions: Winter 2019 CISC101 - Prof. McLeod Prof. Alan McLeod

os Module, Cont. remove() – deletes a file rename() – renames a file walk() – generates filenames in a directory tree (a generator object) chdir() – change working directory chroot() – change root directory of current process listdir() – list files and folders in directory getcwd() – gets current working directory mkdir() – creates directory Winter 2019 CISC101 - Prof. McLeod

os Module, Cont. rmdir() – removes directory access() – verify permission modes (2 parameters) chmod() – change permission modes In os.path: basename() – returns name of file dirname() – returns name of directory join() – joins directory and filename split() – returns both as tuple splitext() – returns filename and extension as tuple Winter 2019 CISC101 - Prof. McLeod

os Module, Cont. In os.path, Cont: getatime() – returns last file access time getctime() – returns file creation time getmtime() – returns file modification time getsize() – returns file size in bytes exists() – does file or directory name exist? isdir() – is this a directory name and does it exist? isfile() – is this a file and does it exist? Winter 2019 CISC101 - Prof. McLeod

os Module, Cont. Demo program – see LargeFileSearch.py Winter 2019 CISC101 - Prof. McLeod

os Module, Cont. Python already has an exec() BIF that allows you to run external python code from within your program. The os module also has many commands that allow you to run other non-python programs from within your program. For example, os.system() allows you to run a system command (a DOS command, for example). Winter 2019 CISC101 - Prof. McLeod

sys Module See section 28.1 in the Python Library Reference. CISC101 sys Module See section 28.1 in the Python Library Reference. More system functions and attributes. Here is a small sampling: argv – is a list of all command line parameters sent to the interpreter. builtin_module_names – a list of all built-in modules. exit(0) – immediately exits a python program giving a zero exit status code. getwindowsversion() – returns a tuple consisting of major, minor, build, platform, and service pack status. Winter 2019 CISC101 - Prof. McLeod Prof. Alan McLeod

sys Module, Cont. path – a list of the module search paths used by python. platform – the current OS platform. prefix – the folder where python is located. version – the version of python being used. See sysModuleDemo.py Winter 2019 CISC101 - Prof. McLeod

sys Module, Cont. Not so much about functionality. Mostly about getting information about how Python is working, along with more information about your OS. Winter 2019 CISC101 - Prof. McLeod

CISC101 The exec() BIF You can use the os module to execute code in many other languages from a python program. But there is already an easy-to-use BIF, exec(), that can execute python code if it is supplied to the BIF as a string. The string could come from a file, for example. Winter 2019 CISC101 - Prof. McLeod Prof. Alan McLeod

The exec() BIF, Cont. See RemoteFileExecution.py which uses the Message.py file. You must supply the output of the BIF globals() to the exec() BIF – this supplies a dictionary of all the global variables currently in use and their values. Winter 2019 CISC101 - Prof. McLeod

Files from the Web A file is a file, no matter where it comes from! CISC101 Files from the Web A file is a file, no matter where it comes from! You can read a file from anywhere in the web as long as you have the URL (or web address) for that file. Winter 2019 CISC101 - Prof. McLeod Prof. Alan McLeod

fh = urllib.request.urlopen(url_string) Files from the Web Use the module urllib.request The function fh = urllib.request.urlopen(url_string) accepts a URL as a string and returns a handle to the file. (What does “URL” stand for?) The entire file can be read using (guess what!) the read() method: contents = fh.read() You can supply a number to the read() method to limit the number of bytes read. Winter 2019 CISC101 - Prof. McLeod

contentsString = contents.decode() Files from the Web These are byte strings, which are better to decode using the decode() method. For example: contentsString = contents.decode() The decode method attempts to use the proper encoding, which is commonly “utf-8”, but there are others… (See: http://www.w3.org/International/O-charset for more information on encoding.) Winter 2019 CISC101 - Prof. McLeod

Files from the Web See the demo program URLFileDemo.py Just hints at what you can do! Winter 2019 CISC101 - Prof. McLeod

Dictionaries Or a “dict”. Another built-in collection type in Python. Mutable collection (like lists). A “mapped” data structure – not index based. Uses a set of key : value associations. You cannot use the slice operator with a dictionary, instead you use the key name. The key name goes in a set of [ ], but this is not the slice operator in the context of dictionaries. Winter 2019 CISC101 - Prof. McLeod

Dictionaries, Cont. Create an empty dictionary using: emptyDict = dict() or: emptyDict = { } Add or modify a key : value pair simply as: emptyDict['name'] = "Alan" emptyDict is now {'name': 'Alan'} Each key must be unique and hashable – strings are most common. Winter 2019 CISC101 - Prof. McLeod

Dictionaries, Cont. Add another key : value pair: emptyDict['age'] = 21 emptyDict is now {'name': 'Alan', 'age': 21} key : value pairs are separated by commas. Like sets, you cannot control the order of the key value pairs in the dict – the hashing algorithm controls this. However, since you cannot use index values, the position has no meaning. Winter 2019 CISC101 - Prof. McLeod

A List of Dictionaries – An Example CISC101 A List of Dictionaries – An Example >>> name1 = {'name':'Sam', 'age':18, 'SN':4445555} >>> name2 = {'name':'Boris', 'age':21, 'SN':5554444} >>> name3 = {'name':'Ben', 'age':19, 'SN':5445444} >>> allNames = [name1, name2, name3] >>> allNames [{'age': 18, 'name': 'Sam', 'SN': 4445555}, {'age': 21, 'name': 'Boris', 'SN': 5554444}, {'age': 19, 'name': 'Ben', 'SN': 5445444}] >>> allNames[2]['age'] 19 >>> allNames[1]['name'] 'Boris' Winter 2019 CISC101 - Prof. McLeod Prof. Alan McLeod

Better Yet: >>> allNames = {} >>> allNames['Sam'] = {'age':18, 'SN':4445555} >>> allNames['Boris'] = {'age':21, 'SN':5554444} >>> allNames['Ben'] = {'age':19, 'SN':5445444} >>> allNames {'Boris': {'age': 21, 'SN': 5554444}, 'Ben': {'age': 19, 'SN': 5445444}, 'Sam': {'age': 18, 'SN': 4445555}} >>> allNames['Boris']['age'] 21 Winter 2019 CISC101 - Prof. McLeod

Dictionaries, Cont. A dictionary has a method called keys() that returns an iterable list of key names: >>> name1.keys() dict_keys(['age', 'name', 'SN']) If you apply the sorted() BIF to a dictionary it returns an iterable sorted by key name to use with a for loop. See section 5.5 in the Python Tutorial. Winter 2019 CISC101 - Prof. McLeod

For Loop Example: Winter 2019 CISC101 - Prof. McLeod

Dictionary Methods See section 4.10 in the Python Standard Library: clear() – empties the dictionary fromkeys(keys, [values]) – returns a new dict get(key) – Obtains a key’s value, but returns None if the key is not in the dictionary. items() – returns a iterable of key, value tuples. keys() – returns an iterable of just keys pop(key) – remove key : value and return value. Winter 2019 CISC101 - Prof. McLeod

Dictionary Creation, Demo See DictionaryCreation.py for demos of the dict() BIF and the fromkeys() method. Winter 2019 CISC101 - Prof. McLeod