Python – May 20 Reading data from the Web

Slides:



Advertisements
Similar presentations
Sorting Really Big Files Sorting Part 3. Using K Temporary Files Given  N records in file F  M records will fit into internal memory  Use K temp files,
Advertisements

CS 4432query processing - lecture 161 CS4432: Database Systems II Lecture #16 Join Processing Algorithms Professor Elke A. Rundensteiner.
Lecture 24: Query Execution Monday, November 20, 2000.
Query Execution :Nested-Loop Joins Rohit Deshmukh ID 120 CS-257 Rohit Deshmukh ID 120 CS-257.
Introduction to Database Systems 1 Join Algorithms Query Processing: Lecture 1.
CS146 Overview. Problem Solving by Computing Human Level  Virtual Machine   Actual Computer Virtual Machine Level L0.
External Sorting Problem: Sorting data sets too large to fit into main memory. –Assume data are stored on disk drive. To sort, portions of the data must.
6 Steps of the Programming Process
Selecting and Combining Tools F. Duveau 02/03/12 F. Duveau 02/03/12 Chapter 14.
General Programming Introduction to Computing Science and Programming I.
1 Functions 1 Parameter, 1 Return-Value 1. The problem 2. Recall the layout 3. Create the definition 4. "Flow" of data 5. Testing 6. Projects 1 and 2.
Population Density [Enter slide show description]
Searching and Sorting Chapter Sorting Arrays.
Python Basic. Download python Go to and download Python 2.7.8www.python.org.
Click on the“ Recharge ” tab How to Recharge a Customer.
Announcements Course evaluation Your opinion matters! Attendance grades Will be posted prior to the final Project 5 grades Will be posted prior to the.
Sequencing The most simple type of program uses sequencing, a set of instructions carried out one after another. Start End Display “Computer” Display “Science”
Peeking at Programming with JavaScript. So what’s JavaScript ? A programming language built into your Web Browser program. Adds fun and interactivity.
CS Class 04 Topics  Selection statement – IF  Expressions  More practice writing simple C++ programs Announcements  Read pages for next.
Files in Python Opening and Closing. Big Picture To use a file in a programming language – You have to open the file – Then you process the data in the.
1 Web Search What are easy ways to create a website? 2 Web Search What is a blog? What type of content does this type of website provide? 3 Web.
Introduction to Computing Science and Programming I
Algorithmic complexity: Speed of algorithms
CS 440 Database Management Systems
Database Management System
Introduction Python is an interpreted, object-oriented and high-level programming language, which is different from a compiled one like C/C++/Java. Its.
Lesson 1 An Introduction
Database Applications (15-415) DBMS Internals- Part VII Lecture 16, October 25, 2016 Mohammad Hammoud.
Python’s input and output
David Kauchak CS52 – Spring 2015
Algorithm Analysis CSE 2011 Winter September 2018.
Chapter 12: Query Processing
Teach A level Computing: Algorithms and Data Structures
Algorithmic complexity: Speed of algorithms
Sorting CSCE 121 J. Michael Moore
Python I/O.
JAVA IO.
Analysis of Bubble Sort and Loop Invariant
Advanced Topics in Data Management
Lecture#12: External Sorting (R&G, Ch13)
Databases Lesson 2.
For -G7 programing language Teacher / Shamsa Hassan Alhassouni.
Algorithm Efficiency, Big O Notation, and Javadoc
CS212 Data Structures 2018 Second Semester.
Finding a Eulerian Cycle in a Directed Graph
Coding Concepts (Basics)
CS222P: Principles of Data Management UCI, Fall 2018 Notes #09 External Sorting Instructor: Chen Li.
Lists in Python Outputting lists.
Lecture 2- Query Processing (continued)
Algorithmic complexity: Speed of algorithms
Computer Science G10 T \2017 T shaikhah AlZeyoudi
Lecture 31: The IO Model 2 Repacking
/London.html Show the URL for the first location.
Sorting Example Bubble Sort
CS222: Principles of Data Management Lecture #10 External Sorting
Chapter 12 Query Processing (1)
PYTHON: BUILDING BLOCKS Inputs & Outputs
CSC 143 Java Applications of Trees.
CMSC201 Computer Science I for Majors Final Exam Information
Topics Introduction to File Input and Output
Algorithmic complexity: Speed of algorithms
Loops.
Sorting We may build an index on the relation, and then use the index to read the relation in sorted order. May lead to one disk block access for each.
CS222P: Principles of Data Management Lecture #10 External Sorting
Creating an account to keep you connected with the world
Database Systems (資料庫系統)
Algorithmic complexity: Speed of algorithms
CS222/CS122C: Principles of Data Management UCI, Fall 2018 Notes #09 External Sorting Instructor: Chen Li.
External Sorting Dina Said
CMPT 120 Lecture 26 – Unit 5 – Internet and Big Data
Presentation transcript:

Python – May 20 Reading data from the Web Example: web.py Practice with large amount of I/O See lab sheet

Web input Just like Java, Python has an easy way to automatically read a Web page import urllib2 page = urllib2.urlopen(“http://--------”) Can continue as if “page” is an input file for line in page: Example: web.py reads a stock quote.

Large input Algorithm needs to be practical, in addition to correct. Beware of nested loops Comments about the data; “density” #3 - County population, area, density Assume that all blocks in county are listed together. #5 – Creating tract-level data Need to look up tract in dictionary search for tract, because blocks in input file are not sorted by tract Once you are finished with a county, you can output all its tracts and empty the dictionary.