Iterator Pattern: Generator Expressions

Slides:

Advertisements

Similar presentations

Files CSE 1310 – Introduction to Computers and Programming Vassilis Athitsos University of Texas at Arlington 1.

Advertisements

File I/O Supplemental Material. Background In C++, files can be manipulated in the same manner we manipulate streams such as: cout and cin. Therefore,

Filters using Regular Expressions grep: Searching a Pattern.

 1 Loop and Set operations. Creating a loop Loops are used to repeat a computation. Let’s see the example of factorial computation. Let’s compute 10!

The if statement and files. The if statement Do a code block only when something is True if test: print "The expression is true"

Python Programming Chapter 6: Iteration Saad Bani Mohammad Department of Computer Science Al al-Bayt University 1 st 2011/2012.

Iteration: WHILE Loop Damian Gordon. WHILE Loop Consider the problem of searching for an entry in a phone book with only SELECTION:

Introduction to Unix – CS 21 Lecture 8. Lecture Overview More detail on emacs and vi Regular expression matching in emacs and vi.

Variables and Functions. Open your Encoder program Let’s begin by opening the “Labyrinth Auto Straight” code. Save this file as Labyrinth with variables.

Advanced Topics- Functions Introduction to MATLAB 7 Engineering 161.

Python uses boolean variables to evaluate conditions. The boolean values True and False are returned when an expression is compared or evaluated.

Modularisation Damian Gordon. Modularisation Let’s imagine we had code as follows:

January 13, 2014 Multivariate Expressions and Equations.

POLYNOMIALS - Evaluating When evaluating polynomials, we are simply substituting a value in for a variable wherever that variable appears in the expression.

1 SEEM3460 Tutorial Compiling and Debugging C programs.

App Engine Web App Framework Jim Eng

Python Functions : chapter 3

Pogamut 2 Platform for fast development of cognitive agents inside 3D environment Creating javabot project / pitfalls Startup/running/shutdown sequence.

2000 Copyrights, Danielle S. Lahmani Foreach example = ( 3, 5, 7, 9) foreach $one ) { $one*=3; } is now (9,15,21,27)

Python: Iteration Damian Gordon. Python: Iteration We’ll consider four ways to do iteration: – The WHILE loop – The FOR loop – The DO loop – The LOOP.

PYTHON FOR HIGH PERFORMANCE COMPUTING. OUTLINE  Compiling for performance  Native ways for performance  Generator  Examples.

Files in Python Opening and Closing. Big Picture To use a file in a programming language – You have to open the file – Then you process the data in the.

Iteration: FOR, DO, LOOP Loop Damian Gordon. FOR Loop The FOR loop does the same thing as a WHILE loop but is easier if you are using the loop to do a.

 2008 Pearson Education, Inc. All rights reserved Control Statements: Part 2.

Recursion ● Recursion or Iteration ● A Common Recursive Design Pattern ● Computing factorials ● Searching a filesystem tree ● Faster exponentiation ● Slow.

Lesson 06: Functions Class Participation: Class Chat:

CSE 374 Programming Concepts & Tools

Exam #1 You will have exactly 30 Mins to complete the exam.

Java Programming Lecture 2

I/O Streams File I/O 2-D array review

Input from STDIN STDIN, standard input, comes from the keyboard.

Iterator Pattern: Comprehensions

Python Comprehension and Generators

Testing Key Revision Points.

Chapter 14: Exception Handling

CHAPTER FOUR Functions.

The ‘grep’ Command Colin Masterson.

Using the Python Logging System

Functions BIS1523 – Lecture 17.

Python Primer 2: Functions and Control Flow

File Handling Programming Guides.

Conditions and Ifs BIS1523 – Lecture 8.

ms vısual studıo 2008-Introductıon TUTORIAL

Text Files All the programs you've seen so far have one thing in common: Any data the program uses or calculates is lost when the program ends. In order.

Multiple Inheritance Damian Gordon.

Tonga Institute of Higher Education

Coding Concepts (Basics)

Lesson 06: Functions Class Chat: Attendance: Participation

Debugging at Scale.

Using a Debugger 1-Jan-19.

Basic Inheritance Damian Gordon.

Testing, debugging, and using support libraries

Simple Statistics on Arrays

Some Common Issues: Iteration

EECS 111 Review 11/13/2016.

Sorting Damian Gordon.

Python I/O Peter Wad Sackett.

How to debug a website using IE F12 tools

Introduction to Computer Science

Algorithm Analysis.

“Everything Else”.

Subject:Object oriented programming

Chapter 3 Debugging Section 3.4

Expressions & Equations Jeopardy

Chapter 9 - Action Queries

QTP Test Process

functions are also data! other functions as input!

PseudoCode Damian Gordon.

Presentation transcript:

Iterator Pattern: Generator Expressions Damian Gordon

Generator Expressions

Iterator Pattern Let’s imagine we are processing a large log file:

Iterator Pattern Let’s imagine we are processing a large log file: : Jan 26, 2010 11:25:25 DEBUG This is a debugging message. Jan 26, 2010 11:25:36 INFO This is an information method. Jan 26, 2010 11:25:46 WARNING This is a warning. It could be serious. Jan 26, 2010 11:25:52 WARNING Another warning sent. Jan 26, 2010 11:25:59 INFO Here's some information. Jan 26, 2010 11:26:13 DEBUG Debug messages. Jan 26, 2010 11:26:32 INFO Information is usually harmless, but helpful. Jan 26, 2010 11:26:40 WARNING Warnings should be heeded. Jan 26, 2010 11:26:54 WARNING Watch for warnings.

Iterator Pattern If the log file is very big (a few terabytes) and we were looking only for “WARNING” messages, we shouldn’t use a List Comprehension in this case because it would temporarily create a list containing every line and then search for the appropriate messages. Instead we can use Generator Expressions (or Generators).

Iterator Pattern To create a Generator Expressions we simply wrap the comprehension in parenthesis ‘()’ instead of ‘[]’ or ‘{}’.

Iterator Pattern To create a Generator Expressions we simply wrap the comprehension in parenthesis ‘()’ instead of ‘[]’ or ‘{}’. import sys InName = "InputFile.txt" OutName = "OutputFile.txt" with open(InName) as infile: with open(OutName, "w") as outfile: warnings = (line for line in infile if 'WARNING' in line) for line in warnings: # DO outfile.write(line) # ENDFOR; # END.

Iterator Pattern To create a Generator Expressions we simply wrap the comprehension in parenthesis ‘()’ instead of ‘[]’ or ‘{}’. import sys InName = "InputFile.txt" OutName = "OutputFile.txt" with open(InName) as infile: with open(OutName, "w") as outfile: warnings = (line for line in infile if 'WARNING' in line) for line in warnings: # DO outfile.write(line) # ENDFOR; # END. Write each line that matches the criteria into “warnings” Call each line in the file line, one after the next, after the next Select only lines that have the word “WARNING”

Iterator Pattern To create a Generator Expressions we simply wrap the comprehension in parenthesis ‘()’ instead of ‘[]’ or ‘{}’. import sys InName = "InputFile.txt" OutName = "OutputFile.txt" with open(InName) as infile: with open(OutName, "w") as outfile: warnings = (line for line in infile if 'WARNING' in line) for line in warnings: # DO outfile.write(line) # ENDFOR; # END. Create this variable as a generator expression Write each line that matches the criteria into “warnings” Call each line in the file line, one after the next, after the next Select only lines that have the word “WARNING”

Iterator Pattern Our output file will look like this: Jan 26, 2010 11:25:46 WARNING This is a warning. It could be serious. Jan 26, 2010 11:25:52 WARNING Another warning sent. Jan 26, 2010 11:26:40 WARNING Warnings should be heeded. Jan 26, 2010 11:26:54 WARNING Watch for warnings.

Iterator Pattern All of the lines in the output file say ‘WARNING’:

Iterator Pattern All of the lines in the output file say ‘WARNING’: Jan 26, 2010 11:25:46 WARNING This is a warning. It could be serious. Jan 26, 2010 11:25:52 WARNING Another warning sent. Jan 26, 2010 11:26:40 WARNING Warnings should be heeded. Jan 26, 2010 11:26:54 WARNING Watch for warnings.

Iterator Pattern So we could write a new program to delete the word “WARNING” out of each line:

Iterator Pattern So we could write a new program to delete the word “WARNING” out of each line: import sys inname = "OutputFile.txt" outname = "OutputFile2.txt" with open(inname) as infile: with open(outname, "w") as outfile: warnings = (line.replace(' WARNING', '') for line in infile if 'WARNING' in line) for line in warnings: outfile.write(line) # ENDFOR; # END.

Iterator Pattern And we get: Jan 26, 2010 11:25:46 This is a warning. It could be serious. Jan 26, 2010 11:25:52 Another warning sent. Jan 26, 2010 11:26:40 Warnings should be heeded. Jan 26, 2010 11:26:54 Watch for warnings.

Iterator Pattern To do the same thing in a more object-oriented manner:

Iterator Pattern To do the same thing in a more object-oriented manner: import sys inname = "OutputFile.txt" outname = "OutputFile3.txt" def warnings_filter(insequence): for line in insequence: if 'WARNING' in line: yield line.replace(' WARNING', '') with open(inname) as infile: with open(outname, "w") as outfile: filter = warnings_filter(infile) for line in filter: outfile.write(line)

Iterator Pattern To do the same thing in a more object-oriented manner: import sys inname = "OutputFile.txt" outname = "OutputFile3.txt" def warnings_filter(insequence): for line in insequence: if 'WARNING' in line: yield line.replace(' WARNING', '') with open(inname) as infile: with open(outname, "w") as outfile: filter = warnings_filter(infile) for line in filter: outfile.write(line) The yield command works exactly like the normal return command (when you return a value from a method), but it temporarily returns control to the calling method, and remembers where it was in a sequence in each new call.

etc.