CSE 341 -- S. Tanimoto Regular Expressions 1 Regular Expressions: Theory and Perl Implementation Outline: 1. Theoretical Definitions and Examples 2. Acceptance.

Slides:



Advertisements
Similar presentations
Lexical Analysis Dragon Book: chapter 3.
Advertisements

C O N T E X T - F R E E LANGUAGES ( use a grammar to describe a language) 1.
AND FINITE AUTOMATA… Ruby Regular Expressions. Why Learn Regular Expressions? RegEx are part of many programmer’s tools  vi, grep, PHP, Perl They provide.
1 Regular Expressions & Automata Nelson Padua-Perez Bill Pugh Department of Computer Science University of Maryland, College Park.
Regular Expressions in Java. Regular Expressions A regular expression is a kind of pattern that can be applied to text ( String s, in Java) A regular.
Regular Expressions Regular Expression (or pattern) in Perl – is a template that either matches or doesn’t match a given string. if( $str =~ /hello/){
COS 320 Compilers David Walker. Outline Last Week –Introduction to ML Today: –Lexical Analysis –Reading: Chapter 2 of Appel.
Regular Expressions & Automata Fawzi Emad Chau-Wen Tseng Department of Computer Science University of Maryland, College Park.
1 Overview Regular expressions Notation Patterns Java support.
Regular Expressions Regular Expression (or pattern) in Perl – is a template that either matches or doesn’t match a given string. if( $str =~ /hello/){
Scripting Languages Chapter 8 More About Regular Expressions.
XP Tutorial 14 New Perspectives on HTML, XHTML, and DHTML, Comprehensive 1 Working with Forms and Regular Expressions Validating a Web Form with JavaScript.
Last Updated March 2006 Slide 1 Regular Expressions.
Copyright © Cengage Learning. All rights reserved.
Tutorial 14 Working with Forms and Regular Expressions.
Lecture 7: Perl pattern handling features. Pattern Matching Recall =~ is the pattern matching operator A first simple match example print “An methionine.
Language Recognizer Connecting Type 3 languages and Finite State Automata Copyright © – Curt Hill.
Computer Programming for Biologists Class 5 Nov 20 st, 2014 Karsten Hokamp
Introduction to Computing Using Python Regular expressions Suppose we need to find all addresses in a web page How do we recognize addresses?
Lexical Analysis CSE 340 – Principles of Programming Languages Fall 2015 Adam Doupé Arizona State University
Compiler Phases: Source program Lexical analyzer Syntax analyzer Semantic analyzer Machine-independent code improvement Target code generation Machine-specific.
REGULAR EXPRESSIONS. Lexical Analysis Lexical analysers can be constructed by programs such as LEX These programs employ as input a description of the.
어휘분석 (Lexical Analysis). Overview Main task: to read input characters and group them into “ tokens. ” Secondary tasks: –Skip comments and whitespace;
CSE 311 Foundations of Computing I Lecture 17 Structural Induction: Regular Expressions, Regular Languages Autumn 2011 CSE 3111.
Review: Regular expression: –How do we define it? Given an alphabet, Base case: – is a regular expression that denote { }, the set that contains the empty.
Lexical Analysis (I) Compiler Baojian Hua
COMP 3438 – Part II - Lecture 2: Lexical Analysis (I) Dr. Zili Shao Department of Computing The Hong Kong Polytechnic Univ. 1.
Lexical Analysis I Specifying Tokens Lecture 2 CS 4318/5531 Spring 2010 Apan Qasem Texas State University *some slides adopted from Cooper and Torczon.
COMP313A Programming Languages Lexical Analysis. Lecture Outline Lexical Analysis The language of Lexical Analysis Regular Expressions.
SCRIBE SUBMISSION GROUP 8 Date: 7/8/2013 By – IKHAR SUSHRUT MEGHSHYAM 11CS10017 Lexical Analyser Constructing Tokens State-Transition Diagram S-T Diagrams.
COMP3190: Principle of Programming Languages DFA and its equivalent, scanner.
CSE 311 Foundations of Computing I Lecture 17 Structural Induction Spring
REGEX. Problems Have big text file, want to extract data – Phone numbers (503)
Overview A regular expression defines a search pattern for strings. Regular expressions can be used to search, edit and manipulate text. The pattern defined.
1 Course Overview PART I: overview material 1Introduction 2Language processors (tombstone diagrams, bootstrapping) 3Architecture of a compiler PART II:
Review: Compiler Phases: Source program Lexical analyzer Syntax analyzer Semantic analyzer Intermediate code generator Code optimizer Code generator Symbol.
CPS 506 Comparative Programming Languages Syntax Specification.
20-753: Fundamentals of Web Programming 1 Lecture 10: Server-Side Scripting II Fundamentals of Web Programming Lecture 10: Server-Side Scripting II.
CS 330 Programming Languages 10 / 02 / 2007 Instructor: Michael Eckmann.
Overview of Previous Lesson(s) Over View  Symbol tables are data structures that are used by compilers to hold information about source-program constructs.
1 Lecture 9 Shell Programming – Command substitution Regular expressions and grep Use of exit, for loop and expr commands COP 3353 Introduction to UNIX.
1Computer Sciences Department. Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Reference 3Computer Sciences Department.
CSC3315 (Spring 2009)1 CSC 3315 Lexical and Syntax Analysis Hamid Harroud School of Science and Engineering, Akhawayn University
Introduction to Programming the WWW I CMSC Winter 2004 Lecture 13.
XP Tutorial 7 New Perspectives on JavaScript, Comprehensive 1 Working with Forms and Regular Expressions Validating a Web Form with JavaScript.
An Introduction to Regular Expressions Specifying a Pattern that a String must meet.
using Deterministic Finite Automata & Nondeterministic Finite Automata
CS 404Ahmed Ezzat 1 CS 404 Introduction to Compiler Design Lecture 1 Ahmed Ezzat.
LECTURE 5 Scanning. SYNTAX ANALYSIS We know from our previous lectures that the process of verifying the syntax of the program is performed in two stages:
CS 330 Programming Languages 09 / 30 / 2008 Instructor: Michael Eckmann.
OOP Tirgul 11. What We’ll Be Seeing Today  Regular Expressions Basics  Doing it in Java  Advanced Regular Expressions  Summary 2.
Deterministic Finite Automata Nondeterministic Finite Automata.
ICS611 Lex Set 3. Lex and Yacc Lex is a program that generates lexical analyzers Converting the source code into the symbols (tokens) is the work of the.
CSE 341 S. Tanimoto Perl Perl: User Defined Functions sub funny_print { my ($name, $age) print
CS 330 Class 7 Comments on Exam Programming plan for today:
Theory of Computation Lecture #
Chapter 3 Lexical Analysis.
Lexical Analysis CSE 340 – Principles of Programming Languages
Chapter 2 Scanning – Part 1 June 10, 2018 Prof. Abdelaziz Khamis.
Lecture 9 Shell Programming – Command substitution
CSE 311 Foundations of Computing I
Week 14 - Friday CS221.
Review: Compiler Phases:
Compiler Construction
Regular Expressions: Theory and Perl Implementation
Regular Expressions: Theory and Perl Implementation
Regular Expressions: Theory and Perl Implementation
Regular Expression: Pattern Matching
Lecture 5 Scanning.
REGEX.
Presentation transcript:

CSE S. Tanimoto Regular Expressions 1 Regular Expressions: Theory and Perl Implementation Outline: 1. Theoretical Definitions and Examples 2. Acceptance by Finite Automata 3. Perl’s Syntax 4. Other pattern matching functionality in Perl 5. Program Example

CSE S. Tanimoto Regular Expressions 2 Alphabets and Sets of Strings An alphabet  = {a 1, a 2,..., a n } is a set of characters. A string over  is a sequence of zero or more elements of . Example. If  = {0, 1, 2} then 2201 is a string over . No matter what  is, the empty string  is a string over . A set of strings over  is a set of zero or more strings, each of which is a string over . Example. If  = {0, 1, 2} then { , 111, 121, 0} is a set of strings over .

CSE S. Tanimoto Regular Expressions 3 A Recursive Definition for Regular Expressions A regular expression for an alphabet  is a certain kind of pattern that describes a set of strings over . Any character c in  is a regular expression representing {c} If E, E 1 and E 2 are regular expressions over  then so are E 1 E 2 -- representing the set concatenation of E 1 and E 2. E 1 | E 2 -- representing alternation of E 1 and E 2. ( E ) -- representing E grouped with parentheses. E + -- rep. one or more instances of E concatenated. E* -- zero or more instances of E

CSE S. Tanimoto Regular Expressions 4 Regular Expression Examples Let  = {a, b}. a = {a} ab = {ab} a | b = {a, b} a + = {a, aa, aaa,... } ab* represents the set of strings having a single a followed by zero or more occurrences of b. That is, it’s {a, ab, abb, abbb,... } a (b | c) = {ab, ac} (a | b) (c | d) = {ac, ad, bc, bd} aa* = a + = {a, aa, aaa,... }

CSE S. Tanimoto Regular Expressions 5 Extended Regular Expressions Let letters = a | b | c | d Let digits = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 Let identifiers = letters ( letters | digits )* Thus we can use a name to represent a set of strings and use that name in a regular expression.

CSE S. Tanimoto Regular Expressions 6 Finite Automaton a b a corresponding regular expression: ab*a start stateaccepting state Example: process the string abba Now try abbb Finite number of states, but number of strings is not necessarily finite.

CSE S. Tanimoto Regular Expressions 7 Equivalence of Finite Automata and Regular Expressions b a ab a | b a* a a a b a

CSE S. Tanimoto Regular Expressions 8 Regular Expressions in Perl In Perl, regular expressions are used to specify patterns for pattern matching. $sentence = "Winter weather has arrived." if ($sentence =~ /weather/) { print "Never bet on the weather." ; } # $string =~ /Pattern/ The result of this kind of pattern matching is a true or false value.

CSE S. Tanimoto Regular Expressions 9 A Perl Regular Expression for Identifier $identifier = "[a-z][a-z0-9]*"; $sentence = "012,cse ,ABC]*"; if ($sentence =~ /$identifier/) { print "Seems to be an identifier here." ; } $ident2 = "[a-zA-Z][a-zA-Z0-9]*"; $reservedWord = "begin|end";

CSE S. Tanimoto Regular Expressions 10 Specifying Patterns /Pattern/ # Literal text; # true if it occurs anywhere in the string. /^Pattern/ # Must occur at the beginning. "Pattern recognition is alive" =~ /^Pattern/ "The end" =~ /end$/ \s whitespace \S non-whitespace \w a word char. \W a non-word char. [a-zA-Z_0-9] \d a digit \D a non-digit \b word boundary \B not word boundary

CSE S. Tanimoto Regular Expressions 11 Specifying Patterns (Cont.) $test = "You have new mail "; if ($test =~ /^You\s.+\d+-\d+-\d+/ ) { print "The mail has arrived."; } if ($test =~ m( ^ You \s.+ \d+ - \d+ - \d+ ) { print "The mail has arrived."; }

CSE S. Tanimoto Regular Expressions 12 Extracting Information $test = "You have new mail "; if ($test =~ /^You\s.+(\d+)-(\d+)-(\d+)/ ) { print "The mail has arrived on "; print "day $2 of month $1 in year $3.\n"; } # Parentheses in the pattern establish # variables $1, $2, $3, etc. to hold # corresponding matched fragments.

CSE S. Tanimoto Regular Expressions 13 Search and Replace $sntc = "We surfed the waves the whole day." $sntc =~ s/surfed/sailed/; print $sntc; # We sailed the waves the whole day. $sntc =~ s/the//g; print $sntc; # We sailed waves whole day. # g makes the replacement “global”.

CSE S. Tanimoto Regular Expressions 14 Interpolation of Variables in Replacements $exclamation = "yeah"; $sntc = "We had fun." $sntc =~ s/w+/$exclamation/g; print $sntc; # yeah yeah yeah. # a pattern can contain a Perl variable.

CSE S. Tanimoto Regular Expressions 15 Example of (Crude) Lexical Analysis $ident = "[a-zA-Z][a-zA-Z0-9]*"; $int = "[\-]?[0-9]+"; $op = "[\-\+\*\/\=]|mod"; $exp = "begin x = 5; print sqrt(x); end"; $exp =~ s/$ident/ID/g; $exp =~ s/$int/N/g; $exp =~ s/$op/OP/g; print $exp; ID ID OP N; ID ID(ID); ID

CSE S. Tanimoto Regular Expressions 16 Processing Assignment Submissions Using Forms and Files 1. Form file 2. Perl script to process data from form. 3. Perl script to “compile” data into an index page.

CSE S. Tanimoto Regular Expressions 17 The HTML Form Submission for CSE 341 Miniproject Topic Proposals CSE 341 Miniproject Topic Proposal Submission Form Write a topic-proposal web page, and then fill out this form and submit it by Thursday, February 24 at 5:00 PM. (The web page should follow these guidelines.) <form method=post action=" student/process-topic-proposal.pl">

CSE S. Tanimoto Regular Expressions 18 The HTML Form (2 of 2) Possible name of project: Name of Possible partner (optional): URL of a web page that describes your proposal: If you plan to submit another topic proposal because you are very uncertain about whether to stick with this one, check this box:

CSE S. Tanimoto Regular Expressions 19 Perl Script to Process Data From Form #! /usr/bin/perl # Process the miniproject topic proposal form inputs # S. Tanimoto, 20 Feb 2000 use CGI qw/:standard/; use strict; print header; my $projectname = param("projectname"); my $uncertain = param("uncertain"); my $partner = param("partner"); my $proposal_url = param("proposalurl"); my $student_username = $ENV{"REMOTE_USER"}; my $now = localtime(); $projectname =~ s/[^a-zA-Z0-9\-\~]//g; $partner =~ s/[^a-zA-Z0-9\-\~]//g; $proposal_url =~ s/[^a-zA-Z0-9\-\~]//g;

CSE S. Tanimoto Regular Expressions 20 Perl Script to Process the Data (2 of 2) my $output_line = "STUDENT_USERNAME=$student_username; ". "PROPOSAL_URL=$proposal_url; ". "PROJECT_NAME=$projectname; ". "PARTNER=$partner; ". "UNCERTAIN=$uncertain; ". "DATE=$now; "; if (! (open(OUT, ">>MP-topic-proposal-data.txt"))) { print("Error: could not open topic file for output."); print("Please notify instructor and/or try again later."); print end_html; exit 0; } print OUT $output_line, "\n"; close OUT; print h1("Your miniproject topic proposal has been received. Thanks!"); print end_html;

CSE S. Tanimoto Regular Expressions 21 Perl Script to “Compile” the Data #!/usr/bin/perl # make-MP-index-of-proposed-topics.pl use strict; use CGI qw/:standard/; open(INFILE, "<MP-topic-proposal-data-sorted.txt") || die("Could not open the file MP-topic-proposal-data-sorted.txt.\n"); print<<"EOT"; CSE 341 MP Topic Proposal Index CSE 341 MP Topic Proposal Index EOT print " Student username Proposal Page Partner Certainty When \n"; my $projectname; my $uncertain; my $partner; my $proposal_url; my $student_username; my $date;

CSE S. Tanimoto Regular Expressions 22 Perl Script to “Compile” the Data (2 of 3) while ( ) { if ( /STUDENT_USERNAME=([^\;]+);\s/){$student_username =$1; } else { $student_username =""; } if ( /PROJECT_NAME=([^\;]+);\s/){$projectname =$1; } else { $projectname =""; } if ( /PROPOSAL_URL=([^\;]+);\s/){$proposal_url =$1; } else { $proposal_url =""; } if ( /PARTNER=([^\;]+);\s/){$partner =$1; } else { $partner =""; } if ( /UNCERTAIN=([^\;]+);\s/){$uncertain =$1; } else { $uncertain =""; } if ( /DATE=([^\;]+);/){$date = $1; } else { $date = ""; } if ($proposal_url =~ /http/ ) {} else { $proposal_url = " $proposal_url; } if ($uncertain eq "No") { $uncertain = ""; } else { $uncertain = "Uncertain"; }

CSE S. Tanimoto Regular Expressions 23 Perl Script to “Compile” the Data (3 of 3) my $link = " $projectname "; print " $student_username $link $partner $uncerta in $date \n"; } print " \n";