Presentation is loading. Please wait.

Presentation is loading. Please wait.

Operators Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Published byCharlene Booth Modified over 9 years ago

Similar presentations

Presentation on theme: "Operators Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See"— Presentation transcript:

1 Operators Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information. Regular Expressions

2 Operators Notebook #1 Site Date Evil (millivaders) ---- ---- ------------------ Baker 1 2009-11-17 1223.0 Baker 1 2010-06-24 1122.7 Baker 2 2009-07-24 2819.0 Baker 2 2010-08-25 2971.6 Baker 1 2011-01-05 1410.0 Baker 2 2010-09-04 4671.6 ⋮ ⋮ ⋮

3 Regular ExpressionsOperators Notebook #1 single tab as separator Site Date Evil (millivaders) ---- ---- ------------------ Baker 1 2009-11-17 1223.0 Baker 1 2010-06-24 1122.7 Baker 2 2009-07-24 2819.0 Baker 2 2010-08-25 2971.6 Baker 1 2011-01-05 1410.0 Baker 2 2010-09-04 4671.6 ⋮ ⋮ ⋮

4 Regular ExpressionsOperators Notebook #1 spaces in site names Site Date Evil (millivaders) ---- ---- ------------------ Baker 1 2009-11-17 1223.0 Baker 1 2010-06-24 1122.7 Baker 2 2009-07-24 2819.0 Baker 2 2010-08-25 2971.6 Baker 1 2011-01-05 1410.0 Baker 2 2010-09-04 4671.6 ⋮ ⋮ ⋮

5 Regular ExpressionsOperators Notebook #1 dates in international standard format (YYYY-MM-DD) Site Date Evil (millivaders) ---- ---- ------------------ Baker 1 2009-11-17 1223.0 Baker 1 2010-06-24 1122.7 Baker 2 2009-07-24 2819.0 Baker 2 2010-08-25 2971.6 Baker 1 2011-01-05 1410.0 Baker 2 2010-09-04 4671.6 ⋮ ⋮ ⋮

6 Regular ExpressionsOperators Notebook #2 Site/Date/Evil Davison/May 22, 2010/1721.3 Davison/May 23, 2010/1724.7 Pertwee/May 24, 2010/2103.8 Davison/June 19, 2010/1731.9 Davison/July 6, 2010/2010.7 Pertwee/Aug 4, 2010/1731.3 Pertwee/Sept 3, 2010/4981.0 ⋮ ⋮ ⋮

7 Regular ExpressionsOperators Notebook #2 slashes as separators Site/Date/Evil Davison/May 22, 2010/1721.3 Davison/May 23, 2010/1724.7 Pertwee/May 24, 2010/2103.8 Davison/June 19, 2010/1731.9 Davison/July 6, 2010/2010.7 Pertwee/Aug 4, 2010/1731.3 Pertwee/Sept 3, 2010/4981.0 ⋮ ⋮ ⋮

8 Regular ExpressionsOperators Notebook #2 Site/Date/Evil Davison/May 22, 2010/1721.3 Davison/May 23, 2010/1724.7 Pertwee/May 24, 2010/2103.8 Davison/June 19, 2010/1731.9 Davison/July 6, 2010/2010.7 Pertwee/Aug 4, 2010/1731.3 Pertwee/Sept 3, 2010/4981.0 ⋮ ⋮ ⋮ month names and day numbers of varying length

9 Regular ExpressionsOperators Regular expressions are patterns that match text. 1. Letters and digits match themselves. 2. '|' means OR. 3. '.' matches any single character. 4. Use '()' to enforce grouping. 5. re.search returns a match object or None. 6. match.group(k) is the text that matched group k.

10 Regular ExpressionsOperators # get fields from Notebook #2 with simple string methods record = 'Davison/May 22, 2010/1721.3' site, date, reading = record.split('/') month, day, year = date.split(' ') if day[-1] == ',': day = day[:-1] print year, month, day 2010 May 22

11 Regular ExpressionsOperators # get fields from Notebook #2 with simple string methods record = 'Davison/May 22, 2010/1721.3' site, date, reading = record.split('/') month, day, year = date.split(' ') if day[-1] == ',': day = day[:-1] print year, month, day 2010 May 22 This is procedural (we tell the computer how )

12 Regular ExpressionsOperators # get fields from Notebook #2 with simple string methods record = 'Davison/May 22, 2010/1721.3' site, date, reading = record.split('/') month, day, year = date.split(' ') if day[-1] == ',': day = day[:-1] print year, month, day 2010 May 22 This is procedural (we tell the computer how ) Regular expressions are declarative (we tell the computer what, and it figures out how)

13 Regular ExpressionsOperators # use '*' to break whole record into pieces match = re.search('(.*)/(.*)/(.*)', 'Davison/May 22, 2010/1721.3') print match.group(1) print match.group(2) print match.group(3) Davison May 22, 2010 1271.3

14 Regular ExpressionsOperators # use '*' to break whole record into pieces match = re.search('(.*)/(.*)/(.*)', 'Davison/May 22, 2010/1721.3') print match.group(1) print match.group(2) print match.group(3) Davison May 22, 2010 1271.3 '*' means "zero or more"

15 Regular ExpressionsOperators # use '*' to break whole record into pieces match = re.search('(.*)/(.*)/(.*)', 'Davison/May 22, 2010/1721.3') print match.group(1) print match.group(2) print match.group(3) Davison May 22, 2010 1271.3 '*' means "zero or more" A postfix operator (like the 2 in x 2 )

16 Regular ExpressionsOperators # use '*' to break whole record into pieces match = re.search('(.*)/(.*)/(.*)', 'Davison/May 22, 2010/1721.3') print match.group(1) print match.group(2) print match.group(3) Davison May 22, 2010 1271.3 '*' means "zero or more" A postfix operator (like the 2 in x 2 ) So (.*) means "zero or more characters"

17 Regular ExpressionsOperators # use '*' to break whole record into pieces match = re.search('(.*)/(.*)/(.*)', 'Davison/May 22, 2010/1721.3') print match.group(1) print match.group(2) print match.group(3) Davison May 22, 2010 1271.3 '*' means "zero or more" A postfix operator (like the 2 in x 2 ) So (.*) means "zero or more characters" But the slashes must match exactly for it to work

18 Regular ExpressionsOperators # but this permits false positives match = re.search('(.*)/(.*)/(.*)', '//') print '*', match.group(1) print '*', match.group(2) print '*', match.group(3) *

19 Regular ExpressionsOperators # but this permits false positives match = re.search('(.*)/(.*)/(.*)', '//') print '*', match.group(1) print '*', match.group(2) print '*', match.group(3) *.* can match the empty string (zero characters)

20 Regular ExpressionsOperators # but this permits false positives match = re.search('(.*)/(.*)/(.*)', '//') print '*', match.group(1) print '*', match.group(2) print '*', match.group(3) *.* can match the empty string (zero characters) So this pattern will accept badly-formatted data

21 Regular ExpressionsOperators # force pattern to match _some_ characters print re.search('(.+)/(.+)/(.+)', '//') None

22 Regular ExpressionsOperators # force pattern to match _some_ characters print re.search('(.+)/(.+)/(.+)', '//') None '+' is a postfix operator meaning "1 or more"

23 Regular ExpressionsOperators # force pattern to match _some_ characters print re.search('(.+)/(.+)/(.+)', 'Davison/May 22, 2010/1721.3') print m.group(1) print m.group(2) print m.group(3) Davison May 22, 2010 1721.3 Always check that it still works with valid data...

24 Regular ExpressionsOperators # write a function to show matched groups def show_groups(pattern, text): m = re.search(pattern, text) if m is None: print 'NO MATCH' return for i in range(1, 1 + len(m.groups())): print '%2d: %s' % (i, m.group(i)) show_groups('(.+)/(.+)/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 22, 2010 3: 1721.3

25 Regular ExpressionsOperators # get the year, month, and day at the same time show_groups('(.+)/(.+) (.+), (.+)/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3

26 Regular ExpressionsOperators # why doesn't this work? show_groups('(.+)/(.+) (.+), (.+)/(.+)', 'Davison/May 22 2010/1721.3') None

27 Regular ExpressionsOperators no comma in the data # why doesn't this work? show_groups('(.+)/(.+) (.+), (.+)/(.+)', 'Davison/May 22 2010/1721.3') None

28 Regular ExpressionsOperators # make the comma optional show_groups('(.+)/(.+) (.+),? (.+)/(.+)', 'Davison/May 22 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3 '?' is a postfix operator meaning "0 or 1"

29 Regular ExpressionsOperators # make the comma optional show_groups('(.+)/(.+) (.+),? (.+)/(.+)', 'Davison/May 22 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3 '?' is a postfix operator meaning "0 or 1" I.e., "optional"

30 Regular ExpressionsOperators # make the comma optional show_groups('(.+)/(.+) (.+),? (.+)/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3 Still works on data with a comma

31 Regular ExpressionsOperators # we _don't_ want to match this show_groups('(.+)/(.+) (.+),? (.+)/(.+)', 'Davison/May 22, 201/1721.3') 1: Davison 2: May 3: 22 4: 201 5: 1721.3

32 Regular ExpressionsOperators # years should have four digits, shouldn't they? show_groups('(.+)/(.+) (.+),? (.+)/(.+)', 'Davison/May 22, 201/1721.3') 1: Davison 2: May 3: 22 4: 201 5: 1721.3 Nobody's prefect

33 Regular ExpressionsOperators # force the pattern to match four characters show_groups('(.+)/(.+) (.+),? (....)/(.+)', 'Davison/May 22, 201/1721.3') None

34 Regular ExpressionsOperators # force the pattern to match four characters show_groups('(.+)/(.+) (.+),? (....)/(.+)', 'Davison/May 22, 201/1721.3') None Won't win any awards for readability

35 Regular ExpressionsOperators # force the pattern to match four characters show_groups('(.+)/(.+) (.+),? (.{4})/(.+)', 'Davison/May 22, 201/1721.3') None show_groups('(.+)/(.+) (.+),? (.{4})/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3

36 Regular ExpressionsOperators # force the pattern to match four characters show_groups('(.+)/(.+) (.+),? (.{4})/(.+)', 'Davison/May 22, 201/1721.3') None show_groups('(.+)/(.+) (.+),? (.{4})/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3 '{N}' is a postfix operator meaning "match N times"

37 Regular ExpressionsOperators # force the day to match to width tests = ( 'Davison/May, 2010/1721.3', 'Davison/May 2, 2010/1721.3', 'Davison/May 22, 2010/1721.3', 'Davison/May 222, 2010/1721.3', 'Davison/May 2, 201/1721.3', 'Davison/ 22, 2010/1721.3', '/May 22, 2010/1721.3', 'Davison/May 22, 2010/' ) pattern = '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' show_matches(pattern, tests)

38 Regular ExpressionsOperators # force the day to match to width tests = ( 'Davison/May, 2010/1721.3', 'Davison/May 2, 2010/1721.3', 'Davison/May 22, 2010/1721.3', 'Davison/May 222, 2010/1721.3', 'Davison/May 2, 201/1721.3', 'Davison/ 22, 2010/1721.3', '/May 22, 2010/1721.3', 'Davison/May 22, 2010/' ) pattern = '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' show_matches(pattern, tests) '{M,N}' matches from M to N times

39 Regular ExpressionsOperators # matching against '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' ** Davison/May, 2010/1721.3 ** Davison/May 2, 2010/1721.3 ** Davison/May 22, 2010/1721.3 Davison/May 222, 2010/1721.3 Davison/May 2, 201/1721.3 Davison/ 22, 2010/1721.3 /May 22, 2010/1721.3 Davison/May 22, 2010/

40 Regular ExpressionsOperators # matching against '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' ** Davison/May, 2010/1721.3 ** Davison/May 2, 2010/1721.3 ** Davison/May 22, 2010/1721.3 Davison/May 222, 2010/1721.3 Davison/May 2, 201/1721.3 Davison/ 22, 2010/1721.3 /May 22, 2010/1721.3 Davison/May 22, 2010/ Why does this match?

41 Regular ExpressionsOperators # look at that test case more closely show_groups('(.+)/(.+) (.{1,2}),? (.{4})/(.+)', 'Davison/May, 2010/1721.3') 1: Davison 2: May 3:, 4: 2010 5: 1721.3

42 Regular ExpressionsOperators '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' 'Davison/May, 2010/1721.3' space matches space

43 Regular ExpressionsOperators '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' 'Davison/May, 2010/1721.3' space matches space '.{1,2}' matches ','

44 Regular ExpressionsOperators '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' 'Davison/May, 2010/1721.3' space matches space '.{1,2}' matches ',' ',?' matches nothing (it's optional)

45 Regular ExpressionsOperators '(.+)/(.+) (.{1,2}),? (.{4})/(.+)' 'Davison/May, 2010/1721.3' space matches space '.{1,2}' matches ',' ',?' matches nothing (it's optional) space matches space again

46 Regular ExpressionsOperators # force a match against digits show_groups('(.+)/(.+) ([0-9]{1,2}),? (.{4})/(.+)', 'Davison/May, 2010/1721.3') None show_groups('(.+)/(.+) ([0-9]{1,2}),? (.{4})/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3

47 Regular ExpressionsOperators # force a match against digits show_groups('(.+)/(.+) ([0-9]{1,2}),? (.{4})/(.+)', 'Davison/May, 2010/1721.3') None show_groups('(.+)/(.+) ([0-9]{1,2}),? (.{4})/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3 '[...]' matches any character in a set

48 Regular ExpressionsOperators # force a match against digits show_groups('(.+)/(.+) ([0-9]{1,2}),? (.{4})/(.+)', 'Davison/May, 2010/1721.3') None show_groups('(.+)/(.+) ([0-9]{1,2}),? (.{4})/(.+)', 'Davison/May 22, 2010/1721.3') 1: Davison 2: May 3: 22 4: 2010 5: 1721.3 '[...]' matches any character in a set E.g., '[aeiou]' matches vowels

49 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)'

50 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Month name begins with upper-case letter...

51 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Month name is an upper-case letter......followed by one or more lower-case letters

52 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Day is one or two digits

53 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Day is one or two digits This format allows '0', '00', '99', and so on

54 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Day is one or two digits This format allows '0', '00', '99', and so on Easiest to check that after converting to integer...

55 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Day is one or two digits This format allows '0', '00', '99', and so on Easiest to check that after converting to integer......since valid ranges depend on the month

56 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Year is exactly four digits

57 Regular ExpressionsOperators # match everything against characters and width p = '(.+)/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/(.+)' Year is exactly four digits Again, check for '0000' and the like after conversion

58 Regular ExpressionsOperators # Put it all together. def get_date(record): '''Return (Y, M, D) as strings, or None.''' # 2010-01-01 m = re.search('([0-9]{4})-([0-9]{2})-([0-9]{2})', record) if m: return m.group(1), m.group(2), m.group(3) # Jan 1, 2010 (comma optional, day may be 1 or 2 digits) m = re.search('/([A-Z][a-z]+) ([0-9]{1,2}),? ([0-9]{4})/', record) if m: return m.group(3), m.group(1), m.group(2) return None

59 June 2010 created by Greg Wilson Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.

Download ppt "Operators Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See"

Similar presentations

CSCI 6962: Server-side Design and Programming Input Validation and Error Handling.

CSCI 6962: Server-side Design and Programming Input Validation and Error Handling.
Slicing Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Slicing Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See
Control Flow Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Control Flow Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See
LING 388: Language and Computers Sandiway Fong Lecture 2: 8/23.

LING 388: Language and Computers Sandiway Fong Lecture 2: 8/23.
IS 1181 IS 118 Introduction to Development Tools Chapter 4 String Manipulation and Regular Expressions.

IS 1181 IS 118 Introduction to Development Tools Chapter 4 String Manipulation and Regular Expressions.
Document Type Definitions. XML and DTDs A DTD (Document Type Definition) describes the structure of one or more XML documents. Specifically, a DTD describes:

Document Type Definitions. XML and DTDs A DTD (Document Type Definition) describes the structure of one or more XML documents. Specifically, a DTD describes:
UNIX Filters.

UNIX Filters.
Haskell Chapter 3, Part I. Pattern Matching  Pattern matching with tuples  Pattern matching with list comprehensions  As-patterns.

Haskell Chapter 3, Part I. Pattern Matching  Pattern matching with tuples  Pattern matching with list comprehensions  As-patterns.
Regex Wildcards on steroids. Regular Expressions You’ve likely used the wildcard in windows search or coding (*), regular expressions take this to the.

Regex Wildcards on steroids. Regular Expressions You’ve likely used the wildcard in windows search or coding (*), regular expressions take this to the.
Regular Expressions. String Matching The problem of finding a string that “looks kind of like …” is common  e.g. finding useful delimiters in a file,

Regular Expressions. String Matching The problem of finding a string that “looks kind of like …” is common  e.g. finding useful delimiters in a file,
More on Regular Expressions 1. 107 - Regular Expressions More character classes \s matches any whitespace character (space, tab, newline etc) \w matches.

More on Regular Expressions Regular Expressions More character classes \s matches any whitespace character (space, tab, newline etc) \w matches.
Regular Expression A regular expression is a template that either matches or doesn’t match a given string.

Regular Expression A regular expression is a template that either matches or doesn’t match a given string.
REGULAR EXPRESSIONS CHAPTER 14. REGULAR EXPRESSIONS A coded pattern used to search for matching patterns in text strings Commonly used for data validation.

REGULAR EXPRESSIONS CHAPTER 14. REGULAR EXPRESSIONS A coded pattern used to search for matching patterns in text strings Commonly used for data validation.
Lesson 3 – Regular Expressions Sandeepa Harshanganie Kannangara MBCS | B.Sc. (special) in MIT.

Lesson 3 – Regular Expressions Sandeepa Harshanganie Kannangara MBCS | B.Sc. (special) in MIT.
Mechanics Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See

Mechanics Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See
Chapter 06: Lecture Notes (CSIT 104) 1 Copyright © 2008 Pearson Prentice Hall. All rights reserved. 1 1 Copyright © 2008 Prentice-Hall. All rights reserved.

Chapter 06: Lecture Notes (CSIT 104) 1 Copyright © 2008 Pearson Prentice Hall. All rights reserved. 1 1 Copyright © 2008 Prentice-Hall. All rights reserved.
Last Updated March 2006 Slide 1 Regular Expressions.

Last Updated March 2006 Slide 1 Regular Expressions.
1 Lab Session-III CSIT-120 Fall 2000 Revising Previous session Data input and output While loop Exercise Limits and Bounds Session III-B (starts on slide.

1 Lab Session-III CSIT-120 Fall 2000 Revising Previous session Data input and output While loop Exercise Limits and Bounds Session III-B (starts on slide.
Pattern matching with regular expressions A common file processing requirement is to match strings within the file to a standard form, e.g. email address.

Pattern matching with regular expressions A common file processing requirement is to match strings within the file to a standard form, e.g. address.
Chapter 6 Functions -- QuickStart. The Practice of Computing Using Python, Punch & Enbody, Copyright © 2013 Pearson Education, Inc. What is a function?

Chapter 6 Functions -- QuickStart. "The Practice of Computing Using Python", Punch & Enbody, Copyright © 2013 Pearson Education, Inc. What is a function?

Similar presentations

Ads by Google