1. Introduction to Programming the WWW I CMSC 10100-1 Summer 2004 Lecture 13

2. 2 Today’s Topics CGI module Patterns and regular expression

3. 3 Perl Modules A Perl module is a self-contained piece of Perl code that can be used by a Perl program or by other Perl modules  Conceptually similar to a C link library, or a C++ class  Perl 5 module list Perl 5 module list Each Perl module has a unique name  Perl provides a hierarchal name space for modules  Components of a module name are separated by double colons (::)  Example: CGI Math::Complex

4. 4 Perl Modules (cont’d) Each module is contained in a single file  Module files are stored in a subdirectory hierarchy that parallels the module name hierarchy  All module files have an extension of.pm  Example: Math::Complex is stored in Math/Complex.pm Finding module libraries  The Perl interpreter has a list of directories in which it searches for modules. This list is available in the global array @INC  Use perl –V to see the initial contents of @INC Local modules vs. modules coming from standard distribution CGI stored in /opt/perl/perl-5.005.03/lib/5.00503/CGI.pm Math::Complex is actually stored in /opt/perl/perl-5.005.03/lib/5.00503/Math/Complex.pm

5. 5 Using Perl Modules Modules must be imported in order to be accessible to a script  This is done with the use function  use statements are commonly made at the beginning of a program or subroutine This makes it easier to understand the program and see which modules are loaded.  Example: use Math::Complex; use CGI “:standard”; http://world.std.com/~swmcd/steven/perl/module_mechanics.html A modifier to a module

6. 6 Using CGI.pm to generate HTML The CGI.pm module provides several functions that can be used to concisely output HTML tags For example, $mypage=‘It is a New Day’; print “ $mypage ”; can also be written as: $mypage=’It is a New Day’; print start_html(‘$mypage’);

7. 7 3 Basic CGI.pm Modules header  creates the MIME Content-type line start_html  creates starting HTML tags end_html  creates ending HTML tags 1. #!/usr/local/bin/perl 2. use CGI ‘:standard’; 3. print header; 4. print start_html; 5. print ' '; 6. print 'Welcome humans to my site '; 7. print end_html; http://people.cs.uchicago.edu/~hai/hw4/cgipm1.cgi

8. 8 CGI.pm Basic Functions The various CGI/PM function accept 3 basic syntactic formats:  No argument format functions that can be used without any arguments  Positional argument format functions that can accept comma-separated arguments within parentheses  Name-value argument format functions that accept parameters submitted as name- and-value pairs

9. 9 No Argument Format The Previous Example shows the start_html, header, end_html functions You can place the 1 or more functions directly within a print statement  Would output

10. 10 Some Single Argument Functions CGI.pm Function Example of UseExample Output header - the MIME Content-type line print header;Content- type:text/html\n\n start_html — Tags to start an HTML document print start_html; br —output tag print br; hr —generate horizontal rule print hr; end_html —end an HTML document print end_html;

11. 11 Positional Argument Format Specify multiple arguments based on the position of the argument For example would output Hello World

12. 12 Some Positional Functions CGI.pm Functions Example of UseExample Output start_html() — tags needed to start an HTML document. start_html(‘My Page’); My Page h1() —header level 1 tags. (also h2(), …, h6() ) print h1(‘Hello There’); Hello There strong() – output argument in strong. print strong('Now'); Now p() —creates a paragraph. print p(‘Time to move’); Time to move b() —prints the argument in bold. print b('Exit'); Exit i() —prints the argument in italics. Print i('Quick'); Quick

13. 13 Operating on Variables Can concisely use functions with a single print statement:  print i('Please '),'come when I call you ', strong('immediately.'); This code would output the following:  Please come when I call you immediately.

14. 14 Consider the following example: 1. # !/usr/local/bin/perl 2. use CGI ':standard'; 3. print header, start_html(‘Positional Example’), h1('Simple Math'); 4. print b('two times two='), 2*2; 5. print br, 'but ', b('four times four='), 4*4; 6. print br, 'Finally, ', b('eight times eight='), 8*8; 7. print end_html; http://people.cs.uchicago.edu/~hai/hw4/cgipm2.cgi

15. 15 Name-Value Argument Format Can specify names and values as follows: Would output the following: My Title

16. 16 Some name/value functions CGI.pm Function Example UsageExample Output start_html start HTML document print start_html({ -title=>‘my title’, –bgcolor=>’red’ }); my title img —inserts an image print img({- src=>'myfile.gif', -alt=>’picture’}); a —establishes links print a({ -href =>'http://www.mysite.com'}, 'Click Here'); Click Here font() — creates … tags print font( { -color=>‘BLUE’, –size=> ’4’}, ‘Lean, and mean.’); Lean, and mean.

17. 17 Example Name/Value Program 1. #!/usr/local/bin/perl 2. use CGI ':standard'; 3. print header; 4. print start_html({-title=>'New Day ', -bgcolor=>'yellow'}); 5. print 'Welcome One And ', i('All'); 6. print end_html; http://people.cs.uchicago.edu/~hai/hw4/cgipm3.cgi

18. 18 Using CGI.pm with HTML forms CGI.pm Function Example UsageExample Output start_form start HTML form element print start_form({ -method=>‘post’, –action=> ‘http://people.cs.uchicago.edu/~w freis/cgi-bin/reflector.pl’}); textfield, password_field —inserts a text field or password field print textfield(- name=>'textfield1', -size=>'50', -maxlength=>'50'); scrolling_list —insert a multiple list print scrolling_list(-name=>'list1', -values=> ['eenie', 'minie', 'moe'], -default=> ['eenie','moe'], -size=>5, -multiple=>'true'); eenie minie moe textarea — inserts a text area print textarea(-name=> 'large_field_name', -rows=> 10, -columns=>50);

19. 19 Using CGI.pm with HTML forms (cont’d) CGI.pm Function Example UsageExample Output checkbox_group – insert checkbox print checkbox_group(-name=> 'color', -values=>['red ','orange ','yellow '], -default=>['red ']); red orange yellow raidio-group — inserts a text field print radio_group(-name=>'color blind', -values=>['Yes','No'], - default=>'No'); Yes No submit,reset —insert a submit or reset button print submit('submit', 'Submit'); Print reset; endform — print end form tag print endform(); Perl CGI Reference

20. 20 A CGI Form Example http://people.cs.uchicago.edu/~hai/hw4/cgiform1.cgi

21. 21 Receiving HTML Form Arguments Within the CGI program call param() function  Input variables into CGI/Perl programs are called CGI variables Values received through your Web server as input from a Web browser, usually filled in a form  To use param():

22. 22 Receiving HTML Form Arguments http://people.cs.uchicago.edu/~hai/hw4/cgiform1.cgi

23. 23 Sending Arguments You can send arguments to your CGI program directly from the URL address of a browser http://people.cs.uchicago.edu/~hai/hw4/cgiform1 _checker.cgi?color=red

24. 24 Sending Multiple Arguments Precede first argument with ? Precede next argument with & http://people.cs.uchicago.edu/~hai/hw4/cgiform1_checker.cgi?color=red&secret=nothing

25. 25 Debug CGI Program in Command Line To start and send an argument to the password program can execute the following: perl cgiform1_checker.cgi color=red Enclose blank spaces or multiple arguments in quotation marks: perl cgiform1_checker.cgi ‘color=rose red’ perl cgiform1_checker.cgi 'color=red&secret=none'

26. 26 Check CGI Variables Values Perl provides a simple method to test if any parameters were received or null: $var = param(‘some_cgi_variable’) ; if ($var) { statement(s) to execute when $var has a value } else { statement(s) to execute when $var has no value }

27. 27 Combining Program Files Applications so far have required two separate files; one file for to generate the form, and the other to process the form  Example: cgiform1.cgi and cgiform1_checker.cgi  Can test return value on param() to combine these At least two advantages  With one file, it is easier to change arguments  It is easier to maintain one file

28. 28 Combining Program Files http://people.cs.uchicago.edu/~hai/hw4/cgiform2.cgi

29. 29 CGI Module: Advanced Topic* Functional(procedural) Orientation  use CGI ‘:standard’; Object Orientation  use CGI;  Call new() operator to create a CGI object and stores in a variable. The functions of CGI.pm are accessed through the -> operator with the object variable at the left side $q = new CGI; print $q->header();  http://www.classes.cs.uchicago.edu/classes/archive/2 004/winter/10100-1/02/perl/perl_index.html http://www.classes.cs.uchicago.edu/classes/archive/2 004/winter/10100-1/02/perl/perl_index.html

30. 30 Several Resources URL  http://www.classes.cs.uchicago.edu/classes/archive /2004/winter/10100-1/02/perl/perl_index.html http://www.classes.cs.uchicago.edu/classes/archive /2004/winter/10100-1/02/perl/perl_index.html Topics  How to write your first CGI script How to write your first CGI script  Checking CGI Parameters on the Command Line Checking CGI Parameters on the Command Line  Server-side Validation Server-side Validation  Hidden HTML Form Fields Hidden HTML Form Fields  Sorting with Perl Sorting with Perl

31. 31 Patterns in String Variables Many programming problems require matching, changing, or manipulating patterns in string variables.  An important use is verifying input fields of a form helps provide security against accidental or malicious attacks. For example, if expecting a form field to provide a telephone number as input, your program needs a way to verify that the input comprises a string of seven digits.

32. 32 Four Different Constructs Will look at 4 different Perl String manipulation constructs:  The match operator enables your program to look for patterns in strings.  The substitute operator enables your program to change patterns in strings.  The split function enables your program to split strings into separate variables based on a pattern. (already covered)  Regular expressions provide a pattern matching language that can work with these operators and functions to work on string variables.

33. 33 The Match Operator The match operator is used to test if a pattern appears in a string.  It is used with the binding operator (“ =~ ”) to see whether a variable contains a particular pattern.

34. 34 Possible Values of $name

35. 35 Using Character Class Matching any one in a set of characters enclosed within square brackets  foo[bc]ar will match foobar and foocar Ranges can be expressed inside of a character class by using a dash between two characters  [a-g] is equal to [abcdefg]  [0-9] is equal to any digit  [a-zA-Z] Negative character class: use the caret (^) symbol as the first thing in the character class  a[^bc]d, [^0-9]

36. 36 Other Delimiters? Slash (“/”) is most common match pattern  Others are possible, For example, both use valid match operator syntax:  if ( $name =~ m!Dave! ) {  if ( $name =~ m ) { The reverse binding operator test if pattern is NOT found: if ( $color !~ m/blue/ ) { Demo http://www.people.cs.uchicago.edu/~wfreis/regex/regex_match.pl

37. 37 The Substitution Operator Similar to the match operator but also enables you to change the matched string.  Use with the binding operator (“ =~ ”) to test whether a variable contains a pattern

38. 38 How It Works Substitutes the first occurrence of the search pattern for the change pattern in the string variable. For example, the following changes the first occurrence of t to T : $name = “tom turtle”; $name =~ s/t/T/; print “Name=$name”; The output of this code would be Name=Tom turtle

39. 39 Changing All Occurrences You can place a g (for global substitution) at the end of the substitution expression to change all occurrences of the target pattern string in the search string. For example,  $name = “tom turtle”;  $name =~ s/t/T/g;  print “Name=$name”; The output of this code would be  Name= Tom TurTle Demo http://www.people.cs.uchicago.edu/~wfreis/regex/regex_sub.pl http://www.people.cs.uchicago.edu/~wfreis/regex/regex_sub.pl

40. 40 Using Translate A similar function is called tr (for “translate”). Useful for translating characters from uppercase to lowercase, and vice versa.  The tr function allows you to specify a range of characters to translate from and a range of characters to translate to. : $name="smokeY"; $name =~ tr/[a-z]/[A-Z]/; print "name=$name"; Would output the following Name=SMOKEY

41. 41 A Full Pattern Matching Example 1. #!/usr/local/bin/perl 2. use CGI ':standard'; 3. print header, start_html('Command Search'); 4. @PartNums=( 'XX1234', 'XX1892', 'XX9510'); 5. $com=param('command'); 6. $prod=param('uprod'); 7. if ($com eq "ORDER" || $com eq "RETURN") { 8. $prod =~ s/xx/XX/g; # switch xx to XX 9. if ($prod =~ /XX/ ) { 10. foreach $item ( @PartNums ) { 11. if ( $item eq $prod ) { 12. print "VALIDATED command=$com prodnum=$prod"; 13. $found = 1; 14. } 15. } 16. if ( $found != 1 ) { 17. print br,"Sorry Prod Num=$prod NOT FOUND"; 18. } 19. } else { 20. print br, "Sorry that prod num prodnum=$prod looks wrong"; 21. } 22. } else { 23. print br, "Invalid command=$com did not receive ORDER or RETURN"; 24. } 25. print end_html;

42. 42 Would Output The Following...

43. 43 Using Regular Expressions regular expressions to enable programs to match patterns more completely.  They actually make up a small language of special matching operators that can be employed to enhance the Perl string pattern matching.

44. 44 The Alternation Operator Alternation operator looks for alternative strings for matching within a pattern.  (That is, you use it to indicate that the program should match one pattern OR the other). The following shows a match statement using the alternation operator (left) and some possible matches based on the contents of $address (right); this pattern matches either com or edu.

45. 45 Example Alternation Operator

46. 46 Parenthesis For Groupings You use parentheses within regular expressions to specify groupings. For example, the following matches a $name value of Dave or David.

47. 47 Special Character Classes Perl has a special set of character classes for short hand pattern matching For example consider these two statements if ( $name =~ m/ / ) { will match $name with embedded space char if ($name =~ m/\s/ ) { will match $name with embedded space, tab, newline

48. 48 Special Character Classes

49. 49 Special Character Classes - II

50. 50 Special Character Classes - III

51. 51 Setting Specific Patterns w/ Quantifiers Character quantifiers let you look for very specific patterns For example, use the dollar sign (“ $ ”) to to match if a string ends with a specified pattern. if ($ Name =~ /Jones$/ ) { Matches “John Jones” but not “Jones is here” would not. Also, “The guilty party is Jones” would matches.

52. 52 Selected Perl Character Quantifiers I

53. 53 Selected Perl Character Quantifiers II

54. 54 Selected Perl Character Quantifiers III

55. 55 Match the Special Characters Themselves Use a back slash before the special character  \^, \$, \., \?, \(, \), \+, \*,\\,\/ etc  Examples Will a\??bc matches abc, a?bc Will a\++bc matches a+bc, a++bc

56. 56 Building Regular Expressions That Work Regular expressions are very powerful— but they can also be virtually unreadable.  When building one, tart with a simple regular expression and then refine it incrementally. Build a piece and then test  The following example will build a regular expression for a date checker dd/mm/yyyy format (for example, 05/05/2002 but not 5/12/01 ).

57. 57 1. Determine the precise field rules. - What is valid input and what is not valid input?  E.g., For a date field, think through the valid and invalid rules for the field.  You might allow 09/09/2002 but not 9/9/2002 or Sep/9/2002.  Work through several examples as follows: Building Regular Expressions That Work

58. 58 Work through several examples

59. 59 Building Regular Expressions that Work 2. Get form and form-handling programs working  Build a sending form the input field  Build the receiving program that accepts the field.  For example, a first cut receiving program: $date = param(‘udate’); if ( $date =~ m/.+/ ) { print ‘Valid date=’, $date; } else { print ‘Invalid date=’, $date; } Any Sequence of characters

60. 60 Building Regular Expressions that Work 3. Start with the most specific term possible.  For example, slashes must always separate two characters (for the month), followed by two more characters (for the day), followed by four characters (for the year). if ( $date =~ m{../../....} ) { Any 2 characters Any 2 characters Any 4 characters

61. 61 Building Regular Expressions that Work 4. Anchor and refine. (Use ^ and $ when possible)  if ( $date =~ m{^\d\d/\d\d/\d\d\d\d$} ) { Starts with 2 digits in middle Ends with 4 digits

62. 62 Building Regular Expressions that Work 5. Get more specific if possible.  The first digit of the month can be only 0, 1, 2 or 3. For example, 05/55/2002 is clearly an illegal date.  Only years from this century are allowed. Because we don’t care about dates like 05/05/1999 or 05/05/3003.

63. 63 Add these rules below if ( $date =~ m{^\d\d/[0-3]\d/2\d\d\d$} ) {  Now the regular expression recognizes input like 09/99/2001 and 05/05/4000 as illegal. Year starts with a “2” Month starts with a “0-3” Building Regular Expressions that Work

64. 64 Tip: Regular Expression Special Variables Perl regexs set several special scalar variables:  $& will be equal to the first matching text  $` will be the text before the match, and  $’ will be the text after the first match. $name='*****Marty'; if ( $name =~ m/\w/ ) { print "got match at=$& "; print "B4=$` after=$'"; } else { print "Not match"; } would output: got match at=M B4=***** after=arty

65. 65 Full Example Program 1. #!/usr/local/bin/perl 2. use CGI ':standard'; 3. print header, start_html('Date Check'); 4. $date=param('udate'); 5. if ($date =~ m{^\d\d/[0-3]\d/2\d\d\d$}){ 6. print 'Valid date=', $date; 7. } else { 8. print 'Invalid date=', $date; 9. } 10. print end_html;

66. 66 Would Output The Following...