Introduction to Perl Part II By: Dipak Balabantaray

Some Useful PERL FUNCTIONS System Syntax: system PROGRAM, LIST system PROGRAM Definition And Usages: Executes the command specified by PROGRAM, passing LIST as arguments to the command. The return value is the exit status of the program as returned by the wait function. To obtain the actual exit value, divide by 256 Return Values: Exit status of program as returned by wait Example: Try out following example #!/usr/bin/perl -w system("ls -F /var > /tmp/t.tmp"); It will produce following result: A file in /tmp direcory, check it out.

EXEC Syntax exec EXPR LIST exec LIST Definition and Usage Executes a system command (directly, not within a shell) and never returns to the calling script, except if the command specified does not exist and has been called directly, instead of indirectly through a shell. The operation works as follows: If there is only one scalar argument that contains no shell metacharacters, then the argument is converted into a list and the command is executed directly, without a shell. If there is only one scalar argument that contains shell metacharacters, then the argument is executed through the standard shell, usually /bin/sh on Unix. If LIST is more than one argument, or an array with more than one value, then the command is executed directly without the use of a shell. Return Value 0 only if the command specified cannot be executed Example Following are the usage... exec '/bin/echo', 'Your arguments are: exec "sort $outfile | uniq"; Another example: exec {'/bin/csh'} '-sh';# pretend it's a login shell

DIE Function: Syntax die LIST Definition and Usage Prints the value of LIST to STDERR and calls exit with the error value contained in $!. Return Value Nothing Example Following are the usage... die "Can't cd to spool: $!\n" unless chdir '/usr/spool/news'; or chdir '/usr/spool/news' or die "Can't cd to spool: $!\n" or open( FILEH, ">/tmp/test" ) || die( "Can't open file test $!\n" );

EVAL Syntax eval EXPR eval BLOCK Definition and Usage Evaluates EXPR at execution time as if EXPR were a separate Perl script. This allows you to use a separate, perhaps user-supplied, piece of Perl script within your program. An eval EXPR statement is evaluated separately each time the function is called. The second form evaluates BLOCK when the rest of the script is parsed (before execution). Return Value Value of last evaluated statement in EXPR or BLOCK Example Following are the usage... # make divide-by-zero nonfatal eval { $answer = $a / $b; }; warn if # __DIE__ hooks may modify error messages { local $SIG{'__DIE__'} = sub { (my $x = $_[0]) =~ s/foo/bar/g; die $x }; eval { die "foo lives here" }; print if # prints "bar lives here" }

EXIT Function: Syntax exit EXPR exit Definition and Usage Evaluates EXPR, exits the Perl interpreter, and returns the value as the exit value.Always runs all END{} blocks defined in the script (and imported packages) before exiting. If EXPR is omitted, then the interpreter exits with a value of 0. Should not be used to exit from a subroutine; either use eval and die or use return. Return Value Nothing. Example Following are the usage... $ans = ; exit 0 if $ans =~ /^[Xx]/;

WARN Function: Syntax warn LIST Definition and Usage Prints the value of LIST to STDERR. Basically the same as the die function except that no call is made to the exit and no exception is raised within an eval statement. This can be useful to raise an error without causing the script to terminate prematurely. If the variable contains a value (from a previous eval call) and LIST is empty, then the value of is printed with.\t.caught. appended to the end. If both and LIST are empty, then.Warning: Something.s wrong. is printed. Return Value Nothing Example #!/usr/bin/perl -w warn("Unable to calculate value, using defaults instead.\n"); It will produce following results: Unable to calculate value, using defaults instead

SLEEP Function: Syntax sleep EXPR sleep Definition and Usage Pauses the script for EXPR seconds, or forever if EXPR is not specified. Returns the number of seconds actually slept. Can be interrupted by a signal handler, but you should avoid using sleep with alarm, since many systems use alarm for the sleep implementation. Return Value Integer, number of seconds actually slept Example Try out following example: You will understand the functionality of sleep. #!/usr/bin/perl $num = 5; while($num--){ sleep(1); }

WAIT function: Syntax wait Definition and Usage Waits for a child process to terminate, returning the process ID of the deceased process. The exit status of the process is contained in $?. Return Value -1 if there are no child processes Process ID of deceased process

Passing Arguments To Your Program

Command Line Arguments Command line arguments in Perl are extremely is the array that holds all arguments passed in from the command line. Example:./prog.pl arg1 arg2 would contain ('arg1', ‘arg2', 'arg3’) $#ARGV returns the number of command line arguments that have been passed. Remember $#array is the size of the array!

Reading/Writing Files

File Handlers Opening a File: open (SRC, “my_file.txt”); Reading from a File $line = ; # reads upto a newline character Closing a File close (SRC);

File Handlers cont... Opening a file for output: open (DST, “>my_file.txt”); Opening a file for appending open (DST, “>>my_file.txt”); Writing to a file: print DST “Printing my first line.\n”; Safeguarding against opening a non existent file open (SRC, “file.txt”) || die “Could not open file.\n”;

File Test Operators Check to see if a file exists: if ( -e “file.txt”) { # The file exists! } Other file test operators: -rreadable -xexecutable -dis a directory -Tis a text file

Quick Program with File Handles Program to copy a file to a destination file #!/usr/bin/perl -w open(SRC, “file.txt”) || die “Could not open source file.\n”; open(DST, “>newfile.txt”); while ( $line = ) { print DST $line; } close SRC; close DST;

Some Default File Handles STDIN : Standard Input $line = ; # takes input from stdin STDOUT : Standard output print STDOUT “File handling in Perl is sweet!\n”; STDERR : Standard Error print STDERR “Error!!\n”;

The <> File Handle The “empty” file handle takes the command line file(s) or STDIN; $line = <>; If program is run./prog.pl file.txt, this will automatically open file.txt and read the first line. If program is run./prog.pl file1.txt file2.txt, this will first read in file1.txt and then file2.txt... you will not know when one ends and the other begins.

The <> File Handle cont... If program is run./prog.pl, the program will wait for you to enter text at the prompt, and will continue until you enter the EOF character CTRL-D in UNIX

Example Program with STDIN Suppose you want to determine if you are one of the three stooges #!/usr/local/bin/perl %stooges = (larry => 1, moe => 1, curly => 1 ); print “Enter your name: ? “; $name = ; chomp $name; if($stooges{ lc($name) }) { print “You are one of the Three Stooges!!\n”; } else { print “Sorry, you are not a Stooge!!\n”; }

Combining File Content Given The two Following Files: File1.txt And File2.txt a b c Write a program that takes the two files as arguments and outputs a third file that looks like: File3.txt 1 a 2 b 3 Tip:./mix_files File1.txt File2.txt File3.txt

Combining File Content #! /usr/bin/perl open (F, “$ARGV[0]); open (G, “$ARGV[1]); open (H, “>$ARGV[2]); while ( defined (F) && defined (G) && ($l1= ) && ($l2= )) { print H “$l1$l2”; } close (F); close (G); close (H);

How to read a binary file open(FILEHANDLE, ">text.dat") or die "an error occured: $!" binmode FILEHANDLE; $line= ; while ($line) { print “$line\n”; } close(FILEHANDLE);

XML File Reading: Once you've got the module installed, create the following XML file and call it "data.xml": Pradeep 23 M Programming And then type out the following Perl script, which parses it using the XML::Simple module: #!/usr/bin/perl # use module use XML::Simple; use Data::Dumper; # create object $xml = new XML::Simple; # read XML file $data = $xml->XMLin("data.xml"); # print output print Dumper($data);readprint

When you run this script, here's what you'll see $VAR1 = { 'department' => 'Programming', 'name' => 'Pradeep', 'sex' => 'M', 'age' => '23' }; Data::Dumper Given a list of scalars or reference variables, writes out their contents in perl syntax. The references can also be objects. The content of each variable is output in a single Perl statement. Handles self-referential structures correctly

How to read a SCV file: File prospects.csv "Name","Address","Floors","Donated last year","Contact" "Charlotte French Cakes","1179 Glenhuntly Rd",1,"Y","John" "Glenhuntly Pharmacy","1181 Glenhuntly Rd",1,"Y","Paul" "Dick Wicks Magnetic Pain Relief"," Glenhuntly Rd",1,"Y","George" "Gilmour's Shoes","1187 Glenhuntly Rd",1,"Y","Ringo“. #!/usr/bin/perl use strict; use warnings; use Text::CSV; my $file = 'prospects.csv'; my $csv = Text::CSV->new(); open (CSV, "<", $file) or die $!; while ( ) { if ($csv->parse($_)) { = $csv->fields(); print } else { my $err = $csv->error_input; print "Failed to parse line: $err"; } close CSV;

Continue….. Running the code produces the following output: Name Address Floors Donated last year Contact Charlotte French Cakes 1179 Glenhuntly Rd 1 Y John Glenhuntly Pharmacy 1181 Glenhuntly Rd 1 Y Paul Dick Wicks Magnetic Pain Relief Glenhuntly Rd 1 Y George Gilmour's Shoes 1187 Glenhuntly Rd 1 Y Ringo Where The Text::CSV module provides functions for both parsing and producing CSV data. $status = $csv->parse ($line); This object function decomposes a CSV string into fields, returning success or failure. Failure can result from a lack of argument or the given CSV string is improperly = $csv->fields (); This object function returns the input to combine () or the resultant decomposed fields of successful, whichever was called more recently.

Read a directory in Perl: #!/usr/bin/perl -w $dir = "."; opendir(INDIR, $dir) or die "Directory Not = readdir(INDIR); closedir(INDIR); foreach $file { print $file, "\n"; }

Chomp and Chop Chomp : function that deletes a trailing newline from the end of a string. $line = “this is the first line of text\n”; chomp $line; # removes the new line character print $line; # prints “this is the first line of # text” without returning Chop : function that chops off the last character of a string. $line = “this is the first line of text”; chop $line; print $line; #prints “this is the first line of tex”

Matching Regular Expressions

Regular Expressions What are Regular Expressions.. a few definitions. Specifies a class of strings that belong to the formal / regular languages defined by regular expressions In other words, a formula for matching strings that follow a specified pattern. Some things you can do with regular expressions Parse the text Add and/or replace subsections of text Remove pieces of the text

Regular Expressions cont.. A regular expression characterizes a regular language Examples in UNIX: ls *.c Lists all the files in the current directory that are postfixed '.c' ls *.txt Lists all the files in the current directory that are postfixed '.txt'

Simple Example for... ? Clarity In the simplest form, a regular expression is a string of characters that you are looking for We want to find all the words that contain the string 'ing' in our text. The regular expression we would use : /ing/

The Match Operator What would are program then look like: if($word=~m/ing/) { print “$word\n”;}

Exercise: Download any text you wish from the internet and count all the words in “ing” it contains… wget “

Exercise: #!/usr/local/bin/perl while(<>) { = split/ /; foreach { if($word=~m/ing/) { print “$word\n”;$ing++; } } print “$ing Words in ing\n”;

Regular Expressions Types Regular expressions are composed of two types of characters: Literals Normal text characters Like what we saw in the previous program ( /ing/ ) Metacharacters special characters Add a great deal of flexibility to your search

Metacharacters Match more than just characters Match line position ^start of a line( carat ) $end of a line( dollar sign ) Match any characters in a list : [... ] Example : /[Bb]ridget/matches Bridget or bridget /Mc[Ii]nnes/matches McInnes or Mcinnes

Our Simple Example Revisited Now suppose we only want to match words that end in 'ing' rather than just contain 'ing'. How would we change are regular expressions to accomplish this: Previous Regular Expression: $word =~m/ ing / New Regular Expression: $word=~m/ ing$ /

Ranges of Regular Expressions Ranges can be specified in Regular Expressions Valid Ranges [A-Z]Upper Case Roman Alphabet [a-z]Lower Case Roman Alphabet [A-Za-z]Upper or Lower Case Roman Alphabet [A-F]Upper Case A through F Roman Characters [A-z]Valid but be careful Invalid Ranges [a-Z]Not Valid [F-A]Not Valid

Ranges cont... Ranges of Digits can also be specified [0-9]Valid [9-0]Invalid Negating Ranges / [^0-9] / Match anything except a digit / [^a] / Match anything except an a / ^[^A-Z] / Match anything that starts with something other than a single upper case letter First ^ :start of line Second ^ :negation

Our Simple Example Again Now suppose we want to create a list of all the words in our text that do not end in 'ing' How would we change are regular expressions to accomplish this: Previous Regular Expression: $word =~m/ ing$ / New Regular Expression: !($word=~m/ (ing)$ /)

Matching Interogations $string=~/([^.?]+\?)/ $string=~/[.?]([A-Z0-9][^.?]+\?)/ $string=~/([\w\s]+\?)/

Removing HTML Tags $string=~s/\ ]+\>/ /g g: substitute EVERY instance

Literal Metacharacters Suppose that you actually want to look for all strings that equal ‘$' in your text Use the \ symbol / \$ /Regular expression to search for What does the following Regular Expressions Match? / [ ABCDEFGHIJKLMNOP$] \$/ / [ A-P$ ] \$ / Matches any line that contains ( A-P or $) followed by $

Patterns provided in Perl Some Patterns \d[ 0 – 9 ] \w[a – z A – Z 0 – 9_] \s[ \r \t \n \f ](white space pattern) \D[^ 0 - 9] \W[^ a – z A – Z 0 – 9_] \S[^ \r \t \n \f] Example :( 19\d\d ) Looks for any year in the 1900's

Using Patterns in our Example Commonly words are not separated by just a single space but by tabs, returns, ect... Let's modify our split function to incorporate multiple white space #!/usr/local/bin/perl while(<>) { = split/\s+/, $_; foreach { if($word=~m/ing$/) { print “$word\n”; }

Word Boundary Metacharacter Regular Expression to match the start or the end of a 'word' : \b Examples: / Jeff\b /Match Jeff but not Jefferson / Carol\b /Match Carol but not Caroline / Rollin\b /Match Rollin but not Rolling /\bform /Match form or formation but not Information /\bform\b/Match form but neither information nor formation

DOT Metacharacter The DOT Metacharacter, '.' symbolizes any character except a new line / b. bble/ Would possibly return : bobble, babble, bubble /. oat/ Would possibly return : boat, coat, goat Note: remember '.*' usually means a bunch of anything, this can be handy but also can have hidden ramifications.

PIPE Metacharacter The PIPE Metacharacter is used for alternation / Bridget (Thomson | McInnes) / Match Bridget Thomson or Bridget McInnes but NOT Bridget Thomson McInnes / B | bridget / Match B or bridget / ^( B | b ) ridget / Match Bridget or bridget at the beginning of a line

Our Simple Example Now with our example, suppose that we want to not only get all words that end in 'ing' but also 'ed'. How would we change are regular expressions to accomplish this: Previous Regular Expression: $word =~m/ ing / New Regular Expression: $word=~m/ (ing|ed)/

The ? Metacharacter The metacharacter, ?, indicates that the character immediately preceding it occurs zero or one time Examples: / worl?ds / Match either 'worlds' or 'words' / m?ethane / Match either 'methane' or 'ethane'

The * Metacharacter The metacharacter, *, indicates that the character immediately preceding it occurs zero or more times Example : / ab*c/ Match 'ac', 'abc', 'abbc', 'abbbc' ect... Matches any string that starts with an a, if possibly followed by a sequence of b's and ends with a c. Sometimes called Kleene's star

Our Simple Example again Now suppose we want to create a list of all the words in our text that end in 'ing' or 'ings' How would we change are regular expressions to accomplish this: Previous Regular Expression: $word =~m/ ing$ / New Regular Expression: $word=~m/ ings?$ /

Exercise For each of the strings (a)--(e), say which of the patterns (i)--(xii) it matches. Where there is a match, what would be the values of $MATCH, $1, $2, etc.? 1)the quick brown fox jumped over the lazy dog 2)The Sea! The Sea! 3)(.+)\s*\1 4) )C:\DOS\PATH\NAME 1)/[a-z]/ 2)/(\W+)/ 3)/\W*/ 4)/^\w+$/ 5)/[^\w+$]/ 6)/\d/ 7)/(.+)\s*\1/ 8)/((.+)\s*\1)/ 9)/(.+)\s*((\1))/ 11)/\DOS/ 12)/\\DOS/ 13)/\\\DOS/

Exercise For each of the strings (a)--(e), say which of the patterns (i)--(xii) it matches. Where there is a match, what would be the values of $MATCH, $1, $2, etc.? 1)the quick brown fox jumped over the lazy dog 1,2,3,5 2)The Sea! The Sea! 1,2,3,5,7,9 3)(.+)\s*\1 1,2,3, 5, 6 4) ,4,6 5)C:\DOS\PATH\NAME 2,3,5,10,11,12 1)/[a-z]/ 1,2,3 2)/(\W+)/ 1,2,3,5 3)/\W*/ 1,2,3,5 4)/^\w+$/ 4 5)/[^\w+$]/ 1,2,3,5 6)/\d/ 3,4 7)/(.+)\s*\1/ 2, 8)/((.+)\s*\1)/ 9)/(.+)\s*((\1))/ 2 10)/\DOS/ 5 11)/\\DOS/ 5 12)/\\\DOS/ 5

Modifying Text With Regular Expressions

Modifying Text Match Up to this point, we have seen attempt to match a given regular expression Example : $variable =~m/ regex / Substitution Takes match one step further : if there is a match, then replace it with the given string Example : $variable =~s/ regex / replacement/ $var =~ s/ Cedric / Notredame /g; $var =~ s/ing/ed /;

Substitution Example Suppose when we find all our words that end in 'ing' we want to replace the 'ing' with 'ed'. #!/usr/local/bin/perl -w while(<>) { chomp = split/ \s+/, $_; foreach { if($word=~s/ing$/ed/) { print “$word\n”; } }

Special Variable Modified by a Match $target=“I have 25 apples”  $target=~/(\d+)/ $& => 25 Copy of text matched by the regex $' =>”I have “ A copy of the target text until the first match $` => “ apples” A copy of the target text after the last match $1, $2, $3, ect $1=25 The text matched by 1st, 2nd, ect., set of parentheses. Note : $0 is not included here $+ A copy of the highest numbered $1, $2, $3, ect..

Our Simple Example once again Now lets revise our program to find all the words that end in 'ing' without splitting our line of text into an array of words #!/usr/local/bin/perl -w while(<>) { chomp $_; if($_=~/([A-Za-z]*ing\b)/g) { print "$&\n"; } }

Example #!/usr/local/bin $exp = ; chomp $exp; if($exp=~/^([A-Za-z+\s]*)\bcrave\b([\sA-Za-z]+)/) { print “$1\n”; print “$2\n”; } Run Program with string : I crave to rule the world! Results: “I “ to rule the world!

Example #!/usr/local/bin $exp = ; chomp $exp; if($exp=~/\bcrave\b/) { print “$`\n”; print “$&\n”; print “$’\n”; } Run Program with string : I crave to rule the world! Results: I crave to rule the world!

Database handling in PERL: Database Driver and Parameter First you need to tell the DBI which database you want to use (PostgreSQL, MySQL, etc.), where to find the database server (host, port, etc.) and the parameters for the connection (database, user, password, etc). This done with a string called "data source name". It starts with the characters dbi:, then the name of the driver, followed by another colon, what follows is passed to the driver's own connect() method to be interpreted as it sees fit. Example: DBI->connect( "dbi:Pg:dbname=database,username,password"); DBI->connect() returns a connection handle. You can get a list of all the available drivers installed on your machine by using the following: DBI->available_drivers(); Then you can invoke the DBI->data_sources() method against one or more of the drivers returned by DBI->available_drivers() to enumerate which data sources are known to the driver. Example: = DBI->available_drivers(); foreach $driver ) { print "driver: $driver \n"; = DBI->data_sources( $driver ); print "\t }

Continue…. Output (on my machine): driver: ExampleP sources: dbi:ExampleP:dir=. driver: Pg sources: dbi:Pg:dbname=dump_test dbi:Pg:dbname=template0 dbi:Pg:dbname=template1 dbi:Pg:dbname=testbase dbi:Pg:dbname=workshop. Database Handle: Connection and Disconnection You connect to a database with the DBI->connect() method: $DB_user = 'me'; $DB_name = 'workshop'; $TABLE_name = 'stud'; $DB_pwd = ''; $dbh = DBI->connect("dbi:Pg:dbname=$DB_name","$DB_user","$DB_pwd"); $dbh->disconnect(); DBI->connect() instantiates the driver and returns a connection handle. disconnect() closes the connection again

Continue….. Error Handling The DBI performs basic automatic error reporting when the PrintError attribute is enabled (default). To disable this feature, set the value to 0 either via the connection handle, or via the attribute hash of the connect() method. $dbh->{PrintError} = 0; # disable $dbh->{PrintError} = 1; # enable Or $dbh = DBI->connect("dbi:Pg:dbname=$DB_name","$DB_user","$DB_pwd", { PrintError => 0 } ) or die "Cannot connect: $DBI::errstr\n"; $DBI::errstr() :is the string containing a description of the error, as provided by the underlying database. Prepare and execute function: Before the statement can be executed, it needs to be prepared for execution: $sth = $dbh->prepare("SELECT * FROM stud"); The prepare() function returns a statement handle (commonly called $sth). Once a statement is prepared, you can execute it: my $rv = $sth->execute; The return value from execute() should be nonzero. If it is 0, you need to deal with errors in an appropriate way: if (!$rv) { handleError($dbh->errstr); } else {...}

Continue….. The reason is Performance. With prepare you can use placeholders instead of literal values. With placeholders statements only needs to be prepared once. The bind values for each row can be given to the execute method each time it is called. By avoiding the need to re-prepare the statement for each row, the application typically runs many times faster. Example: my $sth = $dbh->prepare(q{ INSERT INTO sales (product_code, price) VALUES (?, ?) }) or die $dbh->errstr; while (<>) { chomp; my ($product_code, $price) = split /,/; $sth->execute($product_code, $price) or die $dbh->errstr; } Do() Method: The do() method is fuse of prepare() and execute(). It can only be used for non non-SELECT statement, where you do not need the statement handle to access the results of the query: $rows_affected = $dbh->do( "UPDATE your_table SET foo = foo + 1"); do() returns the number of affected rows. When you are done with the query, you should note that to Perl, so that associated information can be released: $sth->finish;

Continue…… Fetching the Data To fetch the results of a SELECT command a row at a time you can use $sth->fetchrow_array(). It returns a new row at each call or the undefined value when no more data is left (this can be used as loop condition): while ( ($id,$name) = $sth->fetchrow_array() ) { print "$id\t\t $name \n"; } Output: 1 fred 3 tom 2 lisa 5 BoB Alternatively you can use the $sth->rows() function for a loop condition. It returns the number of rows affected by the last row affecting command, or -1 if the number of rows is not known or not available (the rows function is not supported by all database drivers). Example: $rows = $sth->rows(); for ($i = 0; $i < $rows; $i++) { ($id,$name) = $sth->fetchrow_array(); print "$id\t\t $name \n"; }

Complete Example: #!/usr/bin/perl use DBI; $DB_name = 'workshop'; $DB_user = 'me'; $DB_pwd = ''; $dbh = DBI->connect("dbi:Pg:dbname=$DB_name","$DB_user","$DB_pwd"); print "\nConnection error: $DBI::errstr\n\n"; $sth = $dbh->prepare("SELECT * FROM stud"); $sth->execute(); while ( ($id,$name) = $sth->fetchrow_array() ) { print "$id\t\t $name \n"; } $sth->finish(); $dbh->disconnect(); Output: Connection error: 1 fred 3 tom 2 lisa 5 BoB

Thank you