1. 240-491 Adv. UNIX:Perl/81 Advanced UNIX v Objectives of these slides: –introduce Perl (version 4.0-5.0) –mostly based on Chapter 1, Learning Perl 240-491 Special Topics in Comp. Eng. 1 Semester 2, 2000-2001 8. Introduction to Perl

2. 240-491 Adv. UNIX:Perl/82 Overview 1. Starting with Perl 2. I/O 3. If-else 4. while 5. Arrays (lists) continued

3. 240-491 Adv. UNIX:Perl/83 6. Handling Varying Input 7. Subroutines 8. File Processing 9. Reporting 10. Using the UNIX DBM Database 11. The Web and Perl

4. 240-491 Adv. UNIX:Perl/84 1. Starting with Perl v Practical Extraction and Report Language –Larry Wall, from 1987 v Features combine shell, awk, many UNIX tools, C libraries (all coded in Perl). v Emphasis on string manipulation, table generation, REs, excellent libraries –especially good Internet/Web libraries continued

5. 240-491 Adv. UNIX:Perl/85 v On many platforms; free  Many man and info pages.  Many newsgroups (e.g. comp.lang.perl ) v Perl v.5 –OOP, more REs, modules –backward compatible with v.4 –check your version by typing: $ perl -v

6. 240-491 Adv. UNIX:Perl/86 1.1. Books v Learning Perl Randal L. Schwartz & Tom Christiansen O'Reilly, 1997, 2nd ed. v Programming Perl Larry Wall & Randal L. Schwartz O'Reilly, 2000, 3rd ed. –not easy to read, more of a reference text In our library

7. 240-491 Adv. UNIX:Perl/87 1.2. Web Resources v The main Perl site: http://www.perl.com –current Perl version, documentation, libraries (CPAN), FAQs, tools and utilities, features –reference sections on: u applications (e.g. games, graphics), Web, networking, system admin, etc. continued

8. 240-491 Adv. UNIX:Perl/88 v Resources about Perl –e.g. tutorials, manuals, Web-related, recommended books, example scripts –http://hem.fyristorg.com/gumby/ computing/perl.html v A beginner's Guide to Perl –http://home.bluemarble.net/~scotty/Perl/

9. 240-491 Adv. UNIX:Perl/89 1.3. Hello World (hw.pl) $ cat hw.pl #!/usr/bin/perl print "Hello, world!\n"; $ chmod u+x hw.pl $ hw.pl Hello, world! Or: $ perl hw.pl

10. 240-491 Adv. UNIX:Perl/810 2. I/O #!/usr/bin/perl # hm.pl print "What is your name? "; $name = ; chop($name); print "Hello, $name!\n"; $ perl hm.pl What is your name? andrew Hello, andrew! $

11. 240-491 Adv. UNIX:Perl/811 3. if-else #!/usr/bin/perl # ch.pl print "What is your name? "; $name = ; chop($name); if ($name eq "Randal") { print "Hello, Sir Randal!\n"; } else { print "Hello, $name!\n"; # ordinary }

12. 240-491 Adv. UNIX:Perl/812 4. while #!/usr/bin/perl # sword.pl $secretword = "llama"; print "What is your name? "; $name = ; chop($name); if ($name eq "Randal") { print "Hello, Sir Randal!\n"; } : continued Not very secret Guess the secret word

13. 240-491 Adv. UNIX:Perl/813 else { print "Hello, $name!\n"; # ordinary print "Secret word? "; $guess = ; chop($guess); while ($guess ne $secretword) { print "Wrong, try again: "; $guess = ; chop($guess); } } $ perl sword.pl What is your name? andrew Hello, andrew! Secret word? foobar Wrong, try again: llama $

14. 240-491 Adv. UNIX:Perl/814 5. Arrays (lists) @words = ("camel", "llama", "oyster"); $words[0] is "camel"  List variables begin with @ –e.g. @words –$words[0] is a value in the list, so uses $

15. 240-491 Adv. UNIX:Perl/815 swords.pl #!/usr/bin/perl @words = ("camel", "llama", "oyster"); print "What is your name? "; $name = ; chop($name); if ($name eq "Randal") { print "Hello, Sir Randal!\n"; } : continued Guess the secret word (v.2) An array of secret words

16. 240-491 Adv. UNIX:Perl/816 else { print "Hello, $name!\n"; # ordinary print "Secret word? "; $guess = ; chop($guess); $i = 0; # index into @words $correct = "maybe"; # dummy value : continued

17. 240-491 Adv. UNIX:Perl/817 while ($correct eq "maybe") { if ($words[$i] eq $guess) { # right? $correct = "yes"; } elsif ($i ; chop($guess); $i = 0; # start words again } } } The user can win by guessing any of the secret words.

18. 240-491 Adv. UNIX:Perl/818 Usage $ perl swords.pl What is your name? andrew Hello, andrew! Secret word? foo Wrong, try again: bar Wrong, try again: llama $ one of the secret words

19. 240-491 Adv. UNIX:Perl/819 5.1. Associative Arrays (tables)  PersonSecret Word fredcamel barneyllama bettyoyster wilmaoyster v %words = ("fred", "camel", "barney","llama", "betty", "oyster", "wilma", "oyster");  $words{"betty"} is "oyster"  Associative array variables begin with % –e.g. %words –$words{"betty"} is a value in the list, so uses $

20. 240-491 Adv. UNIX:Perl/820 swords2.pl #!/usr/bin/perl %words = ("fred","camel","barney","llama", "betty", "oyster", "wilma", "oyster"); print "What is your name? "; $name = ; chop($name); if ($name eq "Randal") { print "Hello, Sir Randal!\n"; } : continued Each user has their own secret word

21. 240-491 Adv. UNIX:Perl/821 else { print "Hello, $name!\n"; # ordinary $secretword = $words{$name}; # get secret word print "Secret word? "; $guess = ; chop($guess); while ($guess ne $secretword) { print "Wrong, try again: "; $guess = ; chop($guess); } }

22. 240-491 Adv. UNIX:Perl/822 Usage $ perl swords2.pl What is your name? barney Hello, barney! Secret word? oyster Wrong, try again: foobar Wrong, try again: llama $ barney's secret word!

23. 240-491 Adv. UNIX:Perl/823 5.2. Adding a Default Secret Word... $secretword = $words{$name}; if ($secretword eq "") { # not found! $secretword = "groucho"; } print "Secret word? ";...

24. 240-491 Adv. UNIX:Perl/824 6. Handling Varying Input  sword.pl (and others) treats "Randal" in a special way.  But: "Randal L. Schwartz" or "randal" is treated like other users.

25. 240-491 Adv. UNIX:Perl/825 6.1. Replace eq by =~ and RE... if ($name =~ /^Randal/) { # it matches, do something } else { # no match, do something else }... REs are one of the big advantages of Perl

26. 240-491 Adv. UNIX:Perl/826 6.2. Extended REs  Check for a word boundary with \b  Ignore case with the i option (after the last / )

27. 240-491 Adv. UNIX:Perl/827... if ($name =~ /^randal\b/i) { # matches "Randal...", "ranDAL..." print "Hello, Sir Randal!\n"; } else { # no match, do something else }... Part of final1.pl

28. 240-491 Adv. UNIX:Perl/828 6.3. Substitution and Translation  Substitution: find and replace (like s/../../ in vi ).  Translation: map characters to others (like tr )

29. 240-491 Adv. UNIX:Perl/829... print "What is your name? "; $name = ; chop($name); $name =~ s/\W.*//; # Get rid of everything # after first word. $name =~ tr/A-Z/a-z/; # Make lowercase. if ($name eq "randal") { print "Hello, Sir Randal!\n"; }... e.g. Dr.Andrew Davison becomes dr

30. 240-491 Adv. UNIX:Perl/830 7. Subroutines sub good_word { # used in final1.pl local($somename, $someguess) = @_; # input arguments $somename =~ s/\W.*//; $somename =~ tr/A-Z/a-z/; if ($somename eq "randal") 1 # return true } elsif (($words{$somename} || "groucho") eq $someguess) { 1; } else { 0; # return false } } # end of subroutine Not a good feature

31. 240-491 Adv. UNIX:Perl/831 Called in final1.pl:... print "Secret word? "; $guess = ; chop($guess); while (! &good_word($name, $guess) ) { print "Wrong, try again: "; $guess = ; chop($guess); } } Don't forget the &

32. 240-491 Adv. UNIX:Perl/832 8. File Processing  Read words from the "wordslist" file: sub init_words { # used inside final1.pl open(WORDSLIST, "wordslist"); while ($name = ) { chop($name); $word = ; chop($word); $words{$name} = $word; } close(WORDSLIST); }

33. 240-491 Adv. UNIX:Perl/833 wordslist Format fred camel barney llama betty oyster wilma oyster The code treats the file as four pairs: name secret-word

34. 240-491 Adv. UNIX:Perl/834 Called in final1.pl: #!/usr/bin/perl &init_words; print "What is your name? "; $name = ; chop($name);...

35. 240-491 Adv. UNIX:Perl/835 8.1. File Tests sub init_words { # second version open(WORDSLIST, "wordslist"); if (-M WORDSLIST > 7) { die "Sorry, wordslist is too old"; } while ($name = ) { chop($name); $word = ; chop($word); $words{$name} = $word; } close(WORDSLIST); } -M is the file's modification date

36. 240-491 Adv. UNIX:Perl/836 8.2. The open() Command  Similar to popen() in C –the command argument is executed and can be written to via an output stream  In good_word :... open(MAIL, "| mail dandrew@ratree.psu.ac.th"); print MAIL "bad news $somename guessed $someguess\n"; close(MAIL);... mail MAIL

37. 240-491 Adv. UNIX:Perl/837 8.3. Filename Expansion v Assume several files of secret words: –ad.secret, yuk.secret  Modify init_words() to read in all of these –but only if the files are less than 7 days old  Store in the %words associative array Also called filename globbing

38. 240-491 Adv. UNIX:Perl/838 sub init_words { # third version while ($filename = ) { open(WORDSLIST, $filename); if (-M WORDSLIST ) { chop($name); $word = ; chop($word); $words($name) = $word; } } close(WORDSLIST); } } filename expansion

39. 240-491 Adv. UNIX:Perl/839 8.4. The Password File v A typical password line: ad:x:42497:100:Dr.Andrew DAVISON,,,: /home/ad:/bin/bash –the user's details are in the 5th field u the GECOS field  $< always contains the user's ID number (UID)

40. 240-491 Adv. UNIX:Perl/840 #!/usr/bin/perl # final1.pl &init_words; @password = getpwuid( $< ); # get password data using the UID $name = $password[6]; # get the GECOS field (6th in Perl!) $name =~ s/,.*//; # throw away everything # after 1st comma if ($name =~ /^randal\b/i) { print "Hello, Sir Randal!\n"; }... There is a getpwuid() in C which does the same thing

41. 240-491 Adv. UNIX:Perl/841 9. Reporting  secret.pl : –list the *.secret files in the format: filename user-name secret-word

42. 240-491 Adv. UNIX:Perl/842 secret.pl #!/usr/bin/perl while ($filename = ) { open(WORDSLIST, $filename); if (-M WORDSLIST ) { chop($name); $word = ; chop($word); write; # invoke STDOUT format } } close(WORDSLIST); } continued

43. 240-491 Adv. UNIX:Perl/843 STDOUT Format Definition format STDOUT = @<<<<<<<< @<<<<<<<<< @<<<<<<<<<<< $filename, $name, $word. field definition line field value line continued

44. 240-491 Adv. UNIX:Perl/844 Top of Page Format v Used every 60 lines by default: format STDOUT_TOP = Page @<< # field definition line $% # number of pages printed Filename Name Word =========== ======= ========.

45. 240-491 Adv. UNIX:Perl/845 Output $ secret.pl Page 1 Filename Name Word ============= ======= ======== ad.secret andrew f1 ad.secret paul f2 ad.secret dr foobar yuk.secret jim c1 yuk.secret bob c2 $ The *.secret files may need to be 'touched'.

46. 240-491 Adv. UNIX:Perl/846 10. Using the UNIX DBM Database  In final1.pl, the time of the most recent correct guess is stored in %last_good $last_good{$name} = time;  Problem: %last_good will be lost when the script terminates.  Solution: store %last_good in a DBM database.

47. 240-491 Adv. UNIX:Perl/847... dbmopen(%last_good, "lastdb", 0666); $last_good{$name} = time; dbmclose(%last_good);... returns time in seconds since 1/1/1970 rw for everyone Revised final1.pl

48. 240-491 Adv. UNIX:Perl/848 Using final1.pl v $ final1.pl Hello, Dr.Andrew DAVISON! What is the secret word? bingo Wrong, try again. What is the secret word? foobar $ foobar is the secret word for dr (see slide 45)

49. 240-491 Adv. UNIX:Perl/849 List the Last Guessses (ex_last.pl) #!/usr/bin/perl dbmopen(%last_good,"lastdb",0666); foreach $name (sort keys(%last_good)) { $when = $last_good{$name}; $hours = (time - $when) / 3600; # compute hours ago write; # use STDOUT format } format STDOUT = User @<<<<<<<<<<<<: last correct guess was @<<< hours ago. $name, $hours.

50. 240-491 Adv. UNIX:Perl/850 Output $ ex_last.pl User Andrew DAVIS: last correct guess was 2066 hours ago. User Dr.Andrew DA: last correct guess was 0.07 hours ago. $ last year, when my user details were different

51. 240-491 Adv. UNIX:Perl/851 11. The Web and Perl v Perl supports sockets for building Web clients and servers. v A easier and faster way is to use the LWP modules –designed for Web (client) tasks –hides low-level networking issues

52. 240-491 Adv. UNIX:Perl/852 Main LWP Features v Object oriented notation v Supports all of the HTTP protocol –e.g. GET, POST, HEAD, redirection v Handles authentication v Deals with proxy servers v Can use timed action v Has a framework for writing Web 'robots'

53. 240-491 Adv. UNIX:Perl/853 11.1. Retrieve a Web Page v !/usr/bin/perl # geturl.pl use LWP::Simple; # use module LWP/Simple.pm print ( get $ARGV[0] ); $ geturl.pl http://fivedots.coe.psu.ac.th/~ad Andrew Davison's Home Page at PSU :

54. 240-491 Adv. UNIX:Perl/854 11.2. Parse a Web Page v #!/usr/bin/perl # showurl.pl use LWP::Simple; use HTML::Parse; print parse_html( get $ARGV[0] )->format; $ showurl.pl http://fivedots.coe.psu.ac.th/~ad Can't locate HTML/Parse.pm in... $ call format method on the result

55. 240-491 Adv. UNIX:Perl/855 11.3. Extract Links v #!/usr/bin/perl # showlinks.pl use LWP::Simple; use HTML::Parse; use HTML::Element; $phtml = parse_html( get $ARGV[0] ); for ( @{ $phtml->extract_links() }) { $link = $_->[0]; # access array print "$link\n"; } $ showlinks.pl http://fivedots.coe.psu.ac.th/~ad call extract_links method on the $phtml object

56. 240-491 Adv. UNIX:Perl/856 11.4. Deal with Proxies v #!/usr/bin/perl # hcat_proxy.pl use LWP::UserAgent; use HTTP::Request; use HTTP::Response; my $ua = new LWP::UserAgent; # local object $ua->proxy('http', 'http://cache.psu.ac.th:8080/'); $ua->no_proxy('coe.p-net'); my $request = new HTTP::Request('GET', $ARGV[0]); my $response = $ua->request($request); if ($response->is_success) { print $response->content; } else { print $response->error_as_HTML; }

57. 240-491 Adv. UNIX:Perl/857 Use (on fivedots) v $ perl hcat_proxy.pl http://www.perl.com www.perl.com - Beginners Intro to Perl - Part 3 www.perl.com - Beginners Intro to Perl - Part 3 <meta name="robots" :

58. 240-491 Adv. UNIX:Perl/858 11.5. More Information on LWP  On calvin : –man LWP –pod2text < /usr/lib/perl5/lwpcook.pod v Perl modules (including LWP) are in: /usr/lib/perl5 /usr/lib/perl5 v Related modules: –HTML, HTTP, MIME, URI, WWW continued

59. 240-491 Adv. UNIX:Perl/859 v Web Client Programming with Perl Clinton Wong O'Reilly, 1997 I have a copy