2. From LDAP to Patron Load From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump Michael Doran, Systems Librarian Endeavor Users Group Meeting, Chicago, IL Session 8, Thursday, April 26, 2007

3. doran@uta.edu Michael Doran, Systems Librarian Cataloging Acquisitions Circulation OPAC Michael Doran 123 Elm St. Arlington, TX PATRON BLOCK Integrated library system patron database patron DB

4. doran@uta.edu Michael Doran, Systems Librarian Getting patron data into Voyager Manual data entry via the Circulation client D’oh!

5. doran@uta.edu Michael Doran, Systems Librarian Getting patron data into Voyager Manual data entry via the Circulation client Batch loading via the Pptrnupdt job  Technical User’s Guide 6 : Patron Update 19 : Patron Record Standard Interface File

6. doran@uta.edu Michael Doran, Systems Librarian The Patron SIF specification Each record will consist of  1 base segment - fixed length 52 fixed length fields certain fields are required  1-9 address segments - fixed length 19 fixed length fields one address is required  1 optional notes segment - variable length 1 variable length field  1 end-of-record mark

7. doran@uta.edu Michael Doran, Systems Librarian Patron SIF – one record

8. doran@uta.edu Michael Doran, Systems Librarian Patron SIF – one record

9. doran@uta.edu Michael Doran, Systems Librarian Patron SIF documentation

10. doran@uta.edu Michael Doran, Systems Librarian Patron SIF documentation Item (field) number – 72 total Item (field) name Offset – starting point Format n - numeric s - string d - date Length of field Item (field) description Required field?

11. doran@uta.edu Michael Doran, Systems Librarian Patron SIF files the easy way… Patron SIF … and voila Here’s your Patron SIF file. Enjoy! You send the Patron SIF documentation to your University IT department… programmer

12. doran@uta.edu Michael Doran, Systems Librarian Patron SIF files the hard way… Voyager Manuals your boss I want you to write a program that generates Patron SIF files. Enjoy!

13. doran@uta.edu Michael Doran, Systems Librarian So we have our marching orders

14. doran@uta.edu Michael Doran, Systems Librarian Somewhere on your campus is… One or more directories of information on people registered to attend classes One or more directories of information on people who work for the university students faculty and staff “patrons”

15. doran@uta.edu Michael Doran, Systems Librarian Directories, databases, protocols, APIs directory an alphabetical or classified list (as of names or addresses) database a collection of data organized for rapid retrieval (as by a computer) protocol a standard that controls the connection, communication, and data transfer between two computing endpoints application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software

16. doran@uta.edu Michael Doran, Systems Librarian Directories, databases, protocols, APIs directory an alphabetical or classified list (as of names or addresses) database a collection of data organized for rapid retrieval (as by a computer) protocol a standard that controls the connection, communication, and data transfer between two computing endpoints application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software LDAP directory Lightweight Directory Access Protocol

17. doran@uta.edu Michael Doran, Systems Librarian Directories, databases, protocols, APIs directory an alphabetical or classified list (as of names or addresses) database a collection of data organized for rapid retrieval (as by a computer) protocol a standard that controls the connection, communication, and data transfer between two computing endpoints application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software Voyager patron load “Standard Interface File” (SIF) Pptrnupdt

18. doran@uta.edu Michael Doran, Systems Librarian Voyager patron database tables and relationships

19. doran@uta.edu Michael Doran, Systems Librarian Voyager LDAP client patron info The pieces and players Pptrnupdt patron SIF Perl program Net::LDAP Directory patron info LDAP server ? LDAP Protocol

20. doran@uta.edu Michael Doran, Systems Librarian Voyager LDAP client patron info The pieces and players Pptrnupdt patron SIF Perl program Net::LDAP Directory patron info LDAP server ? LDAP Protocol patron info

21. doran@uta.edu Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

22. doran@uta.edu Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt Perl program

23. doran@uta.edu Michael Doran, Systems Librarian Voyager LDAP client patron info Before we start Perl programming Pptrnupdt patron SIF Perl program Net::LDAP Directory patron info LDAP server ? LDAP Protocol ldapsearch

24. doran@uta.edu Michael Doran, Systems Librarian What you need to get started command line client  ldapsearch comes standard on Solaris LDAP server info  hostname  port numbers 389 – default ldap port 636 – default ldaps port  LDAP Schema bind credentials  Distinguished Name (DN)  anonymous  your own (e.g. NetID username & password)  “superuser” (provided by LDAP administrator)

25. doran@uta.edu Michael Doran, Systems Librarian ldapsearch search ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w " " \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran"

26. doran@uta.edu Michael Doran, Systems Librarian ldapsearch search ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w " " \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran" LDAP server hostnameBind Distinguished Name Bind Credentials Search BaseSearch Filter

27. doran@uta.edu Michael Doran, Systems Librarian ldapsearch output cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName=doran utaPrevAccountName=doran utaPrimaryAccount=doran utaPrimaryEmail=doran@uta.edu utaHomeStreet1=123 Elm St. utaEmployeeTitle=Librarian utaDeptCode=6800000 utaEmployeeStatus=active utaEmployeeCampusBox=19497 eduPersonPrimaryAffiliation=staff utaHomeCity=Arlington utaEmployeeTitleCode=0100 sn=Doran cedarid=1085691573581181000 utaHomeState=TX homePhone=+1 817 555 1234 utaEmployeeJobCategory=prof_non_faculty cn=Michael D Doran utaEmployeeType=monthly utaHomeZip=76013 utaEmployeeBldg=LIB givenName=Michael displayName=doran, michael d utaEmployeeExemption=exempt utaEmployeePhone=+1 817 272 5326 homePostalAddress=123 Elm St.$Arlington$TX$76013 utaID=1000068572 utaEmployeePercentTime=100.00 utaEmployeeRetirementCode=9 utaEmplID=1000068572 utaMiddleName=D objectClass=top objectClass=person objectClass=inetOrgPerson objectClass=utaPerson objectClass=utaEmployee eduPersonScopedAffiliation=employee@uta.edu eduPersonScopedAffiliation=staff@uta.edu utaPersonAffiliation=employee utaPersonAffiliation=staff utaPersonAffiliation=administrative eduPersonAffiliation=employee eduPersonAffiliation=staff search filter match “patron-worthy” search filter

28. doran@uta.edu Michael Doran, Systems Librarian #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => " "); $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(utaAccountName=doran)" ); foreach $entry ($mesg->all_entries) { $entry->dump; }; $ldap->unbind(); exit(0); LDAP search with Perl Net::LDAP

29. doran@uta.edu Michael Doran, Systems Librarian LDAP search with Perl Net::LDAP #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => " "); $mesg = $ldap->search( base => "cn=people, dc=uta, dc=edu", filter => "(utaAccountName=doran)" ); foreach $entry ($mesg->all_entries) { $entry->dump; }; $ldap->unbind(); exit(0); LDAP server hostnameBind Distinguished Name Bind Credentials Search BaseSearch Filter this Perl module will need to be installed

30. doran@uta.edu Michael Doran, Systems Librarian Perl Net::LDAP output dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497 eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d...:...

31. doran@uta.edu Michael Doran, Systems Librarian #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("cn=library,cn=app,dc=uta,dc=edu", password => " "); $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))" ); foreach $entry ($mesg->all_entries) { $entry->dump; };... Superuser & valid patron filtering a superuser Bind DN patron-worthy LDAP records

32. doran@uta.edu Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

33. doran@uta.edu Michael Doran, Systems Librarian LDAP records simplified dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497 eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d...:... attributesvalues attribute-value pair

34. doran@uta.edu Michael Doran, Systems Librarian additional attributes for employees LDAP records made more complex Not all records have all the same attributes let’s consider “people” records additional attributes for students generic attributes for all people

35. doran@uta.edu Michael Doran, Systems Librarian Attributes continued Some attributes are required, but many (most) are optional Some attributes have a constrained set of possible values Some attributes are “multi-valued” Attributes will not necessarily be sorted in any order LDAP Schema for information…

36. doran@uta.edu Michael Doran, Systems Librarian Sample schema page (UTA LDAP)

37. doran@uta.edu Michael Doran, Systems Librarian Sample schema page (UTA LDAP)

38. doran@uta.edu Michael Doran, Systems Librarian $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))", callback => \&ProcessRecord ); foreach $entry ($mesg->all_entries) { $entry->dump; }; Processing records we want to process records dumps records The “callback” option allows you to process each record as it is received

39. doran@uta.edu Michael Doran, Systems Librarian sub ProcessRecord { my ($mesg,$entry) = @_; if (!$entry) {&Finish(0);} $surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = $entry->get_value('utaID'); @uta_affil = $entry->get_value('utaPersonAffiliation'); $email_addr = $entry->get_value('utaPrimaryEmail'); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; } #...yada, yada, yada ProcessRecord subroutine

40. doran@uta.edu Michael Doran, Systems Librarian sub ProcessRecord { my ($mesg,$entry) = @_; if (!$entry) {&Finish(0);} $surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = $entry->get_value('utaID'); @uta_affil = $entry->get_value('utaPersonAffiliation'); $email_addr = $entry->get_value('utaPrimaryEmail'); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; } #...yada, yada, yada assigning LDAP attribute values to program variablessingle-valued attributes => assigned to a scalarmulti-valued attributes => assigned to an array ProcessRecord subroutine minor reformatting of data

41. doran@uta.edu Michael Doran, Systems Librarian LDAP record dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497 eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d...:... Find the Voyager patron group The LDAP record doesn’t have a patron group

42. doran@uta.edu Michael Doran, Systems Librarian if ($stud_status eq "enrolled" && $stud_class) { if ($stud_class =~ /freshman/i || $stud_class =~ /sophomore/i || $stud_class =~ /junior/i || $stud_class =~ /senior/i ) { $patron_group = "und"; } elsif ($stud_class =~ /masters/i || $stud_class =~ /doctoral/i || $stud_class =~ /graduate/i) { $patron_group = "grad"; $expire_days = "180"; } } elsif ($stud_status eq "enrolled") { $patron_group = "und"; } Decision algorithms patron group assignment exampleprogram looks at student attributes first student classification not a required attribute in LDAP record, so need a default

43. doran@uta.edu Michael Doran, Systems Librarian if ($empl_status eq "active") { if (! $patron_group) { $patron_group = "staff"; } foreach my $affil (@uta_affil) { if (($affil =~ /faculty/i) || ($affil =~ /admin/i) || ($affil =~ /assistant/i)) { $patron_group = "fac"; $expire_days = "400"; } Decision algorithms program looks at employee attributes second default only if no student patron group assigned

44. doran@uta.edu Michael Doran, Systems Librarian Decision algorithms continued work address & phone student address & phone home address & phone multiple sets of address attributes employeesstudents all people

45. doran@uta.edu Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

46. doran@uta.edu Michael Doran, Systems Librarian A poetry break… “Our time is fixed, and all our days are number'd; How long, how short, we know not.” Robert Blair, The Grave “The SIF fields are fixed, and all the fields are number'd; How long, how short, we better know.” anonymous programmer

47. doran@uta.edu Michael Doran, Systems Librarian Patron SIF formatting simplified For each patron SIF record  Format each field sstring – left-justified / blank-filled n numeric – right-justified / zero-filled n*“special” numeric – if no value, then all blanks ddate – YYYY.MM.DD format  Concatenate all fields into one long string  Slap an end-of-record marker on string  Write string to file

48. doran@uta.edu Michael Doran, Systems Librarian Remember this?

49. doran@uta.edu Michael Doran, Systems Librarian Formatting with sprintf* $first_name = sprintf(" % -20s",Michael); $patron_id = sprintf(" % 010d",123456); * sprintf help sprintf help $concatenation = $first_name. $patron_id; print $concatenation; Michael 0000123456

50. doran@uta.edu Michael Doran, Systems Librarian Three easy steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

51. doran@uta.edu Michael Doran, Systems Librarian Things I didn’t cover Encryption Data validation Log files Session file Cron job setup yada, yada, yada…

52. doran@uta.edu Michael Doran, Systems Librarian Perl programming You * can do this! Patron SIF LDAP protocol *…you, or your resident geek geek stigmata

53. doran@uta.edu Michael Doran, Systems Librarian Resources Voyager Technical User’s Guide  Patron SIF specification  Patron Update (Pptrnupdt) Perl Net::LDAP module  download from CPAN download from CPAN  documentation documentation Perl Date::Calc module  download from CPAN download from CPAN  documentation documentation LDAP to patron SIF webpage (ldump.pl) LDAP to patron SIF

54. doran@uta.edu Michael Doran, Systems Librarian Questions? Please fill out the session evaluation!