From LDAP to Patron Load From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump Michael Doran, Systems Librarian Endeavor Users Group Meeting, Chicago, IL Session 8, Thursday, April 26, 2007

Michael Doran, Systems Librarian Cataloging Acquisitions Circulation OPAC Michael Doran 123 Elm St. Arlington, TX PATRON BLOCK Integrated library system patron database patron DB

Michael Doran, Systems Librarian Getting patron data into Voyager Manual data entry via the Circulation client D’oh!

Michael Doran, Systems Librarian Getting patron data into Voyager Manual data entry via the Circulation client Batch loading via the Pptrnupdt job  Technical User’s Guide 6 : Patron Update 19 : Patron Record Standard Interface File

Michael Doran, Systems Librarian The Patron SIF specification Each record will consist of  1 base segment - fixed length 52 fixed length fields certain fields are required  1-9 address segments - fixed length 19 fixed length fields one address is required  1 optional notes segment - variable length 1 variable length field  1 end-of-record mark

Michael Doran, Systems Librarian Patron SIF – one record

Michael Doran, Systems Librarian Patron SIF – one record

Michael Doran, Systems Librarian Patron SIF documentation

Michael Doran, Systems Librarian Patron SIF documentation Item (field) number – 72 total Item (field) name Offset – starting point Format n - numeric s - string d - date Length of field Item (field) description Required field?

Michael Doran, Systems Librarian Patron SIF files the easy way… Patron SIF … and voila Here’s your Patron SIF file. Enjoy! You send the Patron SIF documentation to your University IT department… programmer

Michael Doran, Systems Librarian Patron SIF files the hard way… Voyager Manuals your boss I want you to write a program that generates Patron SIF files. Enjoy!

Michael Doran, Systems Librarian So we have our marching orders

Michael Doran, Systems Librarian Somewhere on your campus is… One or more directories of information on people registered to attend classes One or more directories of information on people who work for the university students faculty and staff “patrons”

Michael Doran, Systems Librarian Directories, databases, protocols, APIs directory an alphabetical or classified list (as of names or addresses) database a collection of data organized for rapid retrieval (as by a computer) protocol a standard that controls the connection, communication, and data transfer between two computing endpoints application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software

Michael Doran, Systems Librarian Directories, databases, protocols, APIs directory an alphabetical or classified list (as of names or addresses) database a collection of data organized for rapid retrieval (as by a computer) protocol a standard that controls the connection, communication, and data transfer between two computing endpoints application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software LDAP directory Lightweight Directory Access Protocol

Michael Doran, Systems Librarian Directories, databases, protocols, APIs directory an alphabetical or classified list (as of names or addresses) database a collection of data organized for rapid retrieval (as by a computer) protocol a standard that controls the connection, communication, and data transfer between two computing endpoints application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software Voyager patron load “Standard Interface File” (SIF) Pptrnupdt

Michael Doran, Systems Librarian Voyager patron database tables and relationships

Michael Doran, Systems Librarian Voyager LDAP client patron info The pieces and players Pptrnupdt patron SIF Perl program Net::LDAP Directory patron info LDAP server ? LDAP Protocol

Michael Doran, Systems Librarian Voyager LDAP client patron info The pieces and players Pptrnupdt patron SIF Perl program Net::LDAP Directory patron info LDAP server ? LDAP Protocol patron info

Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt Perl program

Michael Doran, Systems Librarian Voyager LDAP client patron info Before we start Perl programming Pptrnupdt patron SIF Perl program Net::LDAP Directory patron info LDAP server ? LDAP Protocol ldapsearch

Michael Doran, Systems Librarian What you need to get started command line client  ldapsearch comes standard on Solaris LDAP server info  hostname  port numbers 389 – default ldap port 636 – default ldaps port  LDAP Schema bind credentials  Distinguished Name (DN)  anonymous  your own (e.g. NetID username & password)  “superuser” (provided by LDAP administrator)

Michael Doran, Systems Librarian ldapsearch search ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w " " \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran"

Michael Doran, Systems Librarian ldapsearch search ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w " " \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran" LDAP server hostnameBind Distinguished Name Bind Credentials Search BaseSearch Filter

Michael Doran, Systems Librarian ldapsearch output cedarid= ,cn=People,dc=uta,dc=edu utaAccountName=doran utaPrevAccountName=doran utaPrimaryAccount=doran utaHomeStreet1=123 Elm St. utaEmployeeTitle=Librarian utaDeptCode= utaEmployeeStatus=active utaEmployeeCampusBox=19497 eduPersonPrimaryAffiliation=staff utaHomeCity=Arlington utaEmployeeTitleCode=0100 sn=Doran cedarid= utaHomeState=TX homePhone= utaEmployeeJobCategory=prof_non_faculty cn=Michael D Doran utaEmployeeType=monthly utaHomeZip=76013 utaEmployeeBldg=LIB givenName=Michael displayName=doran, michael d utaEmployeeExemption=exempt utaEmployeePhone= homePostalAddress=123 Elm St.$Arlington$TX$76013 utaID= utaEmployeePercentTime= utaEmployeeRetirementCode=9 utaEmplID= utaMiddleName=D objectClass=top objectClass=person objectClass=inetOrgPerson objectClass=utaPerson objectClass=utaEmployee utaPersonAffiliation=employee utaPersonAffiliation=staff utaPersonAffiliation=administrative eduPersonAffiliation=employee eduPersonAffiliation=staff search filter match “patron-worthy” search filter

Michael Doran, Systems Librarian #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => " "); $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(utaAccountName=doran)" ); foreach $entry ($mesg->all_entries) { $entry->dump; }; $ldap->unbind(); exit(0); LDAP search with Perl Net::LDAP

Michael Doran, Systems Librarian LDAP search with Perl Net::LDAP #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => " "); $mesg = $ldap->search( base => "cn=people, dc=uta, dc=edu", filter => "(utaAccountName=doran)" ); foreach $entry ($mesg->all_entries) { $entry->dump; }; $ldap->unbind(); exit(0); LDAP server hostnameBind Distinguished Name Bind Credentials Search BaseSearch Filter this Perl module will need to be installed

Michael Doran, Systems Librarian Perl Net::LDAP output dn:cedarid= ,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimary utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: utaEmployeeStatus: active utaEmployeeCampusBox: eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: utaHomeState: TX homePhone: utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d...:...

Michael Doran, Systems Librarian #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("cn=library,cn=app,dc=uta,dc=edu", password => " "); $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))" ); foreach $entry ($mesg->all_entries) { $entry->dump; };... Superuser & valid patron filtering a superuser Bind DN patron-worthy LDAP records

Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

Michael Doran, Systems Librarian LDAP records simplified dn:cedarid= ,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimary utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: utaEmployeeStatus: active utaEmployeeCampusBox: eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: utaHomeState: TX homePhone: utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d...:... attributesvalues attribute-value pair

Michael Doran, Systems Librarian additional attributes for employees LDAP records made more complex Not all records have all the same attributes let’s consider “people” records additional attributes for students generic attributes for all people

Michael Doran, Systems Librarian Attributes continued Some attributes are required, but many (most) are optional Some attributes have a constrained set of possible values Some attributes are “multi-valued” Attributes will not necessarily be sorted in any order LDAP Schema for information…

Michael Doran, Systems Librarian Sample schema page (UTA LDAP)

Michael Doran, Systems Librarian Sample schema page (UTA LDAP)

Michael Doran, Systems Librarian $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))", callback => \&ProcessRecord ); foreach $entry ($mesg->all_entries) { $entry->dump; }; Processing records we want to process records dumps records The “callback” option allows you to process each record as it is received

Michael Doran, Systems Librarian sub ProcessRecord { my ($mesg,$entry) if (!$entry) {&Finish(0);} $surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = = $entry->get_value('utaPersonAffiliation'); $ _addr = $entry->get_value('utaPrimary '); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; } #...yada, yada, yada ProcessRecord subroutine

Michael Doran, Systems Librarian sub ProcessRecord { my ($mesg,$entry) if (!$entry) {&Finish(0);} $surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = = $entry->get_value('utaPersonAffiliation'); $ _addr = $entry->get_value('utaPrimary '); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; } #...yada, yada, yada assigning LDAP attribute values to program variablessingle-valued attributes => assigned to a scalarmulti-valued attributes => assigned to an array ProcessRecord subroutine minor reformatting of data

Michael Doran, Systems Librarian LDAP record dn:cedarid= ,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimary utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: utaEmployeeStatus: active utaEmployeeCampusBox: eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: utaHomeState: TX homePhone: utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d...:... Find the Voyager patron group The LDAP record doesn’t have a patron group

Michael Doran, Systems Librarian if ($stud_status eq "enrolled" && $stud_class) { if ($stud_class =~ /freshman/i || $stud_class =~ /sophomore/i || $stud_class =~ /junior/i || $stud_class =~ /senior/i ) { $patron_group = "und"; } elsif ($stud_class =~ /masters/i || $stud_class =~ /doctoral/i || $stud_class =~ /graduate/i) { $patron_group = "grad"; $expire_days = "180"; } } elsif ($stud_status eq "enrolled") { $patron_group = "und"; } Decision algorithms patron group assignment exampleprogram looks at student attributes first student classification not a required attribute in LDAP record, so need a default

Michael Doran, Systems Librarian if ($empl_status eq "active") { if (! $patron_group) { $patron_group = "staff"; } foreach my $affil { if (($affil =~ /faculty/i) || ($affil =~ /admin/i) || ($affil =~ /assistant/i)) { $patron_group = "fac"; $expire_days = "400"; } Decision algorithms program looks at employee attributes second default only if no student patron group assigned

Michael Doran, Systems Librarian Decision algorithms continued work address & phone student address & phone home address & phone multiple sets of address attributes employeesstudents all people

Michael Doran, Systems Librarian Three easy(?) steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

Michael Doran, Systems Librarian A poetry break… “Our time is fixed, and all our days are number'd; How long, how short, we know not.” Robert Blair, The Grave “The SIF fields are fixed, and all the fields are number'd; How long, how short, we better know.” anonymous programmer

Michael Doran, Systems Librarian Patron SIF formatting simplified For each patron SIF record  Format each field sstring – left-justified / blank-filled n numeric – right-justified / zero-filled n*“special” numeric – if no value, then all blanks ddate – YYYY.MM.DD format  Concatenate all fields into one long string  Slap an end-of-record marker on string  Write string to file

Michael Doran, Systems Librarian Remember this?

Michael Doran, Systems Librarian Formatting with sprintf* $first_name = sprintf(" % -20s",Michael); $patron_id = sprintf(" % 010d",123456); * sprintf help sprintf help $concatenation = $first_name. $patron_id; print $concatenation; Michael

Michael Doran, Systems Librarian Three easy steps… Query the LDAP directory  filter for valid patrons Process the records  minor data formatting  decision algorithms  output records in SIF format Load SIF file into Voyager  Pptrnupdt

Michael Doran, Systems Librarian Things I didn’t cover Encryption Data validation Log files Session file Cron job setup yada, yada, yada…

Michael Doran, Systems Librarian Perl programming You * can do this! Patron SIF LDAP protocol *…you, or your resident geek geek stigmata

Michael Doran, Systems Librarian Resources Voyager Technical User’s Guide  Patron SIF specification  Patron Update (Pptrnupdt) Perl Net::LDAP module  download from CPAN download from CPAN  documentation documentation Perl Date::Calc module  download from CPAN download from CPAN  documentation documentation LDAP to patron SIF webpage (ldump.pl) LDAP to patron SIF

Michael Doran, Systems Librarian Questions? Please fill out the session evaluation!