Converting Local Portal Data to MySpace Anne L. Highsmith, Consortia Systems Coordinator & Prabha Acharya, Programmer Analyst II Texas A&M University

Converting Local Portal Data to MySpace ELUNA 2007 Points to Cover Background Data Conversion Issues Testing on a Cloned Database Conversion Process Results and Future Plans

Converting Local Portal Data to MySpace ELUNA 2007 Background (1) TAMU had a local Electronic Resources Locater (ERL) function to list ejournals and databases. Users searched ERL and clicked on links to access eresources Users could store the links in their MyPortal record on the library’s website

Converting Local Portal Data to MySpace ELUNA 2007 Background (2) Metalib went into production at TAMU in January 2006 as Search Now. TAMU had discussions with Ex Libris staff as to how MyPortal links could be converted to Metalib First proposal, for Ex Libris staff to do a major conversion, not feasible Decided to wait for Metalib 4.0 and X-Portal APIs to store data to MySpace

Converting Local Portal Data to MySpace ELUNA 2007 Rude awakening! Fall of 2006, Ex Libris staff informed TAMU there would be no API to store to MySpace Alternatives Margaritas? Convert them ourselves?

Converting Local Portal Data to MySpace ELUNA 2007 Conversion options Assumed conversion would have to be: User-initiated Title by title Discovered a way do the conversion directly, without involving the end user

Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues Patrons Journals Databases

Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Patrons How to match the MyPortal patron with the Metalib user? MyPortal patrons identified by uin (University ID Number) Our Metalib implementation uses uin as user login Needed to match the uin with the z312_source_id

Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Journals How to match MyPortal journals with Metalib? Basic journal data stored in z11 table Wrote Perl script ( journals.pl ) to extract journal identifier and issn from z11 Used MS Access to match MyPortal data, keyed by uin & issn, with Metalib data User journal data stored in z312_jnl_001 Wrote Perl script ( journals_by_patron.pl ) to extract z312_source_id (uin) and journal identifier from Metalib user record in z312 table

Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Databases How to match MyPortal databases with Metalib? During creation of Metalib IRDS, stored the ERL key in the IRD cataloger note Loaded IRD extract into MS Access Matched Metalib user extract with IRD extract Basic database data stored in z122 table Wrote Perl script ( databases_by_patron.pl ) to extract z312_source_id (uin) and Metalib identifier (TEX #)

Converting Local Portal Data to MySpace ELUNA 2007 Data Conversion Issues - Final As a result of matching ERL data and Metalib data, created a file that contained 1 or 2 entries for each user whose data had to be converted: Tab-separated record beginning with uin and list of database id numbers Tab-separated record beginning with uin and list of journal id numbers

Converting Local Portal Data to MySpace ELUNA 2007 Testing on a Cloned Database

Converting Local Portal Data to MySpace ELUNA 2007 Testing – Cloning the database (1) TAMU has a test Metalib DB on separate server Cloned production DB to test DB so could test in a realistic environment mpsync - documented in Appendix D in the System Administrators Guide & Knowledge Base entry #1240. exp_current_library & imp_current_library, documented in 7.8 in the SysAdmin Guide Further doc – “Working with Test/Failover Installations of MetaLib”

Converting Local Portal Data to MySpace ELUNA 2007 Cloning – Export from Production Created files on prod for export to test Run mpsync to copy IRDs TAMU has approx IRDS Elapsed time – 58 sec. Run export current library for vir00 TAMU has 32,000+ user records Elapsed time – 1 hr. Run export current library for jnl01 TAMU has approx. 44,000 journal records Elapsed time – 2:25 min.

Converting Local Portal Data to MySpace ELUNA 2007 Cloning – Import into Test Import IRDS created by mpsync Elapsed time – 3 hrs. 688 new records added Matched existing keys. Appeared to create some duplicates Import User Data Elapsed time – 40 min. Drops & re-adds tables Import Journal Data Elapsed time – 3 min. Drops & re-adds tables

Converting Local Portal Data to MySpace ELUNA 2007 Conversion Process Quality control File preparation Loading new patrons Convert databases and ejournals

Converting Local Portal Data to MySpace ELUNA 2007 Conversion – Quality Control Wrote a Perl script ( pre_migration.pl ) to collect stats and set up QC measures. For each user it reports: Users who don’t have Metalib record # of databases and journals already in Metalib # of databases and journals to convert # of users that have more than maximum number of databases or ejournals

Converting Local Portal Data to MySpace ELUNA 2007 Conversion – File Preparation Testing demonstrated that It takes about 3 seconds per resource to be migrated. With about resources that would be about 10 hours So, wrote Perl script ( breakup_myportal_file.pl ) to break the input file into several files of about 250 records each.

Converting Local Portal Data to MySpace ELUNA 2007 Conversion – Loading new patrons Since TAMU uses external authentication rather than patron load, some older MyPortal patrons not in Metalib database Using file created during QC stage, create_xml_patron_load.pl formats user records for Metalib patron load utility Set expiry date to 1 year in future Based on a review of records created on that date, approx. 65 have since logged in.

Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (1) Run databases_by_patron.pl to get current lists from user records for comparison

Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (2) Using save_to_bookbag.cgi, for each user Log in to Metalib Send request to Metalib pds to get pds handle Send request to Metalib, using just obtained pds handle, to get session id Construct url of the form dshandle&func=find-db-add- res&resource=resourceid

Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (3) Set session id in cookie For each database Check to see if it duplicates db already stored If not, store db to user’s clipboard For each journal Store to user’s record

Converting Local Portal Data to MySpace ELUNA 2007 Conversion - Convert databases and ejournals (4) Do QC to make sure that all of the appropriate databases and ejournals are stored Exit the session Error logging revealed that for some users, 1 database or ejournal was not stored – these were re-run as an added input file

Converting Local Portal Data to MySpace ELUNA 2007 Final results Processing all files took most of the week, off and on Each file, limited to 250 records, took about 1 hour to process Only post-production problem had nothing to do with conversion – ezproxy error. Once users have logged into library portal, can now go directly to My Databases or My Ejournals list within Metalib

Converting Local Portal Data to MySpace ELUNA 2007 Future plans Conversion of Voyager My Bookbag to e- shelf

Converting Local Portal Data to MySpace ELUNA 2007 Contact & download info Anne L. Highsmith Consortia Systems Coordinator Texas A&M University URL to download perl code, cgi, etc. ntations/save_to_bookbag.zip