Studying SFX Logs to Better Understand User Behavior Bennett Claire Ponsford, Digital Services Librarian Anne L. Highsmith, Consortia Systems Coordinator Texas A&M University Libraries

Texas A&M University  46,000-plus undergraduate and 8,500 graduate students  250 degree programs in 10 colleges  2,800 faculty in a research-intensive environment  Branch campuses in Galveston, Texas and Doha, Qatar

University Libraries  Member of ARL  Main library with 5 branches  3 in College Station  1 in Galveston, TX  1 in Doha, Qatar  Spending over $7 million per year on electronic resources  Redesigning our website anyway

Our Current SFX Implementation  Went live with SFX 2 in May 2004  SFX 3 in January 2006  A-Z List in January 2006  A-Z List ver. 3 in January 2008  Current SFX menu design unchanged since we went live

Why study users?  To see how your users search when you’re not watching  To resolve internal disagreements over default features to include, etc.  To see whether changes to SFX menus really improved results  As a counterpoint to focus groups and task- based user testing

What do our users say?  Hated all the pop-up windows  Pop-up windows in general  Highjacking previous content  SFX menus too busy and confusing  Did not understand the 3 catalog links  Never used the FAQ or Ulrich’s link  All they want is full text anyway

What do the numbers say? Sample from the SFX logs – 14 days per year – 3 years, FY 2006 – as much of FY 2008 as available Sample from the apache logs – Data available only for FY 2008

Where are they coming from? Fiscal Year 08 CountPercent From Catalog to API % From AZ list to just SFX Menu % From AZ list to Full Text % From MetaLib % From Databases % Total Sample Requests %

Where do they go? FY08 OverallCountPercent Total Sample Requests % Requests with Full Text % - Clicked On % - Not Clicked on %

What if there is no full text? FY08 OverallCountPercent ILL only % LibCat (main catalog) only % Search Google only % LibCat & ILL % LibCat & Chiron (medical catalog) % Chiron only % All 3 catalogs870.60% All 3 catalogs and ILL700.48% ILL for inadequate metadata570.39% LibCat & Search Google560.39% LibCat, Chiron, & ILL400.28%

Full Text from the Catalogs

Where do they go? FY08 from Catalog to APICountPercent Sample requests from Catalog to API5860 No Full Text Found % Full Text % - Clicked on Full Text (once or more) % - Clicked on Full Text (more than once)924.91% - Clicked on Full Text (total clicks) User Did Not Click on Full Text button %

Requests over Time

Full Text Availability and Actions

No Full Text Actions

Public/Library Usage CatalogSFX

Overall Requests: Public/Library

Clicking on Full Text: Public/Library

No Full Text Behavior: Public/Library

Requests from AZ List : Public/Library

Review of Apache Logs Sample of titlesType administrative science quarterlyJournal title adobeSubject? adolescenceJournal title adolescent literatureSubject? adolescentsSubject? Adopotion of biotechnologySubject? adult children homeSubject? adult children with parents at homeSubject? adult educationJournal title Adult LearningJournal title adv exp med biolJournal title

What Next: SFX Menus?  Redesign SFX Menus using simplified menus  Just display full text, if available, in basic section  Decrease all the verbiage  Reduce duplicate listings with display logic  Display catalog links only if holdings available  Experiment with direct link banner option  User test changes

What Next: Home page?  Search box from Libraries home page  Review apache logs re: size of problem  Wording changes/help text

Technical section -- outline Characteristics of stat tables How statistics are gathered and stored Characteristics of Apache logs Modifications to Apache logging to facilitate stats Statistical sample How you can do this too

Characteristics of stat tables (1) 3 stat tables (& offline equivalents) – stat_object – stat_target_service – stat_repeatables Request has 1 entry in stat_object table Tables join on request_id Request has multiple entries in stat_target_service table – 1 entry (row) for each link on menu

Characteristics of stat tables (2) Exceptions to “request in stat_object table has several corresponding records in stat_target_services” – API requests - 0 entries in stat_target_service – Click on any type of link where direct linking occurs – 1 entry in stat_target_service

Stat object data elements (1) Name REQUEST_ID ISSN ISBN LCCN LOCAL TITLE ATITLE JTITLE BTITLE CTITLE SERIES PUBLISHER PLACE_OF_PUBLICATION OBJECT

Stat object data elements (2) Name SUBCATEGORY STATUS DOI REQ_DATE TIME SOURCE IP OBJECT_TYPE INSTITUTE USER_GROUP FACULTY HAS_FULLTXT DATE_OF_PUBLICATION EPAGE SPAGE PRESENTATION_FORMAT SESSION_ID OPEN_URL

Rows in target service table Target Clicks Service INFORMAWORLD_JOURNALS 1 getFullTxt METAPRESS_ROUTLEDGE 1 getFullTxt AM_VOYAGER 1 getHolding MS_VOYAGER 0 getHolding GA_VOYAGER 0 getHolding WWW_SEARCH_ENGINES 0 getWebSearch AM_PROBLEM_REPORT 0 getWebService AM_SFX_FAQ 0 getWebService ULRICHSWEB_COM 1 getCitedJournal

How stats are gathered & stored (1) Run online to offline daily Run export_tab.pl monthly - Embedded in special perl script that copies monthly cumulations to report server Copy stat_object_offline & stat_target_service_offline, but not stat_repeatables_offline

How stats are gathered & stored (2) Copy these tables in their entirety, except for some open_urls in stat_object Perl script on report server loads data into Oracle tables Create separate tables by academic year because of size -- academic 2008 to date: – 1.3M+ requests – 8.4M+ target service links

Characteristics of Apache logs (1) “Real” ip associated with request available only from reverse Apache log Logs that span long time period – Beware of differences between v2 and v3 a-z lists – Note if you have changed display options, e.g. between brief and detail view

Characteristics of Apache logs (2) Certain data available only from logs, because it doesn’t generate a “request” or hasn’t generated a request yet. – Category search – Auxiliary functions Use of info button on az list Push to Metalib myspace from az list Opening SFX az list from within Metalib

Modifications to Apache logging Set up custom logging statement in httpd.conf – Write the data in a single record – Store tabs between data elements within record

Samples STAT table samples – 2 weeks worth of data across 3 years – Same day of week / same week of month 1 st Thursday in January, 2 nd Friday in March Apache logs – Academic 2008 was only year available with sufficient data elements – Selected representative days from that year

DIY instructions (1) Run online2offline job (server_admin_util) Run /exlibris/sfx_ver/sfx_version_3/[instance]/ad min/database/export_tab.pl – By default writes *.exp file to scratch directory – Must run once for each table Download *.exp files & load into MS Access

DIY instructions (2) Go to and click on link for this presentation to download zip file. Zip file contains – Copy of this presentation – Sample MS Access databases – Perl program to parse Apache log entries – Custom Apache log format for httpd.conf

Contact info Bennett Claire Ponsford – – Anne L. Highsmith – –