Technological Tools for Assessment New Ways of Listening to Library Users Workshop Washington, DC November 4 - 5, 2005 Jonathan D. Sousa Technical Applications Development Manager Association of Research Libraries old.libqual.org

Who I Am Jonathan D. Sousa Technical Applications Development Manager –Responsible for the development of online applications for ARL’s Statistics and Measurement Program –LibQUAL+™, StatsQUAL™, DigiQUAL™, ARL Statistics Survey (new version), MINES for Libraries 4 years at ARL 8 years in application development Worked 2 years in an ARL library Mucking about online since Gopher was new and 14.4 Kbps was fast

old.libqual.org What I Want to Cover Today The technological history of LibQUAL+™ –Nicely representative of how technology and assessment in libraries have grown together Current assessment offerings using technology –StatsQUAL™ project, DigiQUAL™, MINES for Libraries –Tools to help with other types of analysis Future directions (with online demonstration) Whatever you want –I’m happy to digress But one thing you should know:

old.libqual.org

The LibQUAL+™ Online Survey Process Web-based survey hosted at a single location but delivered all over the world Each participating institution receives a URL with a unique code embedded When individuals take the survey, they see a version tailored to their institution’s language and type and further customized by the survey liaison using our online administration tools Results are compiled and analyzed by the LibQUAL+™ team and made available for download at the end of the survey run

old.libqual.org LibQUAL+™ – The Technological Beginnings Texas A&M – Pre-2000: –Simple piece of rough ColdFusion code and a flat-file database –Experimental –Few thoughts of scaling the project –Innovative but localized

old.libqual.org Pilot Program with ARL – : Some new techniques were attempted Numerous obstacles were encountered in moving beyond one institution Production of the results was a laborious process, particularly in 2001 An attempt at gathering data on library users across institutions with a new protocol using a relatively new medium The full power of LibQUAL+™ was starting to appear in its standardized core: cross-institutional analysis

old.libqual.org LibQUAL+™ – A Shift in Development – 2002 The project development moved to ARL Code base was entirely rewritten Database was shifted to a scalable relational structure An interface for institution liaisons was added Administrative pages were created

old.libqual.org LibQUAL+™ – Refinement and Growth – The site was rewritten again in 2003 –Improve the interface –Simplify the process –Increase customization options –Add additional languages and institution types Automation of results production What the public sees is now the tip of the iceberg: nearly all administration of the project takes place through restricted pages on the website Survey instrument refined: the core is now 22 questions, 3 dimensions

old.libqual.org Making LibQUAL+™ Mundane Faced with many people now coming to LibQUAL+™ without first being familiar with ARL Different types of institutions coming onboard, many with very limited staff and few hours to devote to assessment How LibQUAL+™ has responded: –Simplified all steps as much as we are able, while still allowing flexibility –Accept credit cards and consortial payment arrangements –Direct download of results instead of mailing them around the world: faster and it keeps the price of the survey low –Excel format of results for those without access to SPSS –Available all year long—2 sessions to accommodate the Southern Hemisphere –Moved to professional hosting facility for assured uptime

old.libqual.org LibQUAL™ – Rapid Growth

old.libqual.org Rapid Growth in System Complexity 9 Languages –Afrikaans –American English –British English –Dutch –Dutch English –French (Canada) –French (Europe) –Swedish –Swedish English 14 Types of Institutions –Academic Health Sciences –Academic Law –Academic Military –College or University –Community College –Electronic –European Business –Family History –Hospital –Natural Resources Library –Public –Research Centers (FFRDC) Libraries –State –University/TAFE

old.libqual.org LibQUAL+™ as a Mirror of Assessment in Libraries Started in isolation Expanded but suffered growing pains Grant funded or recipient of a one-time allocation of funds Began to prove its worth Became part of library’s and institution’s decision making processes Shifted from a single event to an ongoing process (a culture of assessment) Now frequently treated as a necessary part of the library’s mission as demonstrated by employees and committees dedicated to assessment

old.libqual.org How LibQUAL+™ Technology Concerns Have Changed 2 years ago the things I talked about: –Network issues Either things are getting better or we are all more familiar with how the Internet works… and breaks –Caching Software on the networks and on computers is finally doing its job without getting in the way Less concern about bandwidth and network speed and fewer people looking to caching as a solution –Getting accurate lists Many more institutions now see keeping an up-to-date list to be critical to their functioning Now we need to worry about keeping out of aggressive spam filters –Browser compatibility This one hasn’t changed: there are still holdouts who continue to use browsers from before 1997

old.libqual.org Online Tools for Assessment DigiQUAL™ –Short survey for digital libraries using the 2-column SERVQUAL format –Math Forum, MERLOT, Utopia, etc. MINES for Libraries –Survey on the purpose of use of electronic resources –Run at individual libraries without a centralized server –Mandatory for certain time periods ARL Statistics Survey –A long-running survey of collections, expenditures, staffing, and service activities at ARL institutions –The collection tool was completely rewritten this year while the questions remained the same

old.libqual.org Other Technological Tools Usability studies –Camtasia –Captivate Web log analysis –Urchin –WebTrends

old.libqual.org Future Projects StatsQUAL™ –Unify the various assessment tools under one system with a single login –Share datasets between surveys Interactive Statistics –Beta version demonstration

old.libqual.org What We’re Focusing On Data integration Dynamic analysis Web standards New assessment solutions

old.libqual.org Online Resources StatsQUAL™ Website: LibQUAL+™ Website: Publications: Events and Training: LibQUAL+™ Bibliography: LibQUAL+™ Procedures Manual:

old.libqual.org LibQUAL+™ Contacts Martha Kyrillidou –Director, Statistics & Measurement Program Amy Hoseth –LibQUAL+™ Communications Coordinator Richard Groves –Research Assistant - Statistics & Measurement Jonathan D. Sousa –Technical Applications Development Manager