Jacqui Dowd LibQUAL+ and Beyond: Using Results Effectively University of Glasgow 24 th May 2010 LibQUAL+ & LibQUAL Lite at the University of Glasgow.

Slides:



Advertisements
Similar presentations
LibQUAL+ ® : The UK and Irish Experience Selena Killick Library Quality Officer, Cranfield University J. Stephen Town Director of Information, The University.
Advertisements

Jacqui Dowd Introduction to LibQUAL+ University of Westminster 5 th February 2010 LibQUAL+ v LibQUAL Lite at the University of Glasgow.
LibQUAL+ in the local context: results, action and evaluation Selena Lock & Stephen Town Cranfield University 6th Northumbria International Conference.
Assessment of the Impact of Ubiquitous Computing on Learning Ross A. Griffith Wake Forest University Ubiquitous Computing Conference Seton Hall University.
Does size matter? The effect of resource base size on faculty service quality perceptions in academic libraries Damon Jaggars, Shanna Smith & Fred Heath.
LibQUAL+ in the UK & Ireland: five years experience J. Stephen Town and Selena Lock, Cranfield University.
Reading LibQUAL+ Results The University of Chicago Library LibQUAL+™ Survey Supervisors’ Meeting June 16, 2004.
Using Basic Data Visualization Methods to Explore LibQUAL+ ® Data Ray Lyons & Martha Kyrillidou 9th Northumbria International Conference on Performance.
LibQUAL+® Lite at UNT Presented By Diane Wahl Texas Library Association Annual Conference Houston, Texas April 2, 2009.
Listening To Our Users Queen’s 2010
Library Assessment Conference: Building Effective, Sustainable, Practical Assessment August 5, 2008 LibQUAL+ Lite: A New Model for Conducting Service Quality.
BUSINESS AND FINANCIAL LITERACY FOR YOUNG ENTREPRENEURS: EVIDENCE FROM BOSNIA-HERZEGOVINA Miriam Bruhn and Bilal Zia (World Bank, DECFP)
BPS - 3rd Ed. Chapter 131 Confidence intervals: the basics.
Project web site: old.libqual.org French Canadian 2003 Martha Kyrillidou Toni Olshen Fred Heath Claude Bonnelly Jean-Pierre Cote 5 th Northumbria International.
Glasgow, Scottland May 24, 2010 ITEM SAMPLING IN SERVICE QUALITY ASSESSMENT SURVEYS TO IMPROVE RESPONSE RATES AND REDUCE RESPONDENT BURDEN: THE “LibQUAL+®
Making all research results publically available: the cry of systematic reviewers.
Jacqui Dowd SCONUL Workshop University of Westminster 17 th November 2009 Performance Measures & Metrics at University of Glasgow Library.
LibQUAL + ™ Data Summary An overview of the results of the LibQUAL+™ 2003 survey with comparisons to the 2001 survey.
LibQUAL+ ® Survey Results Presented by: Selena Killick ARL/SCONUL LibQUAL+ Administrator Cranfield University Introduction to LibQUAL+
The votes are in! What next? Introduction to LibQUAL+ Workshop University of Westminster, London 21st January 2008 Selena Killick Association of Research.
Student Engagement Survey Results and Analysis June 2011.
Reliability and Validity of 2004 LibQUAL+™ Scores for Different Language Translations Martha Kyrillidou Colleen Cook Bruce Thompson ALA Annual Conference.
Background on the SCONUL LibQUAL+ implementation Stephen Town, Cranfield University.
Elementary Assessment Data Update Edmonds School District January 2013.
LibQual 2013 Concordia University Montréal, Québec.
School-wide PBIS Tiered Fidelity Inventory
Project URL – TM LibQUAL+ ™ Introduction Seattle / London January, 2007 Presented by: Colleen Cook Bruce Thompson.
Frank Haulgren Collection Services Manager & Assessment Coordinator Western Libraries Lite 2010 Survey Results.
Source: Annual Population Survey, Office for National Statistics. Full time and part time employment Coventry population.
UAA/APU CONSORTIUM LIBRARY 2011 LIBQUAL RESULTS APU Faculty Assembly – February 15, 2012.
Service priority alignment in Association of Research Libraries (ARL) member libraries Damon Jaggars & Shanna Smith University of Texas at Austin Jocelyn.
Assessing SAGES with NSSE data Office of Institutional Research September 25 th, 2007.
LibQUAL 2005 at London South Bank. Peter Godwin London South Bank University 2 February 2006.
LibQUAL+™ Process Management: Using the Web as a Management Tool Amy Hoseth Massachusetts LSTA Orientation Meeting Boston, MA October 21, 2005 old.libqual.org.
Teacher Engagement Survey Results and Analysis June 2011.
Sampling in Research Suter, Chapter 8. Questions about sampling Sample size – do I have enough participants? Is it the right kind of sample? Is it representative?
Essential Statistics Chapter 141 Thinking about Inference.
BPS - 3rd Ed. Chapter 131 Confidence Intervals: The Basics.
Re-Visioning the Future of University Libraries and Archives through LIBQUAL+ Cynthia Akers Associate Professor and Assessment Coordinator ESU Libraries.
The Satisfied Student October 4 th, Today’s Presentation  Present data from Case’s Senior Survey and the National Survey of Student Engagement.
Employment, unemployment and economic activity Coventry working age population by ethnicity Source: Annual Population Survey, Office for National Statistics.
1 Project web site Evaluating Library Service Quality: Use of LibQUAL+  IATUL Kansas City, MO June 2002 Julia Blixrud Association.
UNDERSTANDING 2012 NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE) RESULTS Nicholls State University October 17, 2012.
LibQUAL Survey Results Customer Satisfaction Survey Spring 2005 Sidney Silverman Library Bergen Community College Analysis and Presentation by Mark Thompson,
TM Project web site: Bruce Thompson Colleen Cook Martha Kyrillidou Glasgow January 19-20, 2004 Quantitative Grounding “22 Items.
LibQUAL+® Lite in Hebrew By Dr. Lynne Porat University of Haifa Library LibQUAL+® Exchange, 8th Northumbria International Conference on Performance Measurement.
LibQUAL 2005 at London South Bank and a Lincolnshire man in Chicago.
LibQual+ Spring 2008 results and recommendations Library Assessment Working Group 11/19/2008 Library Faculty Meeting.
School of something FACULTY OF OTHER Leeds University Library LibQUAL+ at Leeds - one year on Pippa Jones Head of Customer Services, Leeds University Library.
TM Project web site Report Meeting Colleen Cook Bruce Thompson Martha Kyrillidou Jonathan Sousa June 21, 2003 ALA Toronto CANADA.
Project URL – TM QUANTITATIVE EVIDENCE Auckland, NZ April 5, 2005 Presented by: Colleen Cook Bruce Thompson.
Old.libqual.org LibQUAL+ TM : A Total Market Survey Duane Webster ARL Executive Director January 2004 San Diego, CA.
Claim 1 Smarter Balanced Sample Items Grade 7 - Target H Draw informal comparative inferences about two populations. Questions courtesy of the Smarter.
TM Project web site New Ways of Listening to Users: Colleen Cook Bruce Thompson Consuella Askew Waller April 10-13, 2003 ACRL 11 th.
Focus on SCONUL Institutions: Cranfield University – DCMT Campus Stephen Town.
LibQUAL + ™ 2004 Data Summary An overview of the results of the LibQUAL+™ 2004 survey with comparisons to past surveys.
LibQUAL+ ® Survey Results Presented by: Martha Kyrillidou Senior Director, Statistics and Service Quality Programs Association of Research.
The Hospital CAHPS Program Presented by Maureen Parrish.
Our 2005 Survey Results. “….only customers judge quality; all other judgments are essentially irrelevant” Delivering Quality Service : Balancing Customer.
QUANTITATIVE EVIDENCE
Martha Kyrillidou, Director, ARL Bruce Thompson, Texas A&M
LibQUAL+ in the UK & Ireland: five years experience
Looking at your program data
LibQUAL+™ Results Meeting
Reading Radar Charts.
Claim 1 Smarter Balanced Sample Items Grade 7 - Target H
LibQUAL+ v LibQUAL Lite at the University of Glasgow
LibQual+ Survey Results 2002
QUANTITATIVE EVIDENCE
QUANTITATIVE EVIDENCE
Presentation transcript:

Jacqui Dowd LibQUAL+ and Beyond: Using Results Effectively University of Glasgow 24 th May 2010 LibQUAL+ & LibQUAL Lite at the University of Glasgow

History of LibQUAL+ at the University of Glasgow Member of the First SCONUL Consortium in 2003, 2004, 2005, and 2006 Again in Spring 2008, Winter 2008 LibQUAL Lite Pilot and Spring 2009 And Spring 2010 LibQUAL Lite 100%

LibQUAL+ Lite 1.Why did we choose LibQUAL+ Lite for Effect of Lite on Longitudinal/Internal benchmarking 3.Effect of Lite on External Benchmarking 4.Effect of Lite on the Library’s Balanced Scorecard

LibQUAL+ Lite Why 100% LibQUAL+ Lite? To increase the response rates - Although we have always had a representative sample, the response rates have been consistently below 10% By Reducing the burden on the respondents - LibQUAL+ requires 97 responses – unreasonable expectation! LibQUAL+ Lite requires only (?) 51 responses

Increase Valid Survey Yield? “Typically about half of the people who view the survey tend to submit a complete version of the survey.” Martha Kyrillidou, Item Sampling in Service Quality Assessment Surveys To Improve Response Rates and Reduce Respondent Burden: The “LibQUAL+ Lite” Randomized Control Trial (RCT) Dissertation, University of Illinois, 2009 At Glasgow? Yes! Completed surveys = 48% Valid Surveys = 46%

Lite Effect 100% Lite response rate increased by 4.2% on LibQUAL Full Protocol 2009

LibQUAL+ Lite Reduction in the burden on Respondents? Yes – in so much as fewer responses were required Not significantly in terms of the average completion time saving only 1 minute 48 seconds Yes – significantly in terms of the median completion time by saving 3 minutes 20 seconds

LibQUAL+ Lite v Full “Thus, the samples of people deciding to provide data are to some extent qualitatively different across the two protocols, with samples for the Lite protocol having somewhat more negative views of library service quality” Item Sampling in Service Quality Assessment Surveys to Improve Response Rates and Reduce Respondent Burden the “LibQUAL+® Lite” Example Bruce Thompson, Martha Kyrillidou, Colleen Cook, Performance Measurement and Metrics, Vol. 10(1), 2009: “The fact that for the most part these small differences are in the direction of Lite producing slightly lower scores, one may argue that Lite produces slightly more accurate estimates of the population statistics for these (LP & IC) concepts……..” Martha Kyrillidou, Item Sampling in Service Quality Assessment Surveys To Improve Response Rates and Reduce Respondent Burden: The “LibQUAL+ Lite” Randomized Control Trial (RCT), Dissertation, University of Illinois, 2009

Effect of Lite: 22 Core Scores All Lite Service Level scores lower than 2008 & 2009

Effect of Lite-Affect of Service Scores All Lite scores ≥ 2008 & 2009

Effect of Lite Information Control scores All Lite Service Level scores < 2008 & 2009

Library As Place Scores All Lite Service Level scores < 2008 & 2009

General Satisfaction Lite score < 2008 & 2009

Academic Support Lite score > 2008 & < 2009

Information Literacy Outcomes Lite score > 2008 & 2009

Effect on Benchmarking Lite scores have a negative effect when benchmarking with other Russell Group Libraries! This pattern is repeated at item level!

Effect on Balanced Scorecard The University of Glasgow Library’s Balanced Scorecard has five User Perspective KPIs based on LibQUAL+ Scores KPI U-2a LibQUAL+ Library as Place: average perceived service level score to be greater than SCONUL average and greater than 6.4 KPI U-2b LibQUAL+ Information Control: average staff & Postgraduate perceived service level score to be greater than SCONUL Average and greater than 6.8

Effect on Balanced Scorecard KPI U-2c LibQUAL+ Affect of Service: average staff & postgraduate perceived service level scores to be greater than SCONUL average and greater than 6.8 KPI U-2d LibQUAL+ General Satisfaction: average score to be greater than SCONUL average and greater than 6.7 KPI U-5 LibQUAL+ Information Literacy: average satisfaction score to be greater than SCONUL average and greater than 6

Effect on Balanced Scorecard KPI U-2a LibQUAL+ Library as Place: average perceived service level score to be greater than SCONUL average and greater than 6.4 ☺ Glasgow Average Score = 6.84 Sconul Average Score = 6.75 KPI U-2b LibQUAL+ Information Control: average staff & Postgraduate perceived service level score to be greater than SCONUL Average and greater than 6.8 ☺ Glasgow Pg Average Score = 6.95 Sconul Pg Average Score = 6.94 ☺ Glasgow Staff Average Score = 7.12 Sconul Staff Average Score = 6.87

Effect on Balanced Scorecard KPI U-2c LibQUAL+ Affect of Service: average staff & postgraduate perceived service level scores to be greater than SCONUL average and greater than 6.8 ☺ Glasgow Pg Average Score = 7.20 Sconul Pg Average Score = 7.09 ☺ Glasgow Staff Average Score = 7.49 Sconul Staff Average Score = 7.26 KPI U-2d LibQUAL+ General Satisfaction: average score to be greater than SCONUL average and greater than 6.7 ☺ Glasgow Average Score = 7.13 Sconul Average Score = 6.97

Effect on Balanced Scorecard KPI U-5 LibQUAL+ Information Literacy: average satisfaction score to be greater than SCONUL average and greater than 6 (Information Literacy Outcome questions 4 & 5, The library helps me distinguish between trustworthy and untrustworthy information & The Library provides me with the information skills I need in my work or study) ☺ Glasgow Average Score = 6.25 Sconul Average Score = 5.99

Summary of Lite Effect Effect of Lite on Longitudinal/Internal benchmarking Lite 22 core mean perceived service levels score lower than Full protocol Information Control & Library as Place lower than Full protocol Affect of Service mean perceived service level scores higher than Full protocol Effect of Lite on External Benchmarking Serious negative effect on 22 core mean perceived service level scores Serious negative effect on Information Control & Library as Place mean perceived service level scores Minor positive effect on Affect of Service mean received service level scores Effect of Lite on the Library’s Balanced Scorecard

Summary of Lite Effect Effect of Lite on the Library’s Balanced Scorecard Targets achieved with improvement on KPI U-2a Library as Place KPI U-2b Information Control KPI U-2c Affect of Service Target achieved with smaller margin on KPI U-2d General Satisfaction KPI U-5 New to Balanced Scorecard

Future Participation 1.“Given that Lite forms have higher participation and significantly lower completion times than the long forms, Lite is advantageous and the preferred form to implement” 2.“Though score conversion is not needed, there are some circumstances under which score conversion may be more useful for large research libraries that rely heavily on the LibQUAL+ protocol through annual or biennial implementations.” Martha Kyrillidou, ibidem All things considered, Glasgow’s future participation will continue as 100% Lite