ARL New Measures —a View from Cornell Anne R. Kenney ARL Survey Coordinators and SPEC Liaisons Meeting ALA June 2007

Cornell and Assessment Participant in ARL Making Library Assessment Work project (August 2006) New Scope of Research and Assessment Unit (RAU) Fall 2006 –organizational effectiveness, use and usability, service innovation

Making Library Assessment Work Recommendations Examine organizational placement, resources, skill sets, and function of assessment Continue LibQUAL and develop customized survey(s) Gather more qualitative data from customers Put ARL statistics into perspective Develop a usability program Utilize measures of success, move from assessment to action Limit scope of data farm project

Research and Assessment Unit Team: Xin Li, Rich Entlich, Ellie Buckley, David Banush, Linda Miller

RAU Charge #1: Report on Library Performance Prioritize efforts Assemble and interpret data Produce final reports

Charge #2: Manage Assessment Data David Banush, “Options for a CUL Data Mart” (July 2007)

The unhappiness with data collection coupled with a perceived need for even more data collection may perhaps be traced to a very basic problem: an unclear or perhaps non-existent rationale for gathering the data… Several interview subjects expressed significant doubts about data- driven decision models generally. Some maintain that data are not used by the CUL administration in decision making and asked why more information would have any added value. David Banush

Charge #3: Conduct Hi-Priority Projects “ intelligence unit ” –Environmental scans –Patterns and trends “ consultant ” –Design assessment instrument –Administer assessment projects –Analyze and report findings with recommendations/ observations

ARL New Measures: The Cornell Perspective Reviewed ARL Board approved action items along with Yvonna Lincoln and Bruce Thompson reports –Find the characteristics of the library of the future insightful –Applaud ARL’s shift from output-focused to outcome- based measures and collaboration –Look to ARL for clear definitions and communication plan for the new measures

ARL New Measures: The Cornell Perspective Assessed potential impact on CUL –No immediate impact other than adjustments of data in some sub-categories –If implemented, significant additional investment in data collection, requiring data outside library’s control –Some data may not be practically attainable –New measures may lead to incomplete picture, e.g., expenditure-focused index. –New measures may help CUL embrace culture of assessment

New Measures: “Cost-Benefit” Estimate

Next Steps Actively participate in ARL decision making process; evaluate and help shape definitions. Discuss/determine in CUL executive committees what decisions to make for functional areas. Decide what success indicators are, what data captures the success levels, how to organize the data to reveal performance. Decide to what extent to invest resources for the new data collection (cost-benefit) and what activities to drop so resources can be re-directed. Connect the New Measurements initiative with ARL visiting program officers’ findings about CUL Be prepared to make tough choices, tying direction and resources to new measures.

Operationalize the to-dos Focus on local so to create an effective way to support ARL directions Examine the value of data for CUL decision making and priorities—internal and external data and measures Tie library measures more closely to university measures/goals/requirements Make cost-benefit choices Streamline existing data collection effort by 50% or more within the next 12 months Identify gaps, estimate additional investment needed for new measures Pilot new approach with co-sponsored project

Make the “so what” Leap Resist the temptation of data collection based on “it would be interesting, if we know…” Focus on outcome measures. Take a hard look at data collection with vague goals. Recognize value of good enough data. Run RAU like a business with customer-service focus. Have clear charge, authority, leadership, and goals Build staff skills Recognize that culture change and politics are hard. Dare to say, “so what.” Spend the money when your skills and timeline may compromise the result, e.g. persona.