Meaningful Measurement How do we articulate the value of public libraries in a contemporary context? SWITCH Conference 2015 Jackie Bailey, Principal BYP Group

(c) BYP Group Exercise: Jot down words or doodle pictures in answer to the following question: When I think about ‘evaluation’ and libraries, I feel….

What is evaluation? What, if anything, has changed because of us? Who would miss us if we were gone? Why would they they miss us? What value do we create for society, which no one else can (or at least, not as efficiently as we do)? (c) BYP Group Society and culture Wider community Other stakeholders Library users You and library staff

Why evaluate? (c) BYP Group Improve Use your findings to continuously improve Challenge your own or others’ assumptions Discover unexpected outcomes Communicate Tell people about the value you create in ways they can understand Advocate Use your findings to advocate for what libraries do, which no one else can The unexamined life is not worth living - Socrates

Formative evaluation Establish the baseline Identify a ‘theory of change,’ linked to the research Identify goals and indicators or success Process evaluation Look at how you work e.g. partnerships, operations Summative evaluation Measure the impact of your activity over the short, medium and long term, and on various people or groups Stages of evaluation (c) BYP Group

What do I evaluate? (c) BYP Group Who is this evaluation for? How will this evaluation be used? What do I need to know? What is a nice to have, and what is a must have? What can actually be measured, given my time and resource constraints? Evaluation is… for us (e.g. program improvement) Process evaluation Impact evaluation for funders (e.g. lobbying for funding) Impact evaluation Return on investment Funders’ objectives For example…

What are ‘impacts’ and how (on earth) do I measure them? (c) BYP Group Warning: these ‘impacts’ are off the top of my head and should just be read as an example! What are the impacts? Access to resources they would not otherwise easily afford Opportunity for social bonding Opportunity for self-expression and identity Sense of community connectedness, belonging Personal capacity development as a result of using library resources How do I measure them? Number of users and frequency of usage User demographics, including access to like resources Survey of user experience, benefits, suggestions, uses, self-assessment of the impact of the library on them Survey of user capacity development (preferably referring to benchmark data) Gather stories of ‘change’ amongst library users – e.g. how people’s lives are different BECAUSE of the library Creative tools e.g. before and after pictures, ‘if there was no library,’ vox pops, chalkboards Library Users

What are ‘impacts’ and how (on earth) do I measure them? (c) BYP Group Warning: these ‘impacts’ are off the top of my head and should just be read as an example! What are the impacts? Sense of community identity, public spaces, civil society ‘Institutional value’ of the existence of libraries (John Holden) Sense of having resources if needed How do I measure them? Survey of wider community perceptions and attitudes towards libraries (e.g. through local council surveys) Contingent valuation survey Community members

What are ‘impacts’ and how (on earth) do I measure them? (c) BYP Group Warning: these ‘impacts’ are off the top of my head and should just be read as an example! What are the impacts? Social and cultural capital Economic outcomes e.g. support for small businesses, further education and employment How do I measure them? Theoretical assertions based on your findings, and drawing sound conclusions about your contribution to the determinants of social and cultural capital as outlined in the research literature Study of business, education and job seeker usage Society, economy and culture

What are ‘impacts’ and how (on earth) do I measure them? (c) BYP Group Warning: these ‘impacts’ are off the top of my head and should just be read as an example! What are the impacts? Contribution to the ongoing development of library culture and information systems Capacity development of staff How do I measure them? Staff surveys Internal team reflection and discussion Conversations with respected peers or mentors Awards You and the art of ‘librarianism’

Possible methods (c) BYP Group Demographics of users Level of access to like resources Suggestions and needs Qualitative impacts e.g. sense of connection, social bonding, confidence, self-efficacy User impact surveys Vox pops Writing exercise Pictures of before and after, or the community with/without a library Chalkboard words Interviews and group discussions Stories of change Contingent valuation Public perception of libraries Wider community value survey Social ROI Economic ROI Return on investment studies

How some organisations use their evaluations  Penrith City Council  Booklet & DVD  Awards – Australian National Disability Award for Excellence in Social Participation; Arts and Health Award  Arts Centre Melbourne:  Published report and summary report (flipbook)  Online video for donors  MCA  Funding reports  Peer to peer learning Improve, Communicate, Advocate (c) BYP Group

Exercise: Jot down words or doodle pictures in answer to the following question: Now, when I think about ‘evaluation’ and libraries, I feel….

(c) BYP Group What, if anything, changed?

Discussion Jackie Bailey, Principal BYP Group M: E: