Evaluation Research A review

Setting the Context Attention to mission, vision, and goals of our institution. Evaluating service and quality Assessing processes and programs Continuous improvement Looking at gaps or areas of weakness in the field- where has little research been focused? Evaluating new trends (folksonomy, digital reference, EAD…)

Four Perspectives 1. The institution and the communities served in the life of the library/archives 2. The library/archives and academic departments and programs in the life of the institution 1. The institution and the communities served in the life of the library/archives 2. The library/archives and academic departments and programs in the life of the institution

Taking Measure: Metrics Input Output Performance Customer-related Outcomes Student outcomes Student learning outcomes impacts Other types--impacts

I’mmmmm Back: The Problem Statement Lead in Originality* Direction Significance/Justification/Value Not just the components, but why? Why are these elements important? Is it really important to co-locate in one place?

Literature Review: Remember Me? Exploring prior research Setting the context Establishing a foundation Identifies variables Helps identify methodologies

Literature Review What else are we really interested in? How relevant are the cited works? Are they tied back to the present study? Skewing focus to support an “angle” How authoritative? How accurately are they used/ cited? Skewing focus to support an “angle”

Literature Review How scholarly? Look for patterns not just to see who’s most cited, but what do we know about the authors and journals being cited? Scholarly, Trade, Conference Proceedings, Personal Communications, etc. Cook, S.J., Parker, R.S. and Pettijohn, C.E. (2005). The public library: An early teen’s perspective. Public Libraries, 44 (3),

Theoretical Framework Logically flows from literature review The theories, methods, or understandings on which your research is based. Marsteller, M.R. & Mizzy, D. (2003). Exploring the synchronous digital reference interaction for query types, question negotiation and patron response. Internet Reference Services Quarterly, 8 (1/2),

Questions/ Objectives/ Hypotheses Continue to refine and focus the problem statement. Guide the data collection. Must relate specifically to the problem statement and methodology. Martinez, G. (2007). Partnering for reading readiness: A case study of Maryland public libraries. Children and Libraries, 5(1),

Research Design Type of study: Exploratory Descriptive Explanatory Types of Designs Experiment Quasi-Experiment Case Study

Research Design: Sampling Defining: population, samples, units of analysis Choosing a representative group Random- each member has a known and equal chance of being selected Systematic- assembling a list and choosing every nth member Stratified- assembling a list by type/ social group, then selecting every nth number Convenience samples Typical, snowball, and quota

Methodology Quantitative, Qualitative, Mixed, and Triangulation How to select an appropriate method? What does quantitative data yield? What does qualitative data yield? What is the purpose of our study?

Methodology Some examples: How has electronic access to journals affected inter-library loan? Dean, E., & Goodier, R. (2004). Changing patterns in interlibrary loan and document supply. Interlending and Document Supply, 32(4),

Methodology What do teenagers think of the public library as: a place for entertainment, a place for research, a place to socialize? How satisfied are our customers with our New Book selection and circulation policies? How efficient is our cataloging department?

Methodology How do hobbyist geneaologists search for information in archives? How and to what extent do patrons use our book review system on our web page? Who are the “non-users” in our library? What factors keep them from using the library? How effective is our instruction program?

Why Research? Accountability Continuous Improvement Effectiveness Impact Efficiency Customer Relations Improved Decision-Making

Acknowledging Obstacles Ethical concerns Researcher bias Response bias Lack of institutional support Lack of time, budget, and staff Inappropriate application of evaluation- as a gesture, to torpedo unwanted programs, to justify pet projects