1 Conclusion

2 Metrics Revisited Inputs Outputs Quality and Satisfaction  Interrelated. Rely heavily on subjective responses and self-reporting Outcomes  Impacts on community- Our effects

3 Outcomes Assessment Measuring change as a direct result of interaction with the library Most often addressed in academic and school libraries Types of change:  Knowledge  Behavior  Awareness/ Percpetions  Attitudes/ Opinions

4 Assessment Defined The ongoing process of:  Establishing clear, measurable expected outcomes of student learning  Ensuring that students have sufficient opportunities to achieve those outcomes  Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations  Using the resulting information to understand and improve student learning [at both the course and program] level Suskie (2004, p. 3)

5 NEASC STANDARD SEVEN: Library and Information Resources 1: The institution makes available the library and information resources necessary for the fulfillment of its mission and purposes. These resources support the academic and research program and the intellectual and cultural development of students, faculty, and staff. Library and information resources may include the holdings and necessary services and equipment of libraries, media centers, computer centers, language laboratories, museums, and any other repositories of information or technological systems required for the support of institutional offerings. Clear and disseminated policies govern access, usage, and maintenance of the library, information resources, and services. The institution ensures that students use these resources as an integral part of their education. 2: Through the institution's ownership or guaranteed access, sufficient collections, information technology systems, and services are readily accessible to students wherever programs are located or however they are delivered. These collections, systems, and services are sufficient in quality, level, diversity, quantity, and currency to support and enrich the institution's academic offerings. The institution provides facilities adequate to house the collections and equipment so as to foster an atmosphere conducive to inquiry, study, and learning among students, faculty, and staff. --- Satisfaction?

6 NEASC STANDARD SEVEN (continued) 3: The institution provides sufficient and consistent financial support for the effective maintenance and improvement of the institution's library, information resources, and services. It makes provision for their proper maintenance, preservation, currency, and security. It allocates resources for scholarly support services compatible with its instructional and research programs and the needs of faculty and students. 4: Professionally qualified and numerically adequate staff administer the institution's library, information resources, and services. The institution provides appropriate orientation and training for use of these resources, as well as instruction in basic information literacy. 5: The institution participates in the exchange of resources and services with other institutions and within networks as necessary to support and supplement its educational programs. It provides appropriate support for distance learning students and faculty, such as on-line reference service and contractual access to relevant off- campus library resources. 6: The institution regularly and systematically evaluates the adequacy and utilization of its library, information resources, and services and uses the results of the data to improve and increase the effectiveness of these services (satisfaction, service quality?)

7 NEASC STANDARD Four: Programs and Instruction Compliance = outcomes student learning outcomes student outcomes Undergraduate Degree Programs Graduate Degree Programs Scholarship and Research Instruction Admissions and Retention 

8 More on Accreditation Regional Accreditation- 3 use “information literacy” 3 use “equivalent language”  NCC makes the point that today’s students must be prepared to be “knowledge workers” in the sense that they must not just master certain information, but must be able to comprehend, synthesize and apply that information. More than just technologically literate, these workers will be valued for their “capacity to sift and winnow massive amounts of information in order to discover or create new or better understandings” Support for collaboration Professional and Discipline-Specific Accreditors also paying attention to information literacy Important to fields focused on Evidenced-Based Practice

9 Student Outcomes versus Student Learning Outcomes Student outcomes are aggregate statistics on groups of students (e.g., graduation rates, retention rates, transfer rates, course and program completion rates, and job placement: employment rates for a graduating class)  Such outcomes are institutional outcomes and are used to compare institutional performance  They do not measure changes in students themselves due to their college experience  these outcomes are outputs and reflect what the institution has accomplished; they do not reflect what (or how much) students learned Student learning outcomes — development of students: demonstrable acquisition of specific knowledge and skills  How well do students Transfer and apply concepts, principles, ways of knowing, and problem solving across their major program of study? Integrate their core curriculum, general studies, or liberal studies into their major program or field of study? Develop understanding, behaviors, attitudes, values, and dispositions that the institution asserts it develops?* * Maki (2004)

10 Student Learning Outcomes What should students learn? How does the content of one course relate to another? How well are they learning it? Across courses What evidence do we gather to answer these questions? Beyond graded assignments, course evaluations How do we use that evidence to improve learning Looking a courses as comprising a program

11 Purposes of Assessment Impacting student learning Working collaboratively: thinking beyond the set of courses “I” teach Demonstrating accountability and accreditation—  Meeting institutional mission  Demonstrating institutional effectiveness

12 Institutional Effectiveness Defined as how well an institution achieves its mission and major institutional goals “Since student learning is the heart of most institutional missions, the assessment of student learning is a major component of the assessment of institutional effectiveness.” However, institutional effectiveness does examine other aspects such as scholarship and research, community service, etc.  Suskie, 9-10

13 Student Learning Outcomes Conceptual Leadership Critical Thinking Creative Thinking Problem Solving Information Literacy Global Citizen Values (moral, etc.) Foundations and Skills for Lifelong Learning Skills Oral/written communication Foreign language communication Technological sophistication Quantitative reasoning ability Other Do “satisfaction” and “service quality” impact performance?

14 Direct Methods Embedded course assessment (performance on assignments, etc.; minute paper) Portfolio assessment Performance (internships, practicum, student teaching) Professional jurors or evaluators Capstone course/experience Experimental research designs), with pre- and post- testing Use of standardized tests Think-aloud protocol Directed conversation Videotape/audiotape evaluation Analysis of theses/dissertations/ senior papers (content analysis, interviews, or oral defense)

15 Indirect Methods Surveys (self-reporting) and self-assessments Curriculum and syllabus evaluation Exit interviews Observation Other

16 A culture of Assessment Evidence Evaluation

17 Have They Learned Anything?

18 So What? Or Why Evaluation Research?

19 Evaluation Research Addresses Questions Such As What improvements in a program, service, or policy might result in continuous quality improvement and better accountability? How well does a program, service, or policy reach its target population and meet the group’s information needs and expectations? Is the program, service, or policy being implemented in the ways envisioned? Is it effective?

20 Utility What are the findings good for? What decision in the real world do the findings impact? What insights will the study provide? Can the study findings be dismissed with “so what?”

21 Evaluation (for a Purpose) Feedback loop (basic systems model) Decision making Accountability Summary judgment (summative evaluation) Continuous [quality] improvement Benchmarking and Best Practices

22 Evaluation Research Is also a political and managerial activity, an input into the complex mosaic from which we can make policy decisions and allocations for the planning, design, implementation, and continuance of programs to better the human In this sense evaluation research is part of social policy and public administration

23 Evaluation Research--Steps Reflective inquiry Procedures Conduct study Analyze data Prepare/deliver report Reliability Validity Communication skills Setting up action plan

24 What are important areas for “evaluation research”? ______________ etc.

25 Evaluation of Information Services Systems Analysis Other Research Courses* *Strongly recommended for anyone planning a career in academic libraries, especially university libraries Courses in Individual Concentrations

26 Scholarly Communication Key Issues Time lag between authorship, peer review, publication, and dissemination Information overload

27 Future of Publishing Our Core Interests Competitive market Easy distribution and reuse Innovative applications of technology Quality assurance Permanent preservation (“archiving”)

28 Electronic Scholarly Publishing Some Models Digital library model Electronic book model Electronic collection model Retrospective model Preprint model Peer review lite model Commercial publishing model Academic server model

29 Electronic Scholarly Publishing Some Models (continued) Prestigious publishing model University publishing cooperative Public domain model Government server model