Continuous Assessment = Continuous Improvement VALE USERS CONFERENCE, 1/9/09 Mark Thompson, Assistant Director for Patron Information Services Sidney Silverman Library Bergen Community College Paramus, NJ

Simple – Yes?? Sidney Silverman Library2

It’s Needed More Than Ever Assessment is needed Academic environments are changing; User dynamics are changing; Resource and content platforms are changing; Accountability demands are increasing; Funding sources are under stress. Sidney Silverman Library3

4 Working Premise Basic Premise If you continually assess, then you will continually find ways to improve. If you assess the core issues, then the core problems will be relieved. Versus If you assess sporadically, Then you will improve sporadically. If you assess urgent problems, then key priorities will be neglected. “Tyranny of the Urgent”

Working Assumptions We Need to address BOTH active assumptions – Assumption 1: To be effective, assessment is iterative; Research  Identify issues -  Address issues  Research  Identify remaining issues  address again  Research  ………… Assumption 2: And, perspectives must be active. Do you have active statements for? Mission / Values / Goals / Strategies If so, they can lead you to identify important priorities, assessments, and actions. Sidney Silverman Library5

But, Did Any Of It Work? YES! More usable web pagesweb pages Driven by usability studies Effective “Quiet Library Campaign”Quiet Library Campaign Developed a logo and displays Better first step: tour and tutorialstour and tutorials Benchmarks for library instruction Tracking of unique users Informed decisions on renovations New customer service initiatives Sidney Silverman Library6

Bergen Community College LIBRARY WEB: Usability Studies Thought of as many ways as possible to get user input (as quickly as possible) Students first Including Student Ambassadors Librarians as superusers AND experts Teaching faculty and administrators Adhoc: 1:1, 1:2, and small groups Tied to class schedules At Library terminals What did NOT work: Testing at end of library instruction classes

Our New Logo! Developed with input from students and staff with a focus on the positive traits necessary for success in college.

Display: Reclaim the Quiet Sidney Silverman Library9

OK, but how do I do this?... Take the strategic planning plunge; Adopt an assessment framework -- any framework! Note your progress and keep asking users: over the long term. Ask in different ways, at different times. Track your on-going issues and don’t give up. Sidney Silverman Library10

Getting to THE Idea Sidney Silverman Library11

Vantage Point: Strategic Planning Sidney Silverman Library12 Strategic planning framework Priorities of what to address Available research techniques ASSESSMENT

Strategic Planning Model © Bruce Flye—Society for College and University Planning (SCUP)

Strategic Planning Process - Details

Bergen Community College - Sidney Silverman Library15 Background Bergen Community College, Paramus, NJ 16,000 students; 50% FT/50% PT 140 different countries represented Library: open 88 hours/week; 4,000 gate count per day Library Instruction Program 275 sessions(80 min. ea.) per semester 9 librarians teach; 2 classrooms (24 PCs ea.) Learning objectives written for: ALP (i.e. ESL), EBS (basic English), Comp I, and Comp II

Sidney Silverman Library16 Bergen Community College We Adopted Assessment Framework College-wide effort on strategic planning and for assessment: Built by team which included a library manager. Began fall ’05 3 ½ years of effort And, We Ask Our Users Often LibQUAL surveys: 2005, 2008 Customer suggestions / feedback forms Other adhoc s and surveys Qualitative research 1:1, adhoc groups, expert groups Student Advisory Group

Sidney Silverman Library17 Example Framework A Framework for Assessment 6 Key Elements: 1. Choose your outcome (goal); 2. Relate to College mission/goals; 3. Select means of assessment; 1. Methodology and data sources 4. Identify criteria for success; 5. Summarize data collected; 6. Describe how results will be used. IMPLEMENT

Sidney Silverman Library18 Relevant Section of College Mission & Goals: (In this area list the relevant section of the College’s Mission and Goals) Your Department or Unit Mission: (State your unit’s mission statement.) BERGEN COMMUNITY COLLEGE Department or Unit: Year: Intended Departmental Outcome #1: (must be explicitly related to College’s M & G) Means of Assessment and Sources of Data: Criteria for Success: Summary of Data Collected: Use of Results: Intended Departmental Outcome #2: (must be explicitly related to the College’s Strategic Plan) Intended Departmental Outcome #3 :(goal related to Core Competencies, if applicable) Intended Departmental Outcome #4: A Framework for Assessment (BCC, Fall 2005) (1)(2)(3) (4) (5) (6)

It does help. Sidney Silverman Library19

Assessment’s Continuous Loop Sidney Silverman Library20 Strategic Priorities Select Goal to Assess Set Criteria for Success Which Measurements? Conduct Research Analyze Results Develop & Apply Remedies

Sidney Silverman Library21 Specific Example in Using Framework Example: Second Assessment Cycle Columns 1 & Assessment Goal: Library First two elements (from Framework): Unit Outcome #1 (Column 1) Reduce noise in the Library. Related College Statement (Column 2) “To provide supportive services and leadership opportunities in an environment that enables and encourages students to achieve…” from Goal #2, Missions and Goals

Sidney Silverman Library22 Example, con’t. Columns 3 & 4 Means of Assessment (Column 3) Analysis of LibQUAL data, Spring ‘05 Focus groups/interviews: users and staff, Summ’05 Instituted a Comments Box for feedback, Fall ’05 Conduct a follow-up survey, Winter ’05-’06 Criteria for Success (Column 4) Identify key problems with noise. Achieve a consensus over how to impact noise issues. Develop policies and methods to improve. Launch a “Noise Campaign” by 9/1/05. After which: a majority of users find the library acceptably quiet.

Sidney Silverman Library23 Example, con’t. Column 5 Summary of Data (Column 5) Observations and complaints (BLOG, , in- person), show noise problems, Fall ‘04 LibQUAL survey, April 2005 “Library as Place” rated the lowest Staff and user meetings, Summer 2005 Identified possible solutions Follow-up assessments after Noise Campaign Show higher satisfaction w/quiet

Excerpts: Summary of Data LP-2 Issue of Noise (Perceived and Desired Means) Scale of 1 low and 9 high

Excerpts LibQUAL: “Library as Place” ratings were low Comments: Noise levels too high; Lack of enforcement; Need more group and classroom areas Feedback Forms Gave Us the Specifics-- Library is Too Noisy Students talk loudly and are inconsiderate. Groups form and talk. Everyone uses a cell phone. Librarians do not enforce the rules. Noise comes from classrooms.

Example, con’t. Use of Results (Column 6) Quiet Campaign launched 9/1/05 All areas designated either Silent or Quiet zones Issues identified and Warning Cards handout out to students: Awareness builders: “yakker trackers,” flyers, & bookmarks Follow-up survey done Oct – Dec Comments Slips received; 46% gave us specific comments Good Results: 65% said it was quieter than last semester 63% said it was now quiet enough for them

Other Assessment Efforts at BCC Capture Individual user comments about any aspect of Library services or place Compare year to year Study Library Instruction Satisfaction survey: faculty and students Unique user tracking Student perspectives on topics and teaching Establish benchmarks of customer service How are we doing? Sidney Silverman Library27

Use a Variety of Measures Sidney Silverman Library28 What we are using already REF desk log statsLibQUAL Feedback forms surveys Suggestion boxStudent Advisory Group Drop in suggestionsWeb site suggestion links What we are planning BI effectiveness (pre/post)Query of new satellite campus users BI tracking surveyWeb site usability team LibQUAL 2011Customer service survey

So far: Individual User Comments Two survey efforts yielded a total of 1,290 user comments & suggestions From 2005 LibQUAL = 200 comments (43% of 468 total respondents) 549 Feedback Forms From 2008 LibQUAL = 287 comments (38% of 758 total respondents) 254 Feedback Forms

Helped Us ID: Concerns & Issues Aggregate comments indicate some problems persist. These are: availability of computers, noise, and customer service. These concerns run as a constant. Since 2005 in over 1,290 comments from LibQUAL 2005, Feedback Forms, and LibQUAL 2008.

Bergen Community College - Sidney Silverman Library31 What to Cover in Library Instruction How to Pick a Topic How to Get Library Help Evaluating Internet Sites Library Homepage Links How to Find Books Finding Articles in the Library Databases Advanced Internet Searching Accessing Library Resources from Home %70%73%59%70%80%77%73% N=176 What Students Want Covered in the Session (simple check-off, Yes/No) lowest highest

32 Libr Instr: More Lecture or Hands-on? In an ideal 80-minute library instruction class, what percent of time should be spent in lecture/demonstration versus hands-on practice?

Benchmarks & Comparisons Comparisons: Between pre-renovation survey in 2005 and post-renovation in 2008 Library usage is higher. 68% of users come in daily or weekly 53% of users go to Library website daily/weekly. Overall satisfaction remains high. On 1 to 9 (high) scale, users are satisfied with: “Overall quality of the service provided in the Library.” 7.43 “The way I am treated at the Library” avg. 7.5

Benchmarks & Comparisons Good overall results, but some causes for concern: Statement: “Giving users individual attention” is lowest rated of all factors. Disaffected or non-users: 14.3% rarely or never use library 24.1% rarely or never use library’s website And on Feedback forms: “Did you accomplish what you set out to do?” and “Was the level of service received sufficient?” -- # of positive responses is decreasing.

So…. What’s Next Take action by: Adopting a strategic planning model Take on an assessment framework Use a variety of measurement tools Follow a cycle Document the results Conduct constant follow-up Assess implemented improvements Benchmark and compare Implement change and Begin again… Sidney Silverman Library35

Contact Thank you! Questions? Mark Thompson Asst Dir, Bergen Community College Paramus, NJ Sidney Silverman Library36

Sidney Silverman Library37