1. VALE REFERENCE SERVICES COMMITTEE January 5th, 2011
Embracing Change at the Reference Desk: Data Collection Methods and Tools in NJ Academic Libraries
VALE REFERENCE SERVICES COMMITTEE
January 5th, 2011

2. VALE MEMBER LIBRARIES -
52 academic institutions
(68 physical locations on the map;
Some institutions operate more than one library )

3. ACRL DEFINITION:
“A reference transaction is an information contact that involves the knowledge, use, recommendations, interpretation, or instruction in the use of one or more information sources by a member of the library staff….”
“…Duration should not be an element in determining whether a transaction is a reference transaction”….. (ACR)

4. TIMELINE February 14, 2011: Survey launched
Mass to NJ Directors list
February 23rd: survey re-sent
March 3rd: individual s sent
Responses received from 37 libraries (including 5 Rutgers libraries)
OF THOSE THAT RESPONDED, TWO LIBRARIES DO NOT COLLECT REFERENCE STATISTICS
30 INSTITUTIONS (58% of VALE members)
35 individual libraries (5 Rutgers libraries)
13 PUBLIC; 12 COM. COLl.; 10 PRIVATE

5. QUESTIONS ASKED
1)  What kind of data is collected in your library (libraries)?
2)  How do you define different reference transaction categories?
3)  Are statistics collected only at the reference desk?
4)  Do you collect the data on a daily basis or as a sample (once a week, once a month)?
5)  Do you register each question or rather each reference transaction?
6)  Do you use an electronic tool to collect the data? Is so, what type of software program is used?
7)  How is the collected data being used?

6. Question 1: Types of data being collected
Three main categories with a variety of terms:
Research: reference, extended reference, demonstration &/or instruction, consultation, professional, research levels;
Directional: information, ready reference, quick reference, basic, activity;
Technology related: computer assistance, technical support;
Other types:
Virtual (electronic) reference includes: IM, Meebo, Q&A NJ, text-a –librarian, blog visits, chat, ;
Phone
Assignments
Government documents

7. Question 1 cont.:
Research/Reference – 34 (1 library did not identify this category)
Directional (5 libraries either do not collect directional encounters or did not specify this category) Technology - 15
/Virtual Ref
Phone - 12
Glouchester: Directional and research transactions and within these two categories further subdivide whether “in-person, phone, or .” Under this division, we further separate by directional/computer-related, and directional/non-computer; research questions we also separate by computer-related and non-computer/print;
Ramapo: Quick Reference questions (less than 10 minutes)
In-Depth reference questions (more than 10 minutes)
Directional/Technical (e.g. use of photocopiers) questions
Reference questions through telephone
Directional/Technical through telephone
Meebo chat reference
Rfaritan Valley: Three tiers of questions: Basic/Directional; Reference/Demonstration; Research/Instruction.
We also track: Online questions (chat, ) and Phone questions. These questions are recorded twice – once for the transaction (phone or online) and once in the appropriate tier (Basic, Reference, Research).
TCNJ: directional and professional and extended professional questions, each category subdivided by phone, /chat, or in-person.
William Paterson: Data is collected based on time of day including walk-in, telephone and chat.
We also collect data based on individual appointments and other non-ref-desk encounters including .

8. OBSERVATIONS: MULTIPLE MODELS IN DATA COLLECTION
Main categories versus tiers or levels (level 1, 2, etc.)
Main categories and subcategories (general ref. vs. extended ref., “2 types of reference”)
General categories vs. specific categories (i.e. research assistance and consultation)
Virtual (phone) versus on-site reference
General categories divided into time periods (under 15 minutes, over 20 minutes etc.)
Outliers: “…data (what type of data?) is collected based on time of day including walk- in, telephone and chat.”
Glouchester: Directional and research transactions and within these two categories further subdivide whether “in-person, phone, or .” Under this division, we further separate by directional/computer-related, and directional/non-computer; research questions we also separate by computer-related and non-computer/print;
Ramapo: Quick Reference questions (less than 10 minutes)
In-Depth reference questions (more than 10 minutes)
Directional/Technical (e.g. use of photocopiers) questions
Reference questions through telephone
Directional/Technical through telephone
Meebo chat reference
Rfaritan Valley: Three tiers of questions: Basic/Directional; Reference/Demonstration; Research/Instruction.
We also track: Online questions (chat, ) and Phone questions. These questions are recorded twice – once for the transaction (phone or online) and once in the appropriate tier (Basic, Reference, Research).
TCNJ: directional and professional and extended professional questions, each category subdivided by phone, /chat, or in-person.
William Paterson: Data is collected based on time of day including walk-in, telephone and chat.
We also collect data based on individual appointments and other non-ref-desk encounters including .

9. Question 2: Definition of Reference Transaction (Category )
Research – more than 5 min.;
Extended time – over 15 min.;
“Reference transaction -questions that involve the use, recommendation, and interpretation of library resources;”
Research - means finding data, helping with search skill sets, search criteria - no time limit. 
“use, recommendation, interpretation, or instruction of one or more information sources;”
Research levels from 1 to 4;
“information contact that involves the use of one or more information sources;
consultation = reference question: skill base, strategy based or non- resource;
“we try to follow the definitions used in the ACRL statistics gathering to differentiate directional from professional.”
. Some of the definitions just use slight variations on the ACRL definition, which I don’t see as a big problem, but some of them use duration or level of difficulty while others do not. However for data comparison purposes you can always group more finely-grained data back into larger sets. For example I could group Rowan’s Level 2, 3, and 4 questions into Research, and put our Level 1, Directions and a few others into Directional. So the main issue is that you would have to pre-process some of the data in order to make cross-library comparisons.

10. Question 3: Place(s) where data is collected

11. Question 3: Locations - issues
Reference desk was omitted in several responses:
“reference service ( and chat) are collected separately,” or
“Off desk/offices s, phones, consultations”
Reference encounters (rowing?, appointments?)
Phone / separately at the Ref. Desk
Ref. desk include in person, phone and ; Chat separately, Libguides and dbase e-book use separately, but:
Ref. desk, s, chat reference, library blog, phone..
Libguides/database usage/e-book usage –probably not complete count – not everyone considered it a reference transaction (is this a reference transaction statistic?).

12. Question 4: Frequency of data collection
SAMPLE (7 libraries):
Three weeks during semester
3 week period during the Fall semester
Once a week
3 sample weeks spaced out during each semester
2-3 times a semester
sample week once per year
Four weeks a year
MONTHLY (1) - cumulative
DAILY (27 libraries – 77%):
Daily in three intervals
Hourly / Daily
Collected daily, reported by sampling

13. Question 5: Transaction vs. Question
Transactions BUT (3 out of 23 libraries are not clear):
“If it is a reference transaction than it is recorded as a reference transaction; should additional assistance for another service be needed, that too is recorded under the appropriate topic.”
“If the students work on several papers, then questions.”
“Each transaction but it’s usually at the discretion to the reference librarian how to record it.”

14. Question 6: ELECTRONIC TOOL TO COLLECT DATA
ELECTRONIC TOOLS IN USE:
Google Tools/Google Docs
In house Access dbase
Excel & “spreadsheet” - 9
NONE, but:
“looking into developing home grown program”
LibAnswers soon
“our data librarian has developed a simple program that can be used at the Reference Desk.. which we're looking to implement”
Gimlet trial version
None 23
Excel 5
Paper to spreadsheet 4
Other 3

15. Question 7: HOW IS DATA BEING USED

16. Question 7: HOW IS DATA BEING USED

17. FEW FORMS/SYSTEMS CURENTLY IN USE

18. Question 7: How is this data used
What kind of:
“Annual reports“
“Reports to various associations”
“We report them to library associations that require them”
“To understand the value of our reference and research services”
“Assessment ” (staffing, col. development, instruction?)
VAGUE

19. WHAT HAVE WE LEARNED FROM THIS QUICK SURVEY ?
Results – snapshot showing how academic libraries define, collect and use reference data
Data is being collected in three main categories with a variety of terms
Definitions of reference transactions vary
Reference desk still leading location for collecting data
Daily - preferable frequency for data collection
Transactions rather than questions
Electronic tools – few; homegrown rather than commercial
Data used for reports, but also for staffing, coll. devel., assessment
Next survey - clarification of terms necessary !

20. STEVENS INSTITUTE OF TECHNOLOGY

21. GEORGIAN
COURT

22. RUTGERS - KILMER

23. RARITAN VALLEY

24. DREW UNIVERSITY REFERENCE STATS FORM

25. Berkeley College Gimlet.us

26. Gimlet.us

27. Gimlet.us

28. COMERCIAL SOFTWARE ON MARKET
LibAnswers:
Libstats:
Desk Tracker:
RefTracker:
Gimlet:

29. Surveys on Reference Services
Novotny, Eric, and Washington, DC. Office of Leadership and Management Services. Association of Research Libraries. "Reference Service Statistics & Assessment. SPEC Kit." SPEC Kit 268 (2002): ERIC. Web. 4 Jan
2009 VALE Reference Services Survey Report, July 2010; prepared by Tony Joachim, edited by Jody Caldwell SurveyReport_final_0.pdf

30. PANEL & DISCUSSION THANK YOU Panelists: Moderator
Denise Brush - Rowan University
Jennifer Heise - Drew University
Pat Dawson Rider University
Moderator
David McMillan – Caldwell College
THANK YOU
Maria Deptula
Berkeley College