CSI Cyberspace: A Multiple Case Study Investigation of the Untimely Demise of Seven Virtual Reference Services Marie L. Radford, Ph.D., Rutgers University M. Kathleen Kern, MLS, University of Illinois at Urbana-Champaign New Reference Research: 11 th Annual Reference Research Forum, ALA, Chicago June 25, 2005

Problem Statement Number of chat reference services growing chat & IM (Coffman & Arret, 2004) Alive & Clicking! Many services are successful and viewed as integral Clicked the Bucket! However, a number of chat services have died untimely deaths. Why?

Research Questions What were the deciding factors for ending these services? Who was involved in making the decision to end the services? How could these failures/false starts have been prevented? What has been the impact of discontinuing these services? What are the similarities and differences among the cases? What lessons can be learned to ensure the success of existing services or newly developing services?

Modus Operandi - Method Multiple Case Study Method Selection of Cases & Informants 7 Cases Data Collection Surveys Telephone Interviews Examination of Documents

Method Continued Analysis of Results Case Descriptions of Each Case Comparison of Cross – Case Findings Similarities Differences Iterative Process Examination of Documents

Results Despite our original & advertised title we had 7 cases not 6. 5 Academic 1 Public 1 Consortium One surprise, all 7 agreed to be interviewed & share their experiences. Another surprise – Most (5-7) had no grant funding!

Starting & Operating the Service – Common Themes No additional staff hired. Broad target audience – same as for in- person service. Distance education students a focus. Limited hours of operation. Average 22.4 hours per week User response positive.

Postmortem - 5 Major Reasons for Discontinuing Services Funding – Budget Cuts Low Use Staffing Technical Problems Institutional Culture

Reasons for Discontinuing Service Low Use FundingStaffingTech. Prob. Inst. Cult Primary Note: Totals to 8 since Case B listed 2 primary factors (Technical Problems and Low Use).

Reasons for Discontinuing Service Low Use FundingStaffingTech. Prob. Inst. Cult Primary Secondary Total

CSI Cyberspace Mystery Why did these services have such low volume?

Exhibit A - Volume CaseMonthly Qs% of total Qs A35<1% B351.5% C30<1% D363% (less if all branches included) E69<1% F Unknown G72%

Exhibit B – Marketing of Service CaseAmountComments AModerate Web, Displays, Flyers, Lib. Instr, Newspaper BExtensive Web, Branding, Articles, Lib. Instruction CModerate Target Pop., Flyers, Bookmarks, Library Instruction, Plan to Increase DExtensive Web, Newspaper, Ads, Signs, Posters, Table Tents ENone Limited hours precluded marketing push. FLittle Pilot training & exploration period. GExtensive Announcement, Meetings, Giveaways, E-Newspaper, Library Instruction

Exhibit C – Marketing of Service –Low Vol. CaseAmountComments AModerate Web, Displays, Flyers, LUI, Newspaper BExtensive Web, Branding, Articles, LUI CModerate Target Pop., Flyers, Bookmarks, LIU, Plan to Increase DExtensive Web, Newspaper, Ads, Signs, Posters, Table Tents ENone Limited hours precluded marketing push. FLittle Pilot training & exploration period. GExtensive Announcement, Meetings, Giveaways, E-Newspaper, LIU

Exhibit D – Hours of Service CaseHours of Service A M-F 3-8pm 20 hrs. per week (changed hours) B M-Th 1-7pm, F 1-5pm 28 hrs. per week C M-Th 5-8pm 12 hrs. per week D M-F 1-5pm 20 hrs. per week E Su-Th 7-10pm 18 hrs. per week F M-Th 2-8pm 24 hrs. per week G M-F 10am-5pm 35 hrs. per week

Exhibit E – Length of Service Giving VR a Fair Trial CaseTrial/PilotLength of service A1/2002 – 4/ months B1/01 – 8/029/2001 – 12/20027 months pilot 15 months C5/2002 – 4/ months D10/2001 – 4/20026 months E1/02 - 5/021/2003 – 5/20045 months pilot 14 months F9/2004 – 3/20036 months G7/2002 – 2/ months

Giving VR a Fair Trial … Shortest services ran 6 months (2 services) Longest run was 26 months Average of 17 months

Exhibit F – Service Evaluation User Eval? Type of Eval? Users Satisfd? Comments A YSurvey & Trans. YesMixed librarian perceptions B YSurvey & Assmt. YesLow (19%) return rate. C N ?Planned 2 nd yr. survey D N ? E YSurvey Yes68% “very satisfied” 40% return rate F YSurvey YesStaff satisfied too G YSurvey YesUsers highly satisfied, low return rate.

Impact of Closing Services Common Trends Demise had limited impact. Users did not notice. reference (phone service) more promotion, more use. Unique Responses Staff relieved, less pressure.

Lessons Learned If doing a multiple case study, don’t put a number in your title for conference papers. Consortium necessities: Full-time coordinator Variety of libraries Some reference questions are better answered by .

More Lessons Learned… Critical factors: Staff training & enough volume for staff to stay primed, comfortable staffing patterns User needs assessment What hours for service? What software? User evaluation Marketing Design prominent & easily identifiable link

The Mystery Remains Unsolved Ongoing Investigation… Season 2? Future Research Directions Low volume, what are critical factors? Have identified 3 additional cases. Perhaps more will be identified today. More rigorous examination of documents. Research on successful services – what factors ensured their success? User expectations of service – what hours, days? More on users. More research on non-users

Future Directions for Services Investigate free & simpler software IM or refTracker Consortia Share Cost Share Staff Continue to scan landscape

There’s Hope… Future Directions for Cases CaseReinstate?Comments AMaybe In consortium, if users want it BMaybe Simple Chat, IM, or RefTracker, Summer ’05 CYes If budget & staffing restored DMaybe Possible if consortium EMaybe Possible if consortium (doubtful) FYes Reorganization in process, hope to reopen in Jan. ’06 GMaybe Something free like IM

Conclusion PPT slides will be posted at: RUSA Reference & Statistics Committee Site Thanks & Questions??? Marie L. Radford, Ph.D., Rutgers University M. Kathleen Kern, MLS, University of Illinois at Urbana-Champaign