Seeking Synchronicity: Evaluating Virtual Reference Transcripts Presented by Lynn Silipigni Connaway and Marie L. Radford QuestionPoint Users Group Meeting June 25, 2006 New Orleans, Louisiana

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives $1,103,572 project funded by: –Institute of Museum and Library Services $684,996 grant –Rutgers, The State University of New Jersey and OCLC Online Computer Library Center $405,076 in kind contributions

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Project duration 10/1/2005-9/30/2007 Four phases: I.Focus group interviews* II.Analysis of 1,000+ QuestionPoint transcripts III.600 online surveys* IV.300 telephone interviews* *Interviews & surveys with VRS users, non-users, & librarians

Phase II: 24/7 Transcript Analysis Generated random sample – July 7, 2004 through June 27, 2005 – 263,673 sessions – 25 transcripts/month = 300 total 256 usable transcripts – Excluding system tests and technical problems

6 Analyses Geographical Distribution Library receiving query Library answering query Type of Library Type of Questions Katz/Kaske Classification Subject of Questions Dewy Decimal Classification Session Duration Interpersonal Communication Radford Classification

n=255

n=238

n=256

n=273

Service Duration Mean Service Duration: 13:53 Median Service Duration: 10:37

Transcript Reading Positive VRS experience –Duration = 1 hour 11 minutes –Academic User –Question – Boston drug company - diabetes –Relational Work –Enthusiastic user –Helpful librarian Less than positive VRS experience –Duration = 39 minutes –Middle school or high school student –Question – physics – car acceleration –Poor reference work –Extreme negative closure

Focus Group Interviews Reasons for Using VRS Convenient Efficient More reliable than search engines & free Allows multi-tasking follow-up & provision of transcript Pleasant interpersonal experience –Librarian on first name basis – more personalized Less intimidating than physical reference desk –Feel comfortable abruptly ending session

Focus Group Interviews Reasons for not using VRS Graduate students –Fear of Bothering librarian Looking stupid & advisors finding out –Questions may not be taken seriously –Potential technical problems –Bad experiences in FtF influence expectations of VRS Screenagers –Virtual stalkers (psycho killers) –Not finding a trusted librarian –Unsure of what to expect

Focus Group Interviews Challenges for Users & Non-Users Speed and technical problems Delayed response time Librarians are not in users libraries –Fear of no subject expertise Fear of overwhelming librarian

Focus Group Interviews Suggestions from Users & Non-Users Inclusion of multiple languages Access to subject specialists Better marketing and publicity –Information on how to connect and use VRS –Reassurance that users will not bother librarians – the library wants the service to be used Faster technology Improved interface design –More color –More attractive

Next Steps Conduct –Three focus group interviews – VRS users –Online survey & telephone interviews with VRS Users Non-users Librarians Analyses –Gender –User Type Child/Young adult Adult Unknown

End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives, Marie L. Radford and Lynn Silipigni Connaway, Co-Principal Investigators. Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center. Project web site: onicity/ onicity/

Questions Marie L. Radford, Ph.D. Lynn Silipigni Connaway, Ph.D.