1. Seeking Synchronicity: Evaluating Virtual Reference Transcripts Presented by Lynn Silipigni Connaway and Marie L. Radford QuestionPoint Users Group Meeting June 25, 2006 New Orleans, Louisiana

2. Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives $1,103,572 project funded by: –Institute of Museum and Library Services $684,996 grant –Rutgers, The State University of New Jersey and OCLC Online Computer Library Center $405,076 in kind contributions

3. Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Project duration 10/1/2005-9/30/2007 Four phases: I.Focus group interviews* II.Analysis of 1,000+ QuestionPoint transcripts III.600 online surveys* IV.300 telephone interviews* *Interviews & surveys with VRS users, non-users, & librarians

4. Phase II: 24/7 Transcript Analysis Generated random sample – July 7, 2004 through June 27, 2005 – 263,673 sessions – 25 transcripts/month = 300 total 256 usable transcripts – Excluding system tests and technical problems

5. 6 Analyses Geographical Distribution Library receiving query Library answering query Type of Library Type of Questions Katz/Kaske Classification Subject of Questions Dewy Decimal Classification Session Duration Interpersonal Communication Radford Classification

6. n=255

7. n=238

8. n=256

9. n=273

14. Service Duration Mean Service Duration: 13:53 Median Service Duration: 10:37

15. Transcript Reading Positive VRS experience –Duration = 1 hour 11 minutes –Academic User –Question – Boston drug company - diabetes –Relational Work –Enthusiastic user –Helpful librarian Less than positive VRS experience –Duration = 39 minutes –Middle school or high school student –Question – physics – car acceleration –Poor reference work –Extreme negative closure

16. Focus Group Interviews Reasons for Using VRS Convenient Efficient More reliable than search engines & free Allows multi-tasking Email follow-up & provision of transcript Pleasant interpersonal experience –Librarian on first name basis – more personalized Less intimidating than physical reference desk –Feel comfortable abruptly ending session

17. Focus Group Interviews Reasons for not using VRS Graduate students –Fear of Bothering librarian Looking stupid & advisors finding out –Questions may not be taken seriously –Potential technical problems –Bad experiences in FtF influence expectations of VRS Screenagers –Virtual stalkers (psycho killers) –Not finding a trusted librarian –Unsure of what to expect

18. Focus Group Interviews Challenges for Users & Non-Users Speed and technical problems Delayed response time Librarians are not in users libraries –Fear of no subject expertise Fear of overwhelming librarian

19. Focus Group Interviews Suggestions from Users & Non-Users Inclusion of multiple languages Access to subject specialists Better marketing and publicity –Information on how to connect and use VRS –Reassurance that users will not bother librarians – the library wants the service to be used Faster technology Improved interface design –More color –More attractive

20. Next Steps Conduct –Three focus group interviews – VRS users –Online survey & telephone interviews with VRS Users Non-users Librarians Analyses –Gender –User Type Child/Young adult Adult Unknown

21. End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives, Marie L. Radford and Lynn Silipigni Connaway, Co-Principal Investigators. Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center. Project web site: http://www.oclc.org/research/projects/synchr onicity/ http://www.oclc.org/research/projects/synchr onicity/

22. Questions Marie L. Radford, Ph.D. Email: mradford@scils.rutgers.edumradford@scils.rutgers.edu www.scils.rutgers.edu/www.scils.rutgers.edu/~mradford Lynn Silipigni Connaway, Ph.D. Email: connawal@oclc.org www.oclc.org/research/staff/connaway.htm