1. Users and Librarians Reveal Critical Factors for Virtual Reference Service Excellence Lynn Silipigni Connaway, Ph.D. Marie L. Radford, Ph.D. Best Practices in Collaborative Virtual Reference ALA 2009 Annual Conference July 11, 2009 Way Sweet or Just Wrong:

2. Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, & Librarian Perspectives IMLS, OCLC, & Rutgers University funded project Four phases: Focus group interviews Analysis of 850 QuestionPoint live chat transcripts Online surveys 137 VRS Users Telephone interviews 76 VRS Users

3. Online Surveys & Telephone Interviews Descriptive statistical analysis –Demographics –Multiple choice (surveys only) –Likert-type (surveys only) Qualitative analysis –Open-ended questions –2 critical incident (CI) questions

4. VRS User Demographics: Online Surveys & Telephone Interviews (N=212) Net Gen (N=70) –Female (54%, 38) –19-28 years old (53%, 37) –Caucasian (66%, 46) –Suburban public libraries Adult (N=142) –Female (72%, 102) –36-45 years old (34%, 48) –Caucasian (85%, 121) –Suburban public libraries

5. VRS Librarian Demographics: Online Surveys & Telephone Interviews (N=275) Academic (63%, 165) Urban (56%, 150) Female (75%, 203) Caucasian (91%, 244) Broad age distribution (29%, 78) are 41-50 1-3 yrs. VRS experience (54%, 146)

6. Methodology for Analysis

7. Relational Theory & Interpersonal Communication Every message has dual dimensions -- both content and relational (Watzlawick, Beavin, & Jackson, 1967)

8. Dual Dimensions Content –Information –The what of the message –Information exchange Relational –Interpersonal –How the message is to be taken –Relationship between participants

9. Critical Incident Technique (CIT) Flanagan (1954) Qualitative technique Focus on most memorable event/experience Categories/themes emerge rather than be imposed

10. CIT Questions Remember 1 specific successful VRS interaction Remember 1 specific unsuccessful VRS interaction Describe each interaction What factors made interactions successful or unsuccessful?

11. Critical Incident Results

12. VRS Users: Whats Hot Word of Mouth Recommendation Interviews N=76 (Adult N=55, Net Gen N=21)

13. VRS Users: Whats Hot Give Me the Facts –Answer the Question –Accuracy Survey N=129 (Adult N=71, Net Gen N=48); Interviews N=64 (Adult N=42, Net Gen N=21)

14. VRS Users: Whats Hot Be On Target Be Quick Be Convenient Survey N=129 (Adult N=71, Net Gen N=48); Interviews N=64 (Adult N=42, Net Gen N=21)

15. VRS Users: Whats Hot Know Your Stuff Survey N=68 (Adult N=38, Net Gen N=30); Interviews N=34 (Adult N=26, Net Gen N=7)

16. VRS Users: Whats Hot Be Personable Be Friendly Survey N=129 (Adult N=71, Net Gen N=48); Interviews N=64 (Adult N=42, Net Gen N=21)

17. VRS Users: Whats Not Negative Closure Survey N=68 (Adult N=38, Net Gen N=30); Interviews N=34 (Adult N=26, Net Gen N=7)

18. VRS Users: Whats Not Did you already check the catalog? Survey N=68 (Adult N=38, Net Gen N=30); Interviews N=34 (Adult N=26, Net Gen N=7)

19. VRS Users: Whats Not Grumpy Librarians Survey N=68 (Adult N=38, Net Gen N=30); Interviews N=34 (Adult N=26, Net Gen N=7)

20. VRS Librarians: Whats Hot Leveraging Complex & Specialized Knowledge Survey N=143; Interviews N=86

21. VRS Librarians: Whats Hot Information Access to Remote Users Survey N=143; Interviews N=86

22. VRS Librarians: Whats Hot Positive Feedback Survey N=143; Interviews N=86

23. VRS Librarians: Whats Hot Tell Me More… Survey N=143; Interviews N=86

24. VRS Librarians: Whats Hot Morphing Modes Survey N=143; Interviews N=86

25. VRS Librarians: Whats Not Convoluted or Confusing Questions Survey N=124; Interviews N=71

26. VRS Librarians: Whats Not Imposed Queries Survey N=124; Interviews N=71

27. VRS Librarians: Whats Not Co-Browsing & Other Pesky Software Survey N=124; Interviews N=71

28. VRS Librarians: Whats Not Grumpy Users Survey N=124; Interviews N=71

29. VRS Librarians: Whats Not Unrealistic Expectations Survey N=124; Interviews N=71

30. VRS Librarians: Whats Not Disappearing Users Survey N=124; Interviews N=71

31. What We Learned VRS Users Want –Access to electronic information Convenient Accurate Timely –Interact w/ friendly librarians

32. What We Learned VRS Librarians Want –Clear questions – Query clarification User feedback –Positive User attitude –Resolved software issues –Ability to serve diverse users

33. What We Can Do Provide Convenient, authoritative, reliable services Accurate information Positive attitude

34. End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Funded by IMLS, Rutgers University, & OCLC Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/ http://www.oclc.org/research/projects/synchronicity/ Special thanks to Patrick Confer, Jocelyn DeAngelis Williams, Timothy J. Dickey, and Erin Hood

35. Questions & Comments Lynn Silipigni Connaway connawal@oclc.org Marie L. Radford mradford@scils.rutgers.edu