1. June 22-25, 2009 i3 Conference Aberdeen, Scotland Convenience, Connections, Correctness, and Choice: Critical Components of Virtual Reference Service Quality Marie L. Radford, Ph.D. Associate Professor Rutgers, The State University of New Jersey Lynn Silipigni Connaway, Ph.D. Senior Research Scientist OCLC Research

2. Critical Components Research to understand needs of diverse user population Design services to meet users needs

3. Information Environment Rapidly changing user characteristics Information-seeking preferences Communication & behavior patterns Global economics Decrease in funding sources Ongoing budget cuts Informed decision-making critical

4. IMLS, OCLC, & Rutgers University funded project Four phases: Focus group interviews Analysis of 850 QuestionPoint live chat transcripts Online surveys 137 VRS Users 184 VRS Non-users Telephone interviews 76 VRS Users 107 VRS Non-users Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, & Librarian Perspectives

5. Online Surveys & Phone Interviews Descriptive statistical analysis Demographics Multiple choice (surveys only) Likert-type (surveys only) Qualitative analysis Open-ended questions 2 critical incident (CI) questions

7. User Demographics: Online Surveys & Telephone Interviews (N=212) Net Gen (N=70) Female (54%, 38) 19-28 years old (53%, 37) Caucasian (66%, 46) Suburban public libraries Adult, 29+ (N=142) Female (72%, 102) 36-45 years old (34%, 48) Caucasian (85%, 121) Suburban public libraries

8. User Demographics: Online Survey (N=137) Net Gen (N=49) Female (51%, 25) 19-28 years old (47%, 23) Caucasian (67%, 33) Adult, 29+ (N=88) Female (68%, 60) 36-45 years old (38%, 33) Caucasian (84%, 74)

9. Experience with Service Modes: User Online Survey (N=137)

10. Frequency of Use: User Online Survey (N=137) How often have you used chat reference?

11. Chat Least Intimidating: User Online Survey Net Gens (N=49) Adults (N=88) I am least intimidated by

12. Likely to Return? User Online Survey Net Gens (N=49) Adults (N=88) The probability that I will use reference services again is

13. User Telephone Interviews: Demographics (N=76) Net Gen (N=21) Female (62%, 13) Caucasian (62%, 13) 19-28 years old (67%, 14) Suburban public libraries Adult (N=54) Female (78%, 42) Caucasian (87%, 47) 36-45 years old (28%, 15) Suburban public libraries

14. Why Users Recommend VRS: Telephone Interviews (N=76) Would recommend VRS (89%, 68) Why? Speed & efficiency (32%, 24) Convenience (32%, 24)

15. User Telephone Interviews: Chat is Convenient (N=76) Chat 1st choice 18% (14) quick answers Why? 60% (45) convenience

16. Recommendation Important: User Online Surveys Net Gens (N=49) Used VRS because recommended Recommended VRS more than adults

17. What Attracts Users to VRS? Online Surveys (N=137) Convenience, Convenience, Convenience Available 24/7 Working from home At night or on weekends Immediate answers Lack of cost Efficient Less intimidating interactions

18. VRS Users Quotes Efficiency Yes, definitely. I already told the librarians I spoke with that I would recommend this service to anyone. It was so easy to use and very helpful. It was brief enough to answer my questions efficiently and not waste my time. I don't really want to have personal chats with librarians over the chat reference service. I really just wanted to ask my question and get an answer and that is exactly what my experience was. It was great! (UOS-30287) Net Gen The chat helped me since it was an efficient use of my time. I was able to continue working online while chatting. (UOS-65381) Adult Cost Effectiveness Yes - it's free, and convenient. The librarians are available till late at night, and they're very productive. The negative experience I described is the only one I've had, and I must have used the http://rpa.hclibrary.org/rpa/webauth.exe?rs=TUTOR over 30 times. It's an easy way to get a better understand your subject and get positive answers from a trained specialist - what more could a student ask for? :-) (UOS-39497) Net Gen Yes, quick and convenient. Cost-effective. Most libraries provide free access but parking can be expensive in the city where major libraries are located, not to mention that the closest major library to me is 60 miles away. (UOS-80908) Adult

19. Important to All Users Knowledgeable librarians Positive attitude Communication skills Valued by Screenagers Personal relationship Successful interactions

20. Non-User Demographics: Online Surveys & Telephone Interviews (N=290) Adults (N=95) Female (76%, 72) 46-55 years old (28%, 27) Caucasian (83%, 79) Suburban public libraries Net Gen (N=195) Female (64%, 124) 19-28 years old (58%, 113) Caucasian (65%, 127) Suburban public libraries

21. Non-User Demographics: Online Survey (N=184) Net Gen (N=122) Female (66%, 81) 19-28 years old (51%, 62) Caucasian (65%, 79) Adult, 29+ (N=62) Female (71%, 44) 46-55 years old (31%, 19) Caucasian (84%, 52)

22. Experience with Reference Modes: Non-User Online Survey (N=184)

23. FtF Preferred: Non-User Online Survey Net Gens (N=122) Adults (N=62) Adults (81%, 50) Net Gens (71%, 87)

24. FtF Preferred by Net Gens: Non-user Online Survey (N=122) I most enjoy using

25. Email Least Intimidating Mode: Non-User Online Survey Net Gens (N=122) I am least intimidated by

26. Critical Considerations for Info Seeking: Non-User Online Survey Net Gens (N=87) Adults (N=51) Convenience Net Gens (87%, 76) Adults (78%, 40)

27. Critical Considerations for Info Seeking: Non-User Online Survey Net Gens (N=41) Adults (N=13) Remote access is important Net Gens (95%, 39) Adults (85%, 11)

28. Critical Considerations for Info Seeking: Non-User Telephone Interview Net Gen (N=73) Adults (N=32) Internet Net Gens (45%, 33) Adults (38%, 12) Google Net Gens (15%, 11) Adults (3%, 1) Wikipedia Net Gen (3%, 2) Adults (3%, 1)

29. Interpersonal Communication is Valued: Non-User Online Surveys Net Gens (N=86) Adults (N=51) Personal Relationship Adults (43%, 22) Net Gens (24%, 24) Specific Librarian Adults (51%, 26) Net Gens (42%, 36)

30. Friendliness & Politeness Valued: Non-User Online Survey Net Gens (N=41) Adults (N=14) The librarian is friendly and polite

31. Net Gen Quote About Friendly Librarian I have once asked a librarian to point me to sources on a particular topic and the librarian personally went and pulled numerous books off the shelf…The librarian took time out of her day to help me, and actually did more than I expected. I asked for sources, and she went and retrieved the books for me…A smile and gernerally being happy prompts others to do the same. While being nice the the librarian and visa versa, this helped the general situation. The same could not be achieved in chat or text messaging. (NOS-87254) Net Gen

32. Net Gen Quote About Unfriendly Librarian It was awhile ago, but I asked the reference librarian where to find books relating to a certain topic I was studying in school at the time and she just kind of said "over there" and pointed…She did not seem engaged or interested in truly helping me find the books and didn't really care that I never found them, I was wondering all over looking and she just sat there…No I think it was more the person I was dealing with, I doubt she would have been any more helpful in another format and I don't care if she would have been great at texting etc. because at the time, I was there looking for books and just wanted to know specifically where they were located. She did not seem interested in helping me, let alone exhaust all of her means for doing so. Ever since then, I usually avoid that person and go to the one who has helped me successfully. (NOS-69073) Net Gen

33. Why They Do Not Choose VRS: Non-User Online Survey Net Gens (N=122) Adults (N=62) Too complicated Adults (53%, 33) Net Gens (35%, 43) Typing skills poor Adults (35%, 22) Net Gens (16%, 19)

34. Why They Do Not Choose VRS: Non-User Online Survey Net Gens (N=122) Adults (N=62) Believe questions might annoy librarian Net Gens (29%, 32) Adults (16%, 10)

35. Why Net Gens Do Not Choose VRS: Non-User Online Survey (N=122) Dont know it is available Believe librarian couldnt help Lack of 24/7 service Satisfied w/ other info sources

36. Net Gen Non-Users Quotes Lack of service availability at all hours After hours help. If the library is closed and e-mail takes 24+ hours to respond, I would try a chat if there is an available librarian on after closing hours. (NOS 87254) Net Gen I would be convinced if it were offered 24/7. This makes it acccessible all the time. That's convenient. (NOS 39998) Net Gen If it is available 24/7 I'll try it. (NOS 84519) Net Gen

37. Why Adults Do Not Choose VRS: Non-User Online Survey (N=62) Same as Net-Gen, but also… Lack computer skills Type slowly Complexity of chat environment

38. Adult Non-Users Quotes Beliefs that the librarian could not help them I do not see myself using chat reference services because in the absence of having a reference librarian help me locate an appropriate or required source, I have friends in the LIS discipline with exemplary reference/research skills who could help me. Additionally, because I am in research, I have cultivated my own knowledge base of where/how to track down information. The only time I could ever imagine using chat reference is if I were incapacitated or unable to physically be in a library or if I were unable to reach aone of my LIS colleagues. Otherwise, I see myself as a self-sufficient resarcher who relies on her own social network and knowledge to locate reference material. (NOS 91666) Adult I have only used a reference librarian once, as I enjoy doing my own research. I am as likely to use chat or email as an in-person service. I don't have an issue with the format; I just don't see a need or value-added to using a reference library service in most searches. I have been able to find what I need relatively efficiently on my own in acceptable timeframes. (NOS 22870) Adult Satisfaction with other information sources I don"t know how to access computer library service. When I need to look somthing up I use google. (NOS 61939) Adult

39. Non-User Demographics: Telephone Interviews (N=106) Net Gen (N=73) Caucasian (66%, 48) Female (59%, 43) 19-28 years old (70%, 51) Suburban public libraries Adults (N=33) Caucasian (82%, 27) Female (85%, 28) 29-35 years old (27%, 9) Suburban public libraries

40. Non-User Telephone Interviews (N=107) FtF Preferred (24%, 26) Do not know VRS exists (16%, 17) Alternatives to library Internet (43%, 45) Personal convenience (38%, 39)

41. Non-User Telephone Interviews: Why Might They Try VRS? (N=107) When asked what might convince them to ask for help from VRS: 30% (n=32) cited a need for immediate answers It would be convenient, because if I was sitting at a computer and I could ask a question and they would answer immediately… that would be good.. Convenience is why I do something as opposed to something else. (NTI 131) Net Gen 17% (n=18) would value accessing VRS from home And if there were somethng like that for the library system a lot of people would be using it, really late at night when the library is closed or there is no reference librarian. (NTI 66) Net Gen

42. Relational Theory and Interpersonal Communication Every message has dual dimensions -- both content and relational (Watzlawick, Beavin, & Jackson, 1967)

43. Dual Dimensions Content Information The what of the message Information exchange Relational Interpersonal How the message is to be taken Relationship between participants

44. Critical Incident Technique (CIT) Flanagan (1954) Qualitative technique Focuses on most memorable event/experience Allows categories or themes to emerge rather than be imposed

45. CIT VRS Users Questions Remember 1 specific successful VRS interaction Remember 1 specific unsuccessful VRS interaction Describe each interaction Identify factors that made interactions successful or unsuccessful

46. VRS User Positive CIs Online Survey Net Gens (N=48) Successful Experience Librarian Accurate answers/info (29%, 14) Quick assistance (13%, 6) Located specific resources (19%, 9) Convenient (21%, 10)

47. Facilitators, Relational Themes – JQs Convenience Theme – User The chat format did help my experience to be successful because I was able to multi-task while the librarian did the search. If I was in a FtF situation, I may have gotten frustrated with the amount of time it took for the librarian to find what I needed and would have just said never mind." (UOS 43528) Adult Relationship Quality Theme - User The librarian was able to guide me through a research problem clearly and thoroughly, assisting me step-by-step. The librarian helped me step-by-step, instead of rushing me through, she was able to work slowly with me. Yes, because I was able to view a transcript of the results at the end of the chat. (UOS 25429) Net Gen

48. Facilitators, Content Themes – JQs Providing Instruction Theme - User I was looking for achievements of [the] Celtic civilization. I needed one more category. I came out not with the answer, but the MEANS of finding the answer. …I managed to find a topic on which there was SO much, I could hardly believe I missed it. (UOS 51182) Age 12-14

49. User Negative CIs Online Survey Net Gens (N=30) Unsuccessful Experience Librarian Impeded info delivery or retrieval (67%, 20) Didnt answer question (30%, 9)

50. Barriers, Content Themes – JQs Information (Lack of) Theme – User I needed information on the West for a book a student was reading. The person did not listen to the question and gave the wrong information. The person did not listen to my needs and did not answer the question. (UOS 33261) Adult

51. Critical Incidents Non-Users Online Survey Questions All VRS Non-Users were asked to Think about one experience in which you felt you achieved (or did not achieve) a positive result after seeking library reference services in any format. Think about one experience in which you felt you did not achieve a positive result after seeking library reference services in any format. Describe each interaction Identify the factors that made these interactions positive or negative

52. Non-User Positive CIs Net Gens (N=108) Successful Experience Librarian Info delivery/retrieval (50%, 54) Answered questions (14%, 15) Located specific resources (20%, 22) Positive attitude (them & task) (36%, 39)

53. Facilitators, Content Themes – JQs Demonstrating Knowledge Theme – Non-User I was looking for books on theoretical physics. My question was, What would be the latest and most comprehensive book on quantum electrodynamics? I felt the encounter was successful because the librarian apparently had a background in physics. He was up to date in his physics knowledge and was aware of the latest books. (NOS 98125) Net Gen

54. Non-User Negative CIs Online Survey Net Gens (N=74) Unsuccessful Experience Librarian Impeded information delivery or retrieval (64%, 47) Missing resources (12%, 9) Slow providing answers (11%, 8) Negative attitude to task (47%, 35)

55. Barriers, Relational Themes – JQs Relationship Quality Theme – Non-User I tried to explain that I wasn't interested in doing a general search on my topic, but that instead I needed this specific article, but she never really listened, and instead I ended up wasting a significant amount of time. The librarian was so overzealous with helping me that she lost sight of what I actually needed, which in this case was quite limited in scope, a specific item. (NOS 32648) Adult

56. Barriers, Content Themes – JQs Lack of Knowledge Theme – Non-User She was only able to start the program, a step I already knew how to do. However, she was not knowledgeable about the software that was on the computers at the library. (NOS 67314) Net Gen Instruction (Lack of) Theme – Non-User I went to get help to access the databases and the librarian just handed me a piece of paper with instructions. The instructions were not clear and I found them rather confusing. (NOS 67443) Adult

57. What We Learned FtF & VRS Users want Extended service hours Access to electronic information Interact w/ friendly librarians Personal relationship with librarians

58. What We Can Do Provide Variety of service modes Convenient, authoritative, reliable services Accurate information

59. What Else We Can Do Creative marketing Promote full range of options Reassure young people VRS safe Build positive relationships whether FtF, phone, or online

60. End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Funded by IMLS, Rutgers University, & OCLC Online Computer Library Center, Inc. Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/ http://www.oclc.org/research/projects/synchronicity/

61. References Connaway, L. S., Radford M. L., Dickey, T. J., Williams, J. D., & Confer, P. (2008). Sense-making and synchronicity: Information- seeking behaviors of millennials and baby boomers. Libri, 58(2), 123- 135. Radford, M. L., & Connaway, L. S. (2007). Screenagers and live chat reference: Living up to the promise. Scan, 26(1), 31-39. Radford, M. L., & Connaway, L. S. (2005-2008). Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives. Funded by the Institute of Museum and Library Services (IMLS). Available: http://www.oclc.org/research/projects/synchronicity. http://www.oclc.org/research/projects/synchronicity

62. June 22-25, 2009 i3 Conference Aberdeen, Scotland Questions & Comments Lynn Silipigni Connaway connawal@oclc.org Marie L. Radford mradford@scils.rutgers.edu