1. Seeking Synchronicity: Evaluating Virtual Reference Transcripts Presented by Marie L. Radford and Lynn Silipigni Connaway 2006 ALISE Conference San Antonio, TX, January 16-19, 2006

2. Presenters Marie L. Radford, Ph.D. Associate Professor, Rutgers University, SCILS Email: mradford@scils.rutgers.edumradford@scils.rutgers.edu www.scils.rutgers.edu/www.scils.rutgers.edu/~mradford Lynn Silipigni Connaway, Ph.D. Consulting Research Scientist Email: connawal@oclc.org www.oclc.org/research/staff/connaway.htm

3. Interpersonal Dimensions of Chat Research Stream S.S. Green Award 44 transcripts (courtesy LSSI) Maryland AskUsNow! 245 transcripts (courtesy Maryland AskUsNow! Statewide Consortium) Library Lawline 113 transcripts (courtesy NELLCO Regional Consortium) Seeking Synchronicity IMLS Grant Rutgers & OCLC Todays presentation 300 transcripts (24/7, National & International) Additional 1000+ transcripts (QuestionPoint & 24/7)

4. 24/7 Transcript Analysis Generated random sample –July 7, 2004 through June 27, 2005 –263,673 sessions –25 transcripts/month = 300 total 256 usable transcripts –Excluding system tests and technical problems

5. 3 Analyses Type of Questions Katz/Kaske Classification Subject of Questions Interpersonal Communication Radford Classification Manual and NVivo coding

6. Types of Questions Combined 1 st and 2 nd Questions n=273 questions

7. Subjects of Questions Combined 1 st and 2 nd Questions n=273 questions

8. Classification Methodology Qualitative Analysis Development/refinement of category scheme Careful reading/analysis Identification of patterns Time intensive, but reveals complexity

9. Research Questions Interpersonal Communication Analysis What relational dimensions are present? Are there differences in relational dimensions/patterns of chat users & librarians? If so, what are they? How do users & librarians compensate for lack of nonverbal cues? What is the relationship between content & relational dimensions in determining quality?

10. Relational Facilitators Interpersonal aspects of the chat conversation that have a positive impact on the librarian-client interaction and that enhance communication. Relational Barriers Interpersonal aspects of the chat conversation that have a negative impact on the librarian-client interaction and that impede communication. Results Interpersonal Communication Analysis 2 Major Themes

11. Manual Coding Results 200 Transcripts –177 Usable Transcripts

12. Librarian Relational Factors (1- 200) Librarian Relational Facilitators: Manual Analysis n=177 transcripts

13. Client Relational Facilitators: Manual Analysis n=177 transcripts

14. Comparison Relational Facilitators: Manual Analysis n=177 transcripts

15. Librarian Relational Barriers: Manual Analysis n=177 transcripts

16. Client Relational Barriers: Manual Analysis n=177 transcripts

17. Comparison of Relational Barriers: Manual Analysis n=177 transcripts

18. NVivo Coding 100 Transcripts –79 Usable Transcripts

19. Librarian Relational Facilitators: NVivo Analysis n=79 transcripts

20. Client Relational Facilitators: NVivo Analysis n=79 transcripts

21. Comparison Relational Facilitators: NVivo Analysis n=79 transcripts

22. Librarian Relational Barriers: NVivo Analysis n=79 transcripts

23. Client Relational Barriers: NVivo Analysis n=79 transcripts

24. Comparison Relational Barriers: NVivo Analysis n=79 transcripts

25. NVivo Search Totals WordTotalPercent sorry2025.32 thank you5164.56 thanks3443.04 thank u33.80 n=79 transcripts Includes words in scripts

26. NVivo Search Totals WordTotalPercent sorry2025.32 thank you5164.56 thanks3443.04 thank u33.80 thanx00.00 TY00.00 ty00.00 n=79 transcripts Includes words in scripts

27. NVivo Search Totals PersonWordTotalPercent Librariansorry1417.72 Clientsorry78.86 Librarianthank you3645.57 Librarianthanks1822.78 Clientthank you2835.44 Clientthanks2430.38 Clientthank u33.80 n=79 transcripts Includes words in scripts

28. NVivo 2.0: Advantages Advantages –Move subnodes between nodes at any time Ability to create a hierarchical classification scheme –Proximity and Boolean searching –Group documents and nodes into sets for use in searching and analysis –Programmatically calculate descriptive statistics after coding

29. NVivo 2.0: Disadvantages Disadvantages –User interface does not match standard Microsoft application Cannot double click on a document or node to open Cannot scroll through a document using the mouse scroll wheel –Cannot import nodes –Cannot set certain tools to always appear Must manually display the Coder bar and coding strips each time for every document/transcript NVivo 7 to be released in Feb. 2006 –May alleviate disadvantages

30. End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives. Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center. Project web site: http://www.oclc.org/research/projects/synchr onicity/ http://www.oclc.org/research/projects/synchr onicity/