1. Explorations in Internet-Based Collaborative Informatics Research: A Cognitive Evaluation Vimla L. Patel, PhD, DSc Departments of Medical Informatics and Psychiatry College of Physicians and Surgeons Columbia University New York, New York

2. AsynchronousSynchronous Telephone Group Conference Calls Video Conferencing Face-to-Face Meetings Email - personal Email Dist’n Lists World Wide Web Regular Mail Communication McGill Evaluation Studies STANFORDCOLUMBIA BWH - HARVARDMGH - HARVARD Academic Research and Practice Graduate Degree Program Clinical Information Systems Medical Logic Standards Decision-Support Systems Clinical Workstations Knowledge-Based Vocabularies Cholesterol Guidelines Academic Research and Practice Graduate Degree Program Clinical Information Systems Medical Logic Standards Decision-Support Systems Clinical Workstations Knowledge-Based Vocabularies Cholesterol Guidelines Academic Research Graduate Degree Program Knowledge Representation Advanced Server Technology Decision-Support Systems Knowledge Acquisition Clinical Trials Workstations Breast Mass Protocols Academic Research Graduate Degree Program Knowledge Representation Advanced Server Technology Decision-Support Systems Knowledge Acquisition Clinical Trials Workstations Breast Mass Protocols Academic Research Medical Informatics Training Component-Based Software Clinical Workstations Integrated Guidelines Decision-Support Systems Knowledge Authoring Tools Breast Cancer Guidelines Academic Research Medical Informatics Training Component-Based Software Clinical Workstations Integrated Guidelines Decision-Support Systems Knowledge Authoring Tools Breast Cancer Guidelines Academic Research and Practice Integrated Clinical Training Decision-Support Systems Clinical Workstations Integrated Guidelines Clinical Information Systems Electronic Medical Records Flu Vaccine Guideline Academic Research and Practice Integrated Clinical Training Decision-Support Systems Clinical Workstations Integrated Guidelines Clinical Information Systems Electronic Medical Records Flu Vaccine Guideline ShortliffeCimino BarnettGreenes Patel

3. Sources of Data Electronic Interviews Audio-Taped Conference Calls Email –Current & Archived Progress Reports Papers, Presentations, and Proposals Developments at Intermed WWW Sites WWW Access Statistics –Access of Resources, Documents, etc.. –File Transfers Between Sites Audiotapes of Mini Workshop(s)

4. Perspectives on Collaboration: E-Mail Interview PLEASE COMMENT ON THE FOLLOWING TWO STATEMENTS: a InterMed is a Highly Integrated Collaborative Endeavor with Largely Shared Assumptions. Work on Common Projects Require Shared Efforts and a High Degree of InterPersonal and Intergroup Integration bInterMed is a More Loosely Coupled Cooperative Project which Each Team Provides Shared Expertise and Resources towards some Common Goal and Towards Institution-Specific Goals. Progress is Largely a Function of Individual Intiative Efforts, which are Later Coordinated within the Larger InterMed Context.

5. Modes of Communication Best Suited To Specific Purposes STANFORD COLUMBIA DSG PURPOSE

6. Categories of Activity During Conference Call

7. Frequency of Guideline-Related Email Messages per Week Email Messages per Week WEEKS FROM JULY 1994 TO APRIL 1996 July 94April 96Jan 95July 95Dec 95Feb 96 Number of Emails 0 5 10 15 20 25 30 35 40 45 1 324649546568768082848688909294 Vocabulary Guidelines Exp. HPCC Meeting, Common Guidelines Representation Intensive Work Begins on Guidelines Model/Detailed Agenda AMIA Fall Collaborative Submissions Mini Workshop on Guidelines

8. Sequence of Goals in Progress Report Specific Goals In Progress Complete MONTHS FROM JULY 1994 TO APRIL 1996 July 94April 96 Common Guideline Model Adapt And Display Guidelines on WWW Prototypical Guideline/ WWW CGI

9. CABEGFHJIMLDK Sociometric Analysis of Email Communication Patterns Stanford DSG Columbia MGH InterMed Central

10. StanfordColumbia MGH CABEFMLDK Email Communication Patterns (Jan-Feb 96) Episode 1: Vocabulary Server and Guidelines InterMed Central

11. Stanford BWH Columbia MGH CABEGFHJIDK Email Communication Patterns (Feb-Mar 96) Episode 2: Guidelines Representation InterMed Central

12. Conclusions Significant Evolution in Communication Patterns –Greater Degrees of Integration –Focused Communication Increased Use of Richer Representations –Diagrams, WWW Integrated Plans and Coordinated Actions Sustained Collaboration Requires Sustained Commitment Different Modalities — Cognitive & Social Consequences Learning Curve in Collaboration Collaboration is difficult!