1. Use of Online Resources While Using a Clinical Information System James J. Cimino, MD; Jianhua Li, MD; Mark Graham, PhD, Leanne M. Currie, RN, MS; Mureen Allen, MB BS; Suzanne Bakken, RN, DNSc, Vimla L. Patel, PhD Department of Biomedical Informatics Columbia University

2. Categories of Generic Question Classification Information Needs Study Study 2 Identify information resources currently being used Characteristics of Resource Use 2A Log file analysis 2B Survey Characteristics of Information Needs Events Observational Study Study 1A Identify information needs events Study 1B Classify Information Needs Events Method Development

3. Understanding Resource Use Are clinicians using resources to answer patient-related questions? What resources do clinicians use? Do they use resources differently depending on the context?

4. Methods Log file captures clinician use of resources –Records user, function, location –User type determined from user directory –Context of use can be inferred

11. Logging Information Resource Use WebCIS Logon Screen Health Resources Page Health Resource System Log (anonymous) Health Resources Page Health Resource WebCIS Functions Logon

12. Log File Examples 16:14:38|Resource|nobody|156.1.5.1|salk.cpmc.columbia.edu/mdxdocs 16:16:46|WebCIS|ciminoj|156.1.5.1|lab^2003-03-02-05.19.00.819650 16:18:55|WebCIS|ciminoj|156.1.5.1|healthresource 16:21:18|Resource|ciminoj|156.1.5.1|salk.cpmc.columbia.edu/mdxdocs 16:24:30|WebCIS|ciminoj|156.1.5.1|lab^1997-09-02-15.54.00.359121 16:25:12|WebCIS|ciminoj|156.1.5.1|pubmed 16:25:48|WebCIS|ciminoj|156.1.5.1|micromedex 16:32:38|WebCIS|ciminoj|156.1.5.1|phar 16:25:48|WebCIS|ciminoj|156.1.5.1|micromedex

13. Log File Results Six months of log files 38,763 health resources –14,036 anonymous health resources –19,913 health resources from WebCIS –4,814 infobuttons 2,607 users –51% resident physicians –34% attending physicians –5% nurses –10% other (pharmacists, administrators, etc.)

14. Information Resource Usage 12% IBs

15. Information Use Contexts

16. Context-Specific Resource Use

17. Methods Log file captures clinician use of resources –Records user, function, location –User type determined from user directory –Context of use can be inferred Log file captures clinician use of resources –Records user, function, location –User type determined from user directory –Context of use can be inferred Questionnaire to determine: –Are they using resources through our system? –Why do they use resources? –What resources do they use? –What resources do they like? –What other resources do clinicians use? –Do survey results match log files?

18. Survey Population 1055 resources users in 01/03 250 randomly surveyed 45 (18%) returned –23 attendings –19 housestaff –3 nurses –1 pharmacist

19. Using Resources through WebCIS 14 anonymous only * 17 in WebCIS only 11 both 3 neither * *All used resources in WebCIS

20. Why do they use resources? 43 for patient-specific issues 23 for general knowledge 3 for other reasons –writing –teaching –studying

21. What resources do they use? Like? No marked difference between user types Log files corroborated usage claims 26/45 said “do not use” (although 38/45 used)

22. What about other resources? Reources used: –Up To Date –MD Consult –New England Journal of Medicine –Epocrates –Google –Books Reasons for use: 28 for patient-specific issues 25 for general knowledge 1 for teaching

23. Lessons Learned Log files representative of resource use: –Our users tend to use “Health Resources” page –Surveys reflect observed usage Micromedex is most popular resource Other resources needed (e.g. UpToDate) Resource use is for patient-specific reasons Most resource use is within patient context Laboratory results are most popular context Context does influence resource use Infobuttons actually get used! Infobuttons can be preferred!

24. Information Needs Study Study 2 Identify information resources currently being used Characteristics of Information Needs Events Characteristics of Resource Use Categories of Generic Question 2A Log file analysis Infobutton Manager 2B Survey Observational Study Study 1A Identify information needs events Study 1B Classify Information Needs Events Classification Method Development

25. Where Do We Go From Here? Up To Date has been added Infobutton Manager matches context to questions Focus on laboratory-related questions Focus on ways to exploit Micromedex Extend Infobutton Manager to other institutions www.dmi.columbia.edu/homepages/ciminoj/Infobuttons.html

26. Conclusions From the Three Studies Information needs arise while using clinical systems Needs can be detected and characterized Information needs are context-specific Information needs are stereotypic Satisfaction is often unsuccessful or deferred Resource use is context-specific Infobuttons are a viable way to resolve needs –Offer context-specific retrievals for specific needs –Provide shortcut to popular resources Infobuttons are an institution- and application- independent way to improve resolution of information needs and reduce medical errors

27. Acknowledgments National Library of Medicine research grant National Library of Medicine training grant Rick Gallagher for log file support www.dmi.columbia.edu/homepages/ciminoj/Infobuttons.html