1. Using the Internet to Answer Clinical Questions
STFM Spring Conference
Brian S. Alper, MD, MSPH
Anne-Marie Lozeau, MD, MS
Beth Potter, MD

2. Disclosure
“A funny thing happened to me on the way to rural family practice…”
Dr. Alper is Editor-in-Chief of DynaMed™ and Medical Director of EBSCO Publishing

3. Learning Objectives
Recognize that using the Internet to meet clinical information needs is practical
Meet many useful web sites
Learn efficient strategies for selecting useful resources for specific needs
Use a free Internet portal to organize Internet use during patient care

4. Our Goal in Medicine Provide the best care…
Provide patients the best information to guide health care decisions…
Improve health outcomes…
…based on the “truth” – separating medical knowledge from folklore

5. How do we know medicine? WE = society MEDICINE = clinical knowledge
Scientific investigation
Original research published in journals
Systematic reviews
EVIDENCE-BASED MEDICINE

6. How do we know medicine? WE = individual clinicians
MEDICINE = clinical knowledge
Consultants Colleagues
Lectures Textbooks
Rounds Precepting
Guidelines CME
Experts Experience
PRACTICAL choices selected for efficiency

7. “Just-in-Case” vs. “Just-In-Time”
Apprentice-style
Memorization
Soak up information now
Retrieve information during practice
Knowledge Mastery
Find information when you need it
Adult learning
Problem-oriented
Clinical application

8. Using Traditional Textbooks
Not always written for clinical practice
May not provide best research evidence (Are opinions well supported, or selectively supported?)
May be years out-of-date
Finding specific information within the text can be time-consuming

9. Using Guidelines Not always written for my clinical practice
May not provide best research evidence (Are opinions well supported, or selectively supported?)
May be years out-of-date
Finding specific information within the text can be time-consuming
May be unavailable for specific question

10. Using Journal Articles
Individual journal articles
may not relate to specific information need
may not provide complete picture
Finding one journal article can be time-consuming, let alone finding all the relevant articles
Articles are written to promote research findings, not often written for clinical application
BUT THIS IS OUR EVIDENCE SOURCE

11. Clinical Questions are Common
Typical primary care physician has
2 questions for every 3 outpatient visits
3 questions for every 1 inpatient admission
Ideal: Answer these questions with the best available evidence.

12. Previous research on answering clinical questions
MEDLINE searches can answer clinical questions from primary care physicians
43% and 46% questions answered by medical librarian searches
Mean search times were 27 minutes and 43 minutes
J Fam Pract. 1996;43:
Bull Med Libr Assoc. 1994;82:

13. Barriers to Finding Best Evidence
6 most important obstacles to answering doctor’s questions with evidence
Time required to find information
Difficulty modifying original question
Difficulty selecting optimal search strategy
Failure of resource to cover the topic
Uncertainty when all relevant evidence found
Inadequate synthesis of multiple bits of evidence
BMJ 2002 Mar 23;324(7339):710

14. Result Busy clinicians use references considered
Fast
Likely to provide answers
PREFERRED over evidence databases
By family physicians (BMJ. 1999;319: )
By family physician residents (Fam Med. 2003;35: )

15. The Need An evidence database must be:
Fast
Able to answer most questions
To make EBM feasible in routine practice.

16. Evidence-Based Medicine
Definition: Integration of best research evidence with clinical expertise and patient values.
Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine. How to Practice and Teach EBM. 2nd ed. London: Harcourt Publishers Ltd p. 1.

17. Best Research Evidence
Comprehensive – evidence only known to be best if all the available evidence is known
Valid – critical appraisal determines potential for bias
Systematic – selection and evaluation of evidence by protocol reduces author bias, investigator bias
Current – every day new evidence could be best
Synthesized – one study vs. the whole picture

18. EBM is not cookbook medicine
“I know a lot, therefore I am” (replaceable by a computer)
“I think, therefore I am”
(never replaceable by computer)

19. EBM is not cookbook medicine
Evidence does not directly advise individual patients
Inclusion and exclusion criteria
Co-morbidities
Clinicians can use evidence to inform care of individual patients
Individual patient values differ
Balance of risks and benefits vary with individual

20. Ideal Sources for Clinicians
Comprehensive – covering all relevant articles AND information needs.
Valid/Systematic – driven by best available evidence.
Relevant – focus on clinically relevant information.
Easy to use – organized for use by clinicians.
Affordable

21. Validity (“Truth”) Consider validity of original research
PLUS validity of translation
Was the best evidence identified?
Was the best evidence selected?
Was the evidence evaluated well?
Is the summary accurate?
Are conclusions based on the underlying evidence?

22. Evidence-Based = conclusions based on best available evidence
“Evidence-based clinical reference” requires the following:
Systematically identifying the evidence
Systematically selecting the best available evidence from that identified
Systematically evaluating the selected evidence (critical appraisal)
Objectively reporting the relevant findings and quality of the evidence
Synthesizing multiple evidence reports
Deriving overall conclusions and recommendations from the evidence synthesis
Changing the conclusions when new evidence alters the best available evidence

23. Systematic Evidence Identification
Systematic Literature Search
Commonly called Systematic Review
Start with question, search for best available answers
Systematic Literature Surveillance
Also called Research Monitoring
Start with literature being published, determine how best to use

24. Systematic Literature Search
Start with clinical question
Define inclusion criteria
Search for all relevant information
Assess methodology (validity)
Combine results
Determine clinical conclusion
Apply to practice

25. Using Systematic Reviews to Answer Clinical Questions
Cochrane Reviews
may be “gold standard” when available and current
many sources use Cochrane reviews
Other Reviews - check whether process (search, evaluation) is systematic

26. Using Systematic Reviews to Answer Clinical Questions
More difficult for non-RCT answers
Consider additional info
Most valid method for answering an individual question
Not adequate to meet most information needs

27. Systematic Literature Surveillance
Start with defined literature set
Define inclusion criteria
Select all relevant information
Assess methodology (validity)
Summarize results
Determine clinical conclusion

28. Systematic Literature Surveillance - Uses
Alert clinicians to critical info
(EBM article summaries)
Catalog for future searches (EBM search engine)
Update clinical reference information (Integrated EBM)

29. Systematic Literature Surveillance - Advantages
Cover all types of information
- Diagnosis - Treatment
- Etiology - Prognosis
Keep up with current research
Address current questions

30. Systematic Literature Surveillance - Disadvantages
Can miss the big picture
1 study may be “rebuttal”
“ping-pong” between sources
overemphasis of study results
Need to read collection of studies
Miss older research (unless retrospective literature surveillance)

31. Ideal Sources
Comprehensive – covering all relevant articles AND information needs.
Valid/Systematic – driven by best available evidence.
Relevant – focus on clinically relevant information.
Easy to use – organized for clinician uses.
USE SYSTEMATIC REVIEWS AND SYSTEMATIC LITERATURE SURVEILLANCE

32. Useful Links for Patient Care
Internet portal: “Dr. Alper’s links”
Links to all resources in this presentation
Organized for effective searches
A starting point for use during patient care
You can make your own!

33. Internet Portals: Useful Links for Patient Care
Organized for effective searches
A starting point for use during patient care
Access from any computer with internet connection
Links to all resources in this presentation
You can make your own!

34. Examples of Portals My HQ: www.myhq.com
“Dr Alper’s links”
I keep bookmarks

37. * Asterisk
- Requires Registration
- MAY Require $$ Subscription

38. EVIDENCE-BASED MEDICINE
Highly Valid EBM sources
Relevant - many have direct clinician input
FIRST CHOICE if fast enough (many are)
FIRST CHOICE when research likely
Quality research is possible?
Research is present? (difficult to guess)

41. Clinical Scenario
52-year old male presents with Shingles (zoster) for 2 days.
Clinical Concerns:
short-term symptom relief
prevention of Post-Herpetic Neuralgia

42. Evidence-based Search Engines
Include systematic reviews
Add other useful sources to increase chances of finding information where systematic reviews unavailable
“medical content” searching instead of using Google or Yahoo

43. Using EBM Search Engines
Examples: TRIP, InfoRetriever
Broader than Systematic Reviews alone
Limitations:
individual article summaries may miss the “big picture”
may have to read multiple items
No subject overview

44. TRIP Database Turning Research Into Practice
Identifies free websites with EBM and other medical content
Catalogs titles and URLs to each page
Inclusion based on Relevance and Rigor
300+ new articles added monthly
Searches synonyms, cues misspelling

45. TRIP Database EBM Resources: systematic reviews, CATs, journal clubs
Clinical Guidelines
Query-answering
Medical Images
eTextbooks: background information
Patient Information
Journals – automatically searches PubMed with EBM filters

53. TRIP Database Comprehensiveness: high Validity: variable
Relevance: moderate
Convenience: easy to use
Affordability: free

54. InfoRetriever POEMs Cochrane Abstracts Clinical Guidelines
Clinical Decision Rules
Diagnostic Test Calculators
H&P calculators
ICD-9 search tool
5-Minute Clinical Consult

64. InfoRetriever Comprehensiveness: moderate Validity: variable
High for most referenced content
Low for 5-Minute Clinical Consult
Relevance: high
Convenience: easy to use
Affordability: $249/year

65. Clinical Evidence Highlights 230 common clinical topics
Lists treatments for those topics
Qualitatively describes Systematic Reviews and RCT data for and each treatment
Compiles results for easy use
Includes relevant background information
Updates searches every 12 months
Peer Reviewed

72. Key sources for Systematic Reviews
Cochrane Library
DARE
Major benefit: best evidence (at time of review)
Major limitation: limited scope of coverage
Clinical Evidence
AHRQ

73. Clinical Evidence Comprehensiveness: modest
Validity: high - limited to RCTs
Relevance: high - clinical focus
Convenience: easy to use
Affordability: $239/year ($120 students)

74. PubMed Clinical Queries
Use search filters that have been researched by Haynes, et al. who are involved with the McMasters EBM group
Help to refine EBM searches in Pubmed

82. Clinical Queries Comprehensiveness: moderate Validity: High
Relevance: Low
Often finds many articles
Still need to sift through a lot of articles
Convenience: takes practice
New feature allows you to receive updates when new info is added to your search
Affordability: Free

83. National Guideline Clearinghouse
Guidelines search engine
FREE -
Links available in other sources
Many guidelines are not “evidence-based”, even when labeled “evidence-based”
Levels of evidence used vary widely across guidelines

88. National Guidelines Clearinghouse
Comprehensiveness: moderate
Validity: Low to High
Not all guidelines are EBM
Some guidelines are expert -based
Relevance: Low
Often finds many guidelines
Convenience: takes practice
Affordability: Free

89. DynaMed
Nearly 2,000 topics combining best available evidence with clinical focus
Updated continuously with systematic literature surveillance
> 500 Journals directly or via multiple journal review services
Cochrane reviews
National Guideline Clearinghouse
Template system for rapid browsing

90. DynaMed Peer Review in process
Includes ICD-9 Codes and patient handout links for each topic
Easy to navigate
Alphabetical topic browse
Standardized templates
Some research-heavy topics can get difficult to navigate UNTIL new topics created

101. DynaMed
Comprehensiveness: shown to answer 69% of clinical questions in primary care in 2004
Validity: variable & explicit
Relevance: high, clinical focus
Convenience: easy to use
Affordability: $350/year or effort ($150 residents, $100 students)