Updating Clinical Knowledge in the Information Age: Rating the “Foraging” Tools Scott Strayer, MD, MPH Assistant Professor University of Virginia Department of Family Medicine Charlottesville, VA Allen F. Shaughnessy, PharmD Adjunct Professor of Family Medicine Tufts University Family Medicine Residency Tufts University School of Medicine Boston, Massachusetts Marit L. Kington, MS David C. Slawson, MD B. Lewis Barnett, Jr. Professor of Family Medicine

Maybe insert picture of electronic library interface from UVA Maybe insert picture of electronic library interface from UVA. Nowadays, this is how I get my information (e.g. the electronic library)

Make contrast of most of us not even physically going to library anymore…particularly contrast between practicing docs (e.g. AAFP participants) and researchers.

Introduction It still takes medical findings several years before they are translated into clinical practice.1, 2 Clinicians cannot keep up with medical advances as there are over 20,000 articles per year in 120 clinical journals and the number of randomized controlled trials has increased from 2,000 to 12,000 per year in the past two decades.3 1. Fineberg HV. Clinical evaluation: how does it influence medical practices? Bulletin du Cancer 1987, 74: 333-346 2. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalamers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infraction. JAMA 1992; 268:240-248. 3. Ebell MH, Barry HC, Slawson DC, Shaughnessy AF. Finding POEMs in the medical literature. J Fam Pract. 1999 May; 48(5):350-355.

Two Tools Needed to Master Clinical Information- BMJ 1999 A method of being alerted to new clinical information (a “foraging” tool) A tool for finding the information again when you need it. (a “hunting” tool) Without both: You don’t know that new info. is available You can’t find it when you do Clinical example- Riboflavin for migraines Shaughnessy AF, Slawson DC. Are we providing doctors with the training and tools for lifelong learning? British Medical Journal 1999 (13 Nov): www.bmj.com. (http://bmj.com/cgi/reprint/319/7220/1280.pdf) Add better definitions (e.g. from academic medicine paper).

Criteria for a High-Quality Foraging Tool A high-quality foraging tool employs a transparent process that filters out disease-oriented research and presents only patient-oriented research outcomes, demonstrates that a validity assessment has been performed using appropriate criteria, assigns levels of evidence, based on appropriate validity criteria, to individual studies, provides specific recommendations, when feasible, on how to apply the information, placing it into clinical context, comprehensively reviews the literature for a specific specialty or discipline, and coordinates with a high-quality hunting tool. Slawson DC, Shaugnessy AF. Teaching Evidence-Based Medicine: Should We Be Teaching Information Management Instead? Academic Medicine; 80:685-689.

Information Mastery and Information Tools Not always assessed by information tools Usefulness = Relevance x Validity Work Can be reduced by Information tools----but not always Slawson DC, Shaughnessy AF, Bennett JH. Becoming a Medical Information Master:Feeling Good About Not Knowing Everything. The Journal of Family Practice 1994;38:505-13.

What is the quality of currently available clinical information foraging tools?

Methods Identified foraging tools for inclusion between May and September 2004. Defined foraging tools as any method of communicating updates in medicine (e.g. web-based, e-mail, paper, software) Excluded drug-only tools Excluded tools that were not updated at least quarterly

Search Strategy Various medical sites (e.g. Medscape, MD Consult, WebMD, Journal Watch, etc.) Searched well-known evidence based databases (e.g. Cochrane, TRIP, Clinical Evidence, FPIN) Monitored list-serves and invited list-serve members to make suggestions (e.g. Palm-Med, STFM EBM) Consulted experts and clinicians

Identified Foraging Tools Accompany Medical Journals (4) ACP Journal Club, AFP Tips, JAMA Abstracts, Minerva-BMJ Medical Content Providers (5) CogniQ, MD Consult Mobile, MedScape Primary Care, Skyscape ARTBeat, Treatment Guidelines Journal Abstracting (8) Clinical Updates, JADE, Journal Watch, Journals to Go, MD Linx Network, MedJet, PeerView Institute, Primary Care Abstracts Other (3) EBM2GO, Epocrates Doc Alerts, Global Family Doctor Daily Alerts

Foraging Tool Evaluation Foraging tools independently evaluated by three investigators (SS, AS, DS) Used criteria developed from previously published relevance and validity criteria Reviewed any Internet or print material describing the method for identifying and including information in the foraging tool, and also evaluated examples of each tool

Relevance and Validity Criteria Relevance Criteria Are specific criteria designated for which types of research are reviewed (filter criteria)? Does the filter specifically distinguish between research that has patient- vs. disease-oriented outcomes? Is the scope of the sources limited to a specific specialty? Are specific recommendations made regarding how to apply the results of the current research into clinical context? Validity Criteria Was a validity assessment performed? Were Levels of Evidence (LOE) assigned to individual studies? Other Is a high-quality coordinated hunting tool available?*

Analysis Independently analyzed by a fourth investigator (MK) All disagreements on relevance and validity criteria resolved by consensus

Results 1/20 tools had a specific relevance filter (PeerView Institute) 1/20 tools had specific validity criteria (ACP Journal Club) 0/20 tools had both 0/20 tools distinguished between patient-oriented and disease-oriented evidence 0/20 tools assigned levels of evidence

Results 6/20 tools made specific recommendations about applying results in a clinical context 10/20 tools limited scope to specific specialties 7 covered primary care specialties 3 enabled selection of specialty 0/20 had a high-quality coordinated hunting tool

Conclusions Clinicians are increasingly dependent on clinical decision support tools and information awareness systems to keep up with clinical advances Not all these systems are created equally There are many shortcomings in currently available foraging tools

Discussion Ideally, these tools and systems will evolve in the direction of applying relevance and validity criteria to new medical information in order to increase their usefulness to clinicians in the real world. This process should be transparent, reproducible, and measurable.