1. Critical Appraisal of the Biomedical Literature Welcome June 2012 Peggy Edwards, AMLS TTUHSC - Preston Smith Library Lubbock, Texas 79430

2. Objectives During this learning module, participants will: 1) know the definition and steps in critical appraisal 2) become familiar with the hierarchy of study design 3) identify the appropriate study design for therapy, diagnosis, etiology/harm, and prognosis 4) know how to categorize the levels of evidence 5) learn the definition of “N-of-1 Randomized Controlled Trials” 6) critically appraise an article using a JAMAevidence worksheet 7) search a practice topic in PubMed’s Clinical Queries During this learning module, participants will: 1) know the definition and steps in critical appraisal 2) become familiar with the hierarchy of study design 3) identify the appropriate study design for therapy, diagnosis, etiology/harm, and prognosis 4) know how to categorize the levels of evidence 5) learn the definition of “N-of-1 Randomized Controlled Trials” 6) critically appraise an article using a JAMAevidence worksheet 7) search a practice topic in PubMed’s Clinical Queries

3. Steps in the Evidence-Based Process are: (Citrome, & Ketter, 2009) This module discusses critical appraisal

4. What is Critical Appraisal? “The process of assessing and interpreting evidence by systematically considering its validity, results, and relevance.” The Cochrane Collaboration Glossaryhttp://www.cochrane.org/glossary/5#letterc

5. “Critical appraisal is the first step in transferring research knowledge into practice.” Das, K., Malick, S., & Khan, K. (2008). Tips for teaching evidence-based medicine in a clinical setting: lessons from adult learning theory. part one. Journal of the Royal Society of Medicine, 101(10), 493-500. doi: 10.1258/jrsm.2008.080712 web address: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2586873/?tool=pubmed

6. Purpose of Critical Appraisal “Randomized controlled trials and systematic reviews are the highest levels of evidence but they are not automatically of good quality and should always be appraised critically.” Das, K., Malick, S., & Khan, K. (2008). Tips for teaching evidence-based medicine in a clinical setting: lessons from adult learning theory. part one. Journal of the Royal Society of Medicine, 101(10), 493-500. doi: 10.1258/jrsm.2008.080712 web address: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2586873/?tool=pubmed A Consultation of Surgeons, old negative no. 62-116 [engraving]. [Bethesda, MD., United States of America]: Images from the History of Medicine, U.S. National Library of Medicine, National Institutes of Health; accessed June 12, 2012. The National Library of Medicine believes this item to be in the public domain. print: 11 x 16 cm.

7. Clinical Questions and Study Design “There are 5 fundamental types of clinical questions: 1) therapy 2) harm 3) differential diagnosis 4) diagnosis 5) prognosis” (Guyatt, 2008) “There are 5 fundamental types of clinical questions: 1) therapy 2) harm 3) differential diagnosis 4) diagnosis 5) prognosis” (Guyatt, 2008) It is important “to correctly identify the category of study, because, to answer your question, you must find an appropriately designed study.” (Guyatt, 2008)

8. Suggested Best Method of InvestigationStudy Category RCT>cohort>case control>case series Therapy prospective, blind comparison to a gold standard Diagnosis RCT>cohort>case control>case series Etiology/Harm cohort>case control>case series Prognosis RCT>cohort>case control>case series Prevention prospective, blind comparison to a gold standard Clinical Exam Economic Analysis Cost Medical Library Association. MLANET, Education, Web-based Learning. Hp. Nov, 2001. Web-based Courses: EBM and the Medical Librarian. Available: http://www.mlanet.org/education/web/web_courses.html 10 Apr. 2005.http://www.mlanet.org/education/web/web_courses.html

9. Hierarchy of Study Design (TTUHSC Preston Smith Library, 11/21/2008)

10. Evidence–Based Treatment “Clinicians should use the results of randomized controlled trials (RCTs) of groups of patients to guide their clinical practice. However, clinicians cannot always rely on the results of RCTs … To determine the best care for an individual patient, clinicians can conduct n–of–1 randomized controlled trials in individual patients.” (Guyatt, 2008)

11. Hierarchy of Strength of Evidence for Prevention & Treatment Decisions  N–of–1 randomized trial  Systematic reviews of randomized trials  Single randomized trial  Systematic review of observational studies addressing patient–important outcomes  Single observational study addressing patient–important outcomes  Physiologic studies (studies of blood pressure, cardiac output, exercise capacity, bone density, and so forth)  Unsystematic clinical observations (Guyatt, 2008)

12. N–of–1 Randomized Controlled Trials (Guyatt, 2008) Experiment designed to determine effect of an intervention/exposure on a single study participant In a one N–of–1 design the patient undergoes pairs of treatment periods 1 period involves the use of the experimental treatment 1 period involves the use of an alternate treatment/placebo if possible, patient and clinician are blinded outcomes are monitored Treatment periods are replicated until clinician and patient are convinced that treatments are definitely different or definitely not different

13. Assignment Go to JAMAevidence create a profile (there is no fee) Read Chapter 6 Therapy (Randomized Trials) Locate the Therapy Worksheet in JAMAevidence Evaluate the article using the worksheet

14. www.ttuhsc.edu/libraries Hover mouse on Databases Click Evidence Based Databases

15. Click JAMAevidence

16. Click My JAMAevidence

17. Create a Profile Account

18. Click Therapy

19. Read Chapter 6 Therapy (Randomized Trials)

20. Read this Article Sweeting, M. J., Thompson, S. G., Brown, L. C., Greenhalgh, R. M., & Powell, J. T. (2010). Use of angiotension converting enzyme inhibitors is associated with increased growth rate of abdominal aortic aneurysms. Journal of vascular surgery, 52(1), 1-4. doi: 10.1016/j.jvs.2010.02.264 PMID: 20494541 To retrieve this article, use the instructions on the following screens:

21. www.ttuhsc.edu/libraries Mouse over Databases Click PubMed

22. Enter the PMID: 20494541

23. Click the Elsevier icon

24. The full-text article will open.

25. After reading the article, critically appraise it using the therapy worksheet from JAMAevidence. Use the following instructions:

26. Click Critical Appraisal Worksheet: Therapy Click Critical Appraisal Worksheet: Therapy From the page at the end of Chapter 6:

27. From the Home Page: Click My JAMAevidence

28. Click Therapy Worksheet

29. Critical Appraisal Worksheet - Therapy

30. Additional Information http://ktclearinghouse.ca/cebm/practise/ca

31. Applying the Results “Each (patient management) decision involves a consideration of the relevant evidence and a weighing of the likely benefits and downsides in light of the patient’s values and preferences. When considering choices, clinicians may benefit from structured enumeration of the options and outcomes, systematic reviews of the evidence regarding the relationship between options and outcomes, and recommendations regarding the best choices.” (Guyatt, 2008)

32. * Medical evidence or recommendations can vary in quality * Sources of evidence range from: 1. small laboratory studies to 2. well-designed large clinical studies with minimized bias * Is a recommendation is strong or weak? * Can you be confident in the recommendation? *Grading by strength of recommendations is a systematic approach which can minimize bias and aid interpretation * Quality of evidence can be categorized as high, moderate, low, or very low (The GRADE Working Group, 2005) Quality and Levels of Evidence

33. A High B C D Moderate Low Very Low Further research is very unlikely to change our confidence in the estimate of effect. Several high–quality studies with consistent results In special cases: one large, high–quality multi– center trial Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. One high–quality study Several studies with some limitations Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. One or more studies with severe limitations Any estimate of effect is very uncertain. Expert opinion No direct research evidence One or more studies with severe limitations Code Quality of Evidence Definition (Essential Evidence Plus EBM Guidelines Editorial Team, 2010) G rading of R ecommendations, A ssessment, D evelopment, and E valuation – GRADE

34. Strength of Recommendation "Recommendations to administer, or not administer, an intervention, should be based on the tradeoffs between benefits on the one hand, and risks, burden and, potentially, costs on the other. If benefits outweigh risks and burden, experts will recommend that clinicians offer a treatment to typical patients. The uncertainty associated with the tradeoff between the benefits and risks and burdens will determine the strength of recommendation." (The GRADE Working Group, 2005) Strength Of Recommendation Taxonomy (SORT) Code Definition A B C Consistent, good–quality patient–oriented evidence * Inconsistent or limited–quality patient–oriented evidence * Consensus, disease–oriented evidence *: usual practice, expert opinion, or case series for studies of diagnosis, treatment, prevention, or screening *Patient–oriented evidence measures outcomes that matter to patients: morbidity, mortality, symptom improvement, cost reduction, and quality of life. Disease– oriented evidence measures: immediate, physiologic, or surrogate end points that may or may not reflect improvements in patient outcomes (e.g. blood pressure, blood chemistry, physiologic function, pathologic findings). (Essential Evidence Plus EBM Guidelines Editorial Team, 2010)

35. Summary Topics/exercises covered in this section included: 1) the definition and steps of critical appraisal 2) the hierarchy of study design 3) the appropriate study design for therapy, diagnosis, etiology/harm, and prognosis 4) the levels of evidence categories 5) the definition of “N-of-1 Randomized Controlled Trials” 6) critical appraisal of an article using a JAMAevidence worksheet Topics/exercises covered in this section included: 1) the definition and steps of critical appraisal 2) the hierarchy of study design 3) the appropriate study design for therapy, diagnosis, etiology/harm, and prognosis 4) the levels of evidence categories 5) the definition of “N-of-1 Randomized Controlled Trials” 6) critical appraisal of an article using a JAMAevidence worksheet

36. Assessment 1) What is the definition of critical appraisal? What is the purpose? 2) Name three high quality study designs. 3) What are the four categories of levels of evidence in the GRADE framework? 1) What is the definition of critical appraisal? What is the purpose? 2) Name three high quality study designs. 3) What are the four categories of levels of evidence in the GRADE framework?

37. EBP Literature from PubMed’s Clinical Queries EBP Literature from PubMed’s Clinical Queries

38. Biomedical journal literature PubMed Database Published by: The National Library of Medicine Published by: The National Library of Medicine International scope

39. Accessing PubMed Freely available over the Internet Also through library’s home page @ www.pubmed.gov

40. tool that searches for evidence-based literature PubMed’s Clinical Queries

41. Diagnosis Therapy Etiology (cause) Prognosis (prediction of probable outcome) Clinical Queries Searchable by:

42. uses pre-formulated search strategies with a keyword search PubMed’s Clinical Queries

43. www.ttuhsc.edu/libraries Mouse over Databases Click PubMed

44. TTUHSC Libraries - Proxy Server A Problems with eRaider call: 806-743-2870 or 806-743-2875 a When Searching from Off Campus: Enter the Eraider username and password assigned to you by Information Services.

45. Click Clinical Queries

46. Enter asthma Search Click

47. Results for Therapy and Broad Scope Results for Therapy and Broad Scope Change to Narrow Scope

48. Click See all (7542)

49. Click Limits

50. Click on the article title you wish to read

51. Click for Free Full Text TTUHSC ONLINE

52. Click

53. Full Text

54. Summary Topics/exercises covered in this section included: 1) using one of the study categories, therapy, diagnosis, etiology, or prognosis, to narrow search results in PubMed’s Clinical Queries 2) practicing a search topic in PubMed’s Clinical Queries 3) downloading a full-text article from PubMed’s Clinical Queries Topics/exercises covered in this section included: 1) using one of the study categories, therapy, diagnosis, etiology, or prognosis, to narrow search results in PubMed’s Clinical Queries 2) practicing a search topic in PubMed’s Clinical Queries 3) downloading a full-text article from PubMed’s Clinical Queries

55. Assessment 1) What is the purpose of PubMed’s Clinical Queries?

56. Conclusion In this module, we have considered and applied some basic skills in evaluating the validity of a clinical study. You are encouraged to continue to increase your skill level in critical appraisal. Two useful resources are: JAMAevidence - an online resource User’s Guides to the Medical Literature: a manual for evidence-based clinical practice by Gordon Guyatt, M.D., Drummond Rennie, M.D., et al. In this module, we have considered and applied some basic skills in evaluating the validity of a clinical study. You are encouraged to continue to increase your skill level in critical appraisal. Two useful resources are: JAMAevidence - an online resource User’s Guides to the Medical Literature: a manual for evidence-based clinical practice by Gordon Guyatt, M.D., Drummond Rennie, M.D., et al.

57. The End Jan 2012