1. Searching for Relevant Studies Interactive Case Study Quiz: C. Michael White, Pharm.D., FCP, FCCP Professor and Director University of Connecticut / Hartford Hospital Evidence-based Practice Center Author has no actual or potential conflicts of interest in relation to this activity

2. Interactive Case Quiz: Instructions Open this presentation as a slideshow. This will activate the hyperlinks. When you come to a decision slide, choose the red box corresponding to the correct choice. If you are correct you will be directed forward in the case. If you are incorrect, you will be directed back to the decision slide to choose again. – Click on forward hyperlinks (Red Boxes) to follow through case based on your responses – Click on home hyperlink (Blue House Icon) to go back to the last correct step in the series

3. Case Study Quiz: Searching the Literature You are interested in evaluating the benefits and harms associated with the use of thrombectomy devices in acute coronary syndromes. You need to determine what literature is available to conduct your review.

4. Participation Quiz: Question 1 You decide to conduct a systematic review (SR). What are two ways in which SRs differ from narrative reviews? They are the same thing Systematic reviews use comprehensive searches and explicit methods Systematic reviews always employ quantitative synthesis (meta-analysis

5. They are the Same This is incorrect, narrative reviews do not, as a rule, employ comprehensive literature searches or explicit methods for conducting the review [Click on Blue Box to go Back]

6. Comprehensive Search and Explicit Methods You are correct! Systematic reviews employ comprehensive searches and use explicit methods for conducting the review. [SELECT RED BOX]

7. Meta-Analysis This is incorrect. While some systematic reviews employ quantitative synthesis such as meta-analysis, others do not. [Click on Blue Box to go Back]

8. Participation Quiz: Question 2 So a comprehensive search is required. When conducting this search, what do you need to assure? You balanced precision and recall You have gotten every possible relevant citation regardless of the workload involved That you limit the search so that only relevant citations are found

9. Precision and Recall This is correct! Precision is the proportion of retrieved articles that are relevant while recall is the proportion of potentially relevant articles retrieved by the search. You want to employ methods to maximize precision and recall. Overly restrictive searches increase recall but compromise precision. Overly broad searches increase precision but compromise recall. [SELECT RED BOX]

10. You’ve Gotten it All This is incorrect. While it is laudable to want to acquire every possible relevant citation, this approach compromises precision and makes the SR unnecessarily onerous. You need to balance precision and recall. [Click on Blue Box to go Back]

11. You Only Have Relevant Citations in your Search This is incorrect. Every citation found is relevant but it is likely that many relevant citations were missed by this overly restrictive search strategy. You need to balance precision and recall. [Click on Blue Box to go Back]

12. Participation Quiz: Question 3 Given the need to balance precision and recall. What should you do before initiating a search? Enter in the search as “thrombectomy AND acute coronary syndromes” Enter the search as “thrombectomy OR acute coronary syndromes” Understand the topic, devise an analytic framework, pose clearly defined key questions, and understand the scope of the review before devising your strategy

13. Thrombectomy AND ACS This is incorrect. Searching based only on readily known disease and treatment information would be problematic. You need to understand the topic area before starting your search. [Click on Blue Box to go Back]

14. Thrombectomy OR ACS This is incorrect. This would be a voluminous search for citations for thrombectomy anywhere in the body in addition to any citation published on acute coronary syndromes. [Click on Blue Box to go Back]

15. Due Diligence This is correct. Understanding the topic and devising an analytic framework drive clearly defined key questions. Combining this with a understanding the scope of the project allows an optimal balance between precision and recall when sculpting the search. [SELECT RED BOX]

16. The Power of Due Diligence Your due diligence pays off, you now realize that your initial search “thrombectomy AND acute coronary syndrome” would have the compromised recall. ((Thrombectomy OR MerciClot.mp OR “other devices names”) AND (acute coronary syndromes OR myocardial infarction OR angina, unstable)) would assure better recall without appreciably impacting precision.

17. Participation Quiz: Question 4 You try to decide whether to do one search or two searches (one for benefits and another for harms). Which of the following would cause you to perform two searches? Performing two searches is redundant and should be avoided You decide a priori to only allow RCT data when evaluating benefits but allowing RCTs and observational trials for harms If the available literature base on the topic is extensive

18. Never Do Multiple Searches This is incorrect. If the size of the available literature is small, it may be appropriate to conduct one single broad search. However, if the literature is extensive and observational trials are allowed in the harms but not the benefits evaluations, a single broad search would reduce precision for studies reviewing efficacy endpoints without appreciably improving the recall. [Click on Blue Box to go Back]

19. Separate Benefits and Harms Search This is correct! Benefits searches is usually limited to randomized controlled trials since they have greater internal validity. Searches usually include the disease in question, the intervention, and a hedge for controlled trials. Harms searches are usually broader and include observational trials. Searches usually include the harm (or subheadings for general harms) and the intervention. When the available literature is extensive, two searches improves precision without appreciably impacting recall. [SELECT RED BOX]

20. Participation Quiz: Question 5 You need to decide which databases to use. The minimum requirement for a thorough search of the literature would include which two databases? MEDLINE + TOXNET MEDLINE + CINAHL MEDLINE + Cochrane CENTRAL

21. MEDLINE + TOXNET This is incorrect. Searching MEDLINE and the Cochrane CENTRAL databases are needed to fulfill the minimum broad search requirements. TOXNET is a database for adverse events or toxicology and can be used in addition to these others databases. [Click on Blue Box to go Back]

22. MEDLINE + CINAHL This is incorrect. Searching MEDLINE and the Cochrane CENTRAL databases are needed to fulfill the minimum broad search requirements. CINAHL is a database that focuses on nursing literature and can be added to these other databases. [Click on Blue Box to go Back]

23. MEDLINE + Cochrane CENTRAL This is correct. Combining MEDLINE with Cochrane CENTRAL would be a minimum requirement for a thorough search of the literature. Using MEDLINE alone may miss between 12% and 25% of relevant citations found in other databases and may produce a North American bias. [SELECT RED BOX]

24. Participation Quiz: Question 6 You decide to use hand searching of references from identified systematic reviews, studies, and abstract booklets from prominent meetings within the field of interest. Why would you do this? Hand searching can capture citations that are not indexed or are improperly indexed. Limiting hand searching to these sources can target this time intensive activity to areas where the yield will be the greatest.

25. Both are Correct You are correct! Hand searching can yield up to 13-25% of citations that do not come up in database searches. Since hand searching is the most time intensive search strategy, it should be targeted to maximize the yield. [SELECT RED BOX]

26. Participation Quiz: Question 7 You decide to search the FDA website and ClinicalTrials.gov to identify studies that might be appropriate for your systematic review. These website are an example of: Black literature Grey literature SIPs

27. Black Literature This is incorrect. You chose the wrong color. [Click on Blue Box to go Back]

28. Grey Literature This is correct! Grey literature can be important for identifying trials that were conducted but are, as of now, unpublished or incompletely published. Searching for grey literature is a useful way to minimize publication bias. [SELECT RED BOX]

29. SIPs This is incorrect. SIPs are Scientific Information Packets provided by the manufacturer of a drug or device. [Click on Blue Box to go Back]

30. Participation Quiz: Question 8 You conduct a thorough search of MEDLINE and Cochrane CENTRAL, use two searches (one for benefits and another for harms, and employ targeted hand searching and grey literature evaluation. What is the last thing you need to remember to do germane to searching. Only choose citations that agree with your preconceived notions Selectively report aspects of your search strategy Provide transparent reporting of search strategies and the citations you identified

31. Preconceived Notions This is incorrect. This would be akin to a narrative review and should not be used in a systematic review. [Click on Blue Box to go Back]

32. Selective Reporting This is incorrect. In order to build confidence in the systematic review results, transparent and defensible methods need to be employed. You need to report enough information about your search strategy so that a person, skilled in the art, could reproduce it. [Click on Blue Box to go Back]

33. Transparent Reporting This is correct! Transparent reporting is important in supporting the credibility of systematic review results. Reporting of search strategies and disposition of the citations should allow a person, skilled in the art, to reproduce the search you conducted. [SELECT RED BOX]

34. Congratulations! You have successfully maneuvered through the pitfalls of literature searching. A comprehensive understanding of the topic, the use of multiple searches, multiple databases, use of hedges, use of hand searching and grey literature, and development of a transparent reporting structure will give readers confidence in your systematic review.