EBM --- Journal Reading Presenter ：呂宥達 Date ： 2005/10/27

Users’ Guides to the Medical Literature Ⅵ. How to Use an Overview Andrew D. Oxman, MD. MSc; Deborah J. Cook, MD. MSc; Gordon H. Guyatt, MD. MSc; for the Evidence- Based Medicine Working Group JAMA; Nov 2, 1994; 272, 17

 Systematic overviews of the medical literature that summarize scientific evidence are prevalent.  These overviews address questions of treatment,causation,diagnosis or prognosis. SO: how to distinguish a good overview from the bad one and for using the result?

Key questions: 1. Are the results of the overview valid? 2.What are the result? 3.Will they be helpful in my patient care

 I. Are the results of the study valid?  Primary guides: –Did the overview address a focused clinical question? Did the overview address a focused clinical question?Did the overview address a focused clinical question? –Were the criteria used to select articles for inclusion appropriate? Were the criteria used to select articles for inclusion appropriate?Were the criteria used to select articles for inclusion appropriate?  B. Secondary guides: –Is it unlikely that important, relevant studies were missed? Is it unlikely that important, relevant studies were missed?Is it unlikely that important, relevant studies were missed? –Was the validity of the included studies appraised? Was the validity of the included studies appraised?Was the validity of the included studies appraised? –Were assessments of studies reproducible? Were assessments of studies reproducible?Were assessments of studies reproducible? –Were the results similar from study to study? Were the results similar from study to study?Were the results similar from study to study?

Are the results of the overview valid?  Primary guides: 1.did the overview address a focused clinical question? Most questions are formulated in terms of a simple relationship between: Most questions are formulated in terms of a simple relationship between: Patients Exposure/intervention: e.g. Treatment, diagnosis test, potentially harmful agent. outcome *many overviews address a number of questions, so they provide a useful introduction to an area but offer limited support to their conclusion. Thus, you must then study the references in order to judge the validity of the authors’ conclusion

2. Were the criteria used to select articles for inclusion appropriate? to determine if the investigator reviewed the appropriate research, we have to know the criteria they use to select research. to determine if the investigator reviewed the appropriate research, we have to know the criteria they use to select research. This include: specify the patient, exposures outcomes of the interest, and methodological standard s used to select studies. This include: specify the patient, exposures outcomes of the interest, and methodological standard s used to select studies. Difference in the patients, exposures and outcomes can lead to different result among overviews that appear to address the same clinical question. Difference in the patients, exposures and outcomes can lead to different result among overviews that appear to address the same clinical question. the authors will cite studies the support their own prior conclusion. So, the bias in choosing articles to cite happened. the authors will cite studies the support their own prior conclusion. So, the bias in choosing articles to cite happened. Thus, the standards used to select studies should similar to the primary validity criteria we have described for original reports of research.(table below) Thus, the standards used to select studies should similar to the primary validity criteria we have described for original reports of research.(table below)

Guides for selecting articles that are most likely to provide valid results therapy  Was the assignment of patients to treatments randomized?  Were all of the patients who entered the trials properly accounted for and attributed at its conclusion. Diagnosis  Was there an independent, blind comparison with a reference standard?  Did the patient sample include an appropriate spectrum of the sort of patients to whom the diagnostic test will be applied in clinical practice? Harm  Were there clearly identified comparison groups that were similar with respect to important determinants of outcome, others than the one of interest?  Were outcome and exposures measures in the same way in the groups being compared? Prognosis  Was there a representative and well-defined samples of patients at a similar point in the course of disease?  Was follow-up sufficiently long and complete?

Secondary guides 1.Is it unlikely that important relevant studies were missed?  Authors must conduct a thorough search for studies that meet their inclusion criteria.  This should include bibliographic database,e.g.MEDLINE and EMBASE, checking the reference list of the articles they retrieve and personal contact with experts in the area

2. Was the validity of the included studies appraised?  Unfortunately, peer does not guarantee the validity of published research.  So, when considering whether to believe the results of an overview, we should check whether the authors examined criteria similar to those we have presented in deciding on the credibility of their primary studies.(table above)

3. Were assessments of the studies reproducible?  Authors of review articles must decide which studies to include,how valid they are, and which data to extract from them.  Each of these decisions requires judgment by the reviewers and each subject to both mistakes(random error) and bias(systematic errors)  Having more peoples participate in each decision guards against errors.

4.Were the results similar from study to study?  most systematic overviews document important differences in patients, exposures, outcome measures and research methods from study to study.  these factors are so different that it no longer makes sense to combine the study results.  Investigators can test the extent to which differences among the results of individual studies are greater than you would expect if all studies were measuring the same underlying effect and the differences observed were due only to chance.

 The statistical analyses that are used to do this are called "tests of homogeneity.’’  The more significant the test, the less likely it is that the observed differences in the size of the effect are due to chance alone, and the more likely that differences in patients, exposures, outcomes, or study design are responsible for the varying treatment effect.  Both the "average" effect and the confidence interval around the average effect need to be interpreted cautiously when there is significant heterogeneity (a low probability of the differences in results from study to study being due to chance alone, reflected in a low p-value).

 Unfortunately, a "nonsignificant" test does not necessarily rule out important differences between the results of different studies. So?  when there are large differences between the results of different studies, a summary measure from all of the best available studies may still provide a better estimate for clinical use than the results of any one study

II. What are the results? 1.What are the overall results of the overview?  In overviews, investigators collect data from individual studies. These data must also be summarized, and increasingly, investigators are using quantitative methods to do so.  Simply comparing the number of "positive" studies to the number of "negative" studies is not an adequate way to summarize the results. Why???

Because……. 1.large and small studies are given equal weights 2.One investigator may interpret a study as positive, while another investigator interprets the same study as negative. 3.In addition, there is a tendency to overlook small but clinically important effects if studies with statistically "nonsignificant" (but potentially clinically important) results are counted as “negative“.

 Typically, meta-analysts weight studies according to their size, with larger studies receiving more weight. Thus, the overall results represent a weighted average of the results of the individual studies.

3.Will the results help me in caring for my patients? 1.Can the results be applied to my patient care?  One of the advantages of an overview is that, since many studies are included, the results come from a very diverse range of patients.  If the results are consistent across studies, they apply to this wide variety of patients. Even so, the clinician may still be left with doubts about the applicability of the results

 These questions raise the issue of sub-group analysis.  One of the most important guides is that conclusions drawn on the basis of between- study comparisons (comparing patients in one study with patients in another) should be viewed sceptically.

 2. Were all clinically important outcomes considered? considered?  3. Are the benefits worth the harms and costs?