1. Interpreting the Medical Literature: A real example
Sandra E Moore, MD MSc FAAP
Assistant Professor of Clinical Pediatrics
Morehouse School of Medicine

2. Objectives To determine how to critically evaluate the literature
To determine how to choose an appropriate questions to study

3. Where do you Start
Start with the patient -- a clinical problem or question arises out of the care of the patient
The question - Construct a well built clinical question derived from the case 
The resource - Select the appropriate resource(s) and conduct a search
The evaluation - Appraise that evidence for its validity (closeness to the truth) and applicability (usefulness in clinical practice)
The patient - Return to the patient and integrate that evidence with clinical expertise, patient preferences and apply it to practice
Self-evaluation - Evaluate your performance with this patient

4. Background questions
Information about a disease or process that you should already know or can readily access
Who
What
When
Why
Where and
How

5. Foreground questions compares two or more things/groups Drugs
Treatment modalities
Groups based on exposures
Diagnostic tests or
The harms or benefits of two approaches

6. PICO format P - Patient / Problem I - Intervention (or exposure)
What are the characteristics of the patient or population?
What is the condition or disease you are interested in?
I - Intervention (or exposure)
What do you want to do with this patient (e.g. treat, diagnose, observe)?
C – Comparison (or Control), if any
What is the alternative to the intervention (e.g. placebo, different drug, surgery)?
O – Outcome
What are the relevant outcomes (e.g. morbidity, death, complications)?

7. PP -ICONS P – Problem P - Patient / Problem
I - Intervention (or exposure)
C - Comparison (or Control), if any
O – Outcome (Disease - orientated Outcomes; DOEs or Patient Orientated Evidence that Matters; POEMs)
N - Number of Subjects
S - Statistics

8. Intervention Studies Is the study valid? What are the results? and
Is it applicable to the patient?

9. Clinical Scenario You are in Continuity Clinic
Mother brings a 9 year old daughter in because of warts on hand. Mother read about the efficacy of duct tape in treating warts and wanted to know your opinion

10. Background Questions about Warts
Who?
What?
When?
Why?
Where? and
How?

11. What is our Question? P (problem/patient) Intervention
Control or comparisons
Outcomes
children and warts
duct tape
nothing or other therapies
resolution of warts
Compared to standard treatment, is duct tape
effective for the treatment (resolution) of warts in
children?

12. PP –ICONS for Study P – Problem P - Patient / Problem
I - Intervention (or exposure)
C - Comparison (or Control), if any
O – Outcome (Disease orientated Outcomes; DOEs or Patient Orientated Evidence that Matters;POEMs)
N - Number of Subjects
S - Statistics

13. Search in Appropriate Database
Medline
Pubmed
Cochrane
InfoPOEMS (Patient-Oriented Evidence that Matters)
UpToDate
Md Consult
MSM Library
Focht DR III, et al. The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the common wart). Arch Pediatr Adolesc Med October 2002;156:971-4.

14. Is duct tape effective for the treatment (resolution) of warts in children? Focht DR et.al
Objective  To determine if application of duct tape is as effective as cryotherapy in the treatment of common warts.
Design  A prospective, randomized controlled trial with 2 treatment arms for warts in children.
Patients  A total of 61 patients (age range, 3-22 years) were enrolled in the study from October 31, 2000, to July 25, 2001; 51 patients completed the study and were available for analysis.
Intervention  Patients were randomized using computer-generated codes to receive either cryotherapy (liquid nitrogen applied to each wart for 10 seconds every 2-3 weeks) for a maximum of 6 treatments or duct tape occlusion (applied directly to the wart) for a maximum of 2 months. Patients had their warts measured at baseline and with return visits.
Main Outcome Measure  Complete resolution of the wart being studied.
Results  Of the 51 patients completing the study, 26 (51%) were treated with duct tape, and 25 (49%) were treated with cryotherapy. Twenty-two patients (85%) in the duct tape arm vs 15 patients (60%) enrolled in the cryotherapy arm had complete resolution of their warts (P = .05 by c2 analysis). The majority of warts that responded to either therapy did so within the first month of treatment.
Conclusion  Duct tape occlusion therapy was significantly more effective than cryotherapy for treatment of the common wart.

15. Is this a clinical relevant question?
Warts are common in children
Quick, effective, and inexpensive treatment is not available ( maybe salicylic acid)
Although warts are medically benign, they are unsightly and may cause a child to feel self-conscious
Parents/children often seek medical therapy
Up to 30% resolve in 10 weeks without therapy

16. Are the results of this therapy (intervention) study valid?
Was the assignment of patients to treatment randomized?
Were all the patients who entered the trial properly accounted for at its conclusion?
Were patients analyzed in the groups to which they were (originally) randomized?
Where there enough patients (N)?
Where the statistical test appropriate?
Were the results statistically significance?

17. Was the assignment of patients to treatment randomized?
Random allocation comes closest to insuring the creation of groups of patients who will be similar in their risk of the events you hope to prevent.
Randomization balances the groups for prognostic factors (ie. disease severity) which eliminates over-representation of any one characteristic within the study groups.
Randomization should also be concealed from the clinicians and researchers of the study to help eliminate conscious or unconscious bias.
Were the patient randomized in the duct tape study?

18. Number of Subjects (N)
The number of subjects is crucial to whether accurate statistics can be generated from the data.
Too few patients in a research study may not be enough to show that a difference actually exists between the intervention and comparison groups (known as the "power" of a study).
Many studies are published with less than 100 subjects, which is usually inadequate to provide reliable statistics.
A good rule of thumb is 400 subjects

19. Did the study have sufficient power?
The power of a test refers to its ability to detect what it is looking for. The probability of correctly rejecting the null hypothesis (i.e. the probably of finding a difference if it truly exist)
Alpha (a): Usually set to be 0.05, although this is somewhat arbitrary. This is the probability of a type I error, that is the probability of rejecting the null hypothesis given that that the null hypothesis is true.  In other words it is the probability of thinking we have found something when it is not really there.
Power = 1 – Beta (b), where beta is the probability of a type II error (acceptance of a false null hypothesis). Typically 80% (0.8) is considered adequate power.
Typically increase power by increasing sample size
No mention of power, a or b in duct tape study

20. Were all the patients who entered the trial properly accounted for at its conclusion?
All patients who started the trial should be accounted for at the end of the trial. If patients are not accounted for, the validity of the study may be jeopardized.
A good study will have better than 80% follow-up for their patients.
Patients may drop out of a study for various reasons. If these patients are not included in the results, they can make the treatment look better than it really is (and vice versa).
To be sure of a study's conclusions, lost patients should be assigned to the "worst-case" outcomes and the results recalculated. The results are still valid if the recalculations do not change the end results. This is referred to as “Intention to Treat”

21. Are The Statistics Appropriate?

22. Statistical Test in our study
Relative risk reduction (RRR): the percent reduction in events in the treated group compared to the control group event rate.
Not a good way to compare outcomes
Amplifies small differences and makes insignificant findings appear significant
Doesn’t reflect the baseline risk of the outcome event
Can make weak results look good
Popular and will be reported in almost every journal article
Can mislead you
RRR would be (85 percent – 60 percent) / (60 percent) x 100 = 42%
i.e. duct tapes appears to be 42% more effective than cryotherapy in treating warts
Absolute risk reduction (ARR): the difference in the outcome event rate between the control group and the experimental group.
A better statistic to evaluate outcome, as it does not amplify small differences, but shows the true difference between the experimental and control interventions
ARR for the wart study is the outcome event rate (complete resolution of warts) for duct tape (85 percent) minus the outcome event rate for cryotherapy (60 percent) = 25 percent

23. Statistical Significance
Are the result statistically significant?
Statistical test p < 0.05
Confidence interval (CI) should not include “0”
Statistical significance DOES NOT mean clinical significance
Results of Duct tape study had a p value < .05
CI not listed
The calculated 95% CI for this study's reported treatment effect is 1.1 to 48.1, and we can state with 95% confidence that the true treatment effect is somewhere between these 2 values.
Maximally conservative estimate, including patients lost to follow-up, CI contains 0, and therefore difference not statistically significant

24. Key terminology for estimating the size of the treatment effect
Outcome 
Risk of outcome + -
Treated (Y) a b
Y = a/(a + b)
Control (X) c d
X = c/(c + d)
Relative Risk (RR) is the risk of the outcome in the treated group (Y) compared to the risk in the control group. = Y / X
Relative Risk Reduction (RRR) is the percent reduction in risk in the treated group (Y) compared to the control group (X) = 1 - Y / X x 100%
Absolute Risk Reduction (ARR) is the difference in risk between the control group (X) and the treatment group (Y) = X – Y
Number Needed to Treat (NNT) is the number of patients that must be treated over a given period of time to prevent one adverse outcome = 1 / (X - Y)

25. Intention to Treat (ITT)
Intention to treat: subjects are analyzed according to the categories into which they were originally randomized.
Assumes worse case scenario
Benefits of a treatment are more difficult to demonstrate with intention-to-treat analysis
Helps to mitigate differences by including subjects who are unlikely to have experienced benefit from the intervention
Six patients from cryotherapy group and 4 patients from the duct tape group were lost to follow-up (16% of patients).
Worst case scenario: 6 cryotherapy patients had wart resolution and the 4 duct tape patients had residual wart.
Wart resolution would then be: duct tape 78% and cryotherapy 68% (95% CI, -17 to 28) – therefore not a statistically significant difference between the two treatments.

26. Number needed to treat (NNT)
Number needed to treat (NNT): number of patients who must be treated to prevent one adverse outcome OR the number of patients who must be treated for one patient to benefit
= 1/ARR. In the case of the duct tape study 1/.25 = 4
The lower the NNT the better (intervention studies 10 is good, 5 is excellent; for preventive studies 20 is good)

27. Interpreting our Wart Study
Is this an important clinical question?
Are the patients studied similar to our patient?
Was the intervention acceptable?
Were the outcomes clinically relevant?
Was the assignment of patients to treatment randomized?
Were all the patients who entered the trial properly accounted for at its conclusion?
Were patients analyzed in the groups to which they were (originally) randomized?
Where there enough patients (N)?
Are the results statistically significance?
Are the results clinically significant?
Is the NNT appropriate?

28. Back to the patient
Would you recommend duct tape to your patient?

29. No Yes COMMENTS Introduction, Background and Methodology
Stated the study objective in his/her own words
Described the significance of the problem addressed by the study
Described how the study objectives and /or hypotheses are relevant to community and/or patient care
Described the study design (methodology)
Described the population studied
Described the setting in which the study occurred
Described the statistical analysis plan
RESULTS SECTION
Able to state major outcome (s) that are reported
Able to summarize demographic of population studied
Explained how the major study endpoints are relevant to the population studied
Presented the main results of the study with comparison to the original hypotheses.
CONCLUSION SECTION
Described the study conclusions as presented by the authors.
Stated their evaluation and clinical significance of study results
Identified limitation of the study
Identified how the results could be applied to his/her patients
ADDITIONAL COMMENTS

30. REFERENCES
The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the common wart). Focht DR et al Arch Pediatr Adolesc Med. 2002;156:
Krejcie RV, Morgan DW. Determining sample size for research activities. Educational and Psychological Measurement. 1970;30:
Is Duct Tape Occlusion Therapy as Effective as Cryotherapy for the Treatment of the Common Wart? Ringold et al Arch Pediatr Adolesc Med. 2002;156(10):
Efficacy of Duct Tape vs Placebo in the Treatment of Verruca Vulgaris (Warts) in Primary School Children. de Haen et al. Arch Pediatr Adolesc Med 2006;160:
Interpreting negative results from an underpowered clinical trial: warts and all. Van Cleave et al. Arch Pediatr Adolesc Med 2006;160:
Studies Should Report Estimates of Treatment Effects With Confidence Intervals. Cummings Arch Pediatr Adolesc Med 2007;161:

31. Online References/Tools