Criteria For Critiquing A Research Report Jamie Jenkins, Ph.D. Biostatistician Department of Research and Education 50 Old Village Road, Suite 208 Columbus, Ohio 43228 Phone: 566-8027 jjenkin3@ohiohealth.com

Is the title of the article The Research Topic Is the title of the article appropriate and clear?

“Efficacy of Duct Tape vs “Efficacy of Duct Tape vs. Placebo in the Treatment of Verruca Vulgaris (Warts) in Primary School Children”

Is the abstract specific and representative of the article? The Research Topic Is the abstract specific and representative of the article?

Objective of study Design and Setting Participants Outcome measurement Results of study Author’s Conclusions

Is the purpose of the article made clear in the introduction? The Research Topic Is the purpose of the article made clear in the introduction?

Determine the efficacy of a type of treatment compared with placebo

Is the objective something that is researchable? The Research Topic Is the objective something that is researchable?

Duct tape applied to wart Treatment group: Duct tape applied to wart Control group: Corn pad application

Is the problem significant enough to warrant a formal research effort? The Research Topic Is the problem significant enough to warrant a formal research effort?

No quick, effective, inexpensive treatment Most existing therapies have associated adverse events

Does the review provide a critique of the relevant studies? Literature Review Does the review provide a critique of the relevant studies?

Cochrane review (2002): only salicylic acid efficacious Focht et al study (2002): duct tape therapy vs. cryotherapy

Has the author cited only the pertinent literature ? Literature Review Has the author cited only the pertinent literature ?

Only 5 references!! Prevalence of warts Self-consciousness 2 previous studies Editorial review

Are gaps in knowledge about the research question identified? Literature Review Are gaps in knowledge about the research question identified?

Current study proposal came from “idea box” Only 1 other study published

Are the independent and dependent variables clearly defined? Research Variables Are the independent and dependent variables clearly defined?

Which group (treatment or control) Dependent: Measurement of wart size Independent: Which group (treatment or control) Dependent: Measurement of wart size

If any confounding variables are present, are they identified? Research Variables If any confounding variables are present, are they identified?

Gender (randomization) Stickiness of tape Resistant warts Use of pumice stone

Are the hypotheses clear, specific, and testable?

H0: There is no clinical difference between treatment and control HA: Duct tape therapy is better than placebo (one-sided)

Hypotheses Does the alternative hypothesis describe a predicted relationship between the variables?

If duct tape therapy significantly reduces wart size (statistically and clinically), then the prediction was correct

Is the sample size adequate? Sampling Is the sample size adequate?

Significance level: α=0.05 Statistical power: 80% Effect size: differ by 30% Power calculation: at least 39/group; 100 optimal

Is the method of sample selection appropriate? Sampling Is the method of sample selection appropriate?

Recruitment from physician’s office or clinic not ideal Recruitment from schools more representative of community

Was the sample representative of the population? Sampling Was the sample representative of the population?

Long resolution period Resistant to treatment Selection bias

Is the research design appropriate for the objective?

Randomization Double-blinded study Complete follow-up --------------------------------- Only 6 week follow-up Why is 30% meaningful?

Does the study design address issues of validity ? Research Design Does the study design address issues of validity ?

Follow-up in person 100% follow-up Intention-to-treat (not per protocol)

Are the data collection methods appropriate for the study? Research Design Are the data collection methods appropriate for the study?

Measurements made at 4 distinct intervals Tape removed before assessment (blinding) Sliding caliper: tenths of a millimeters

Data Analysis Are the statistical tests the best ones for answering the research question(s)?

Χ2 for differences between the 2 groups in wart disappearance Linear regression for wart diameter reduction

Is the discussion of results clearly and logically organized? Data Analysis Is the discussion of results clearly and logically organized?

HA: Complete resolution (fail to reject) HA: Diameter reduction (reject null) HA: Surrounding warts

Are the tables and figures clear and understandable? Data Analysis Are the tables and figures clear and understandable?

Recruitment flowchart Figure 1: Recruitment flowchart Table 1: Typical demographics Tables 2-4: Outcome results

Interpretation of Results Are the interpretations based on the data obtained?

Table 2 is most important H1: p = 0.12 H2: p = 0.02 H3: p = 0.79, p = 0.22

Interpretation of Results Are limitations of the study addressed?

In future studies: Use stickier duct tape Longer follow-up period Don’t use pumice stone

Interpretation of Results Were all clinically important outcomes considered?

significant in only 6 wks. Important finding: Δ in wart diameter significant in only 6 wks. They don’t feel this is of clinical importance (what??!!)

If resolution of 30%, effect size of 15%: n=350 children! Wrapping Things Up If resolution of 30%, effect size of 15%: n=350 children! Future studies needed to address the flaws of both studies