Critical Appraisal of a RCT Dr Donncha O’Gradaigh, EBP Group South East

appraise ask assess acquire apply

Fair tests Testing Treatments link

Clinical scenario… After a sports event, two of your team-mates are debating whether there is any benefit from anti-inflammatory gel for an ankle sprain

Validity reliability significance

PICO - matching Patient / Population Intervention Control / comparison Will the results be relevant to your patient? Is your patient in a sub-group? Intervention Control / comparison Is the comparison relevant and fair (placebo versus active comparators) Outcome Clinically relevant? Patient preference?

Some definitions www.students4bestevidence.net offer glossary of terms

The METHODS section

Systematic approaches to appraisal of “reliability” in study design RAMMbo and GATES

RAMMbo Recruitment representative sample – age, symptoms… of the population? of the way you will meet the next one (random, consecutive, exclusions)?

RAMMbo Allocation Equal chance of either intervention or control Stratification in randomisation Computer versus envelopes “Blind” generally refers only to allocation – no use if subsequent process reveals allocation “Double dummy”, marks, adverse effects; acupuncture studies

RAMMbo Maintenance in the study Protocol violations versus allowed Consider all the possible differences Maintain blinding Record drop-outs 80:20 guide

RAMMbo Measurement (of outcome) Is the measurement appropriate for the treatment Is the measurement appropriate for your patient’s clinical query carried out without bias Blinded (b) assessor or objective (o) tool (both?)

R Mbo M A P I/C O

the Results section- or “is this any use?”

“I need a p”

Result = truth + bias + random error DESIGN Result = truth + bias + random error MEASURE

Understanding bias and error

the result is an ESTIMATE from a SAMPLE

how do 95% CI and p relate? 95% CI measured difference between the two arms If there was no difference between treatment arms 95% CI The range of values the difference might have

Spot the [deliberate] error “Our results show a highly significant (p<0.001) effect of metwostatin on LDL cholesterol” “our study of 53 individual patients shows that there is no benefit from this intervention (p=0.051)” “While the effect of x-mab on pain did not reach significance (p>0.05), the significant reduction in radiographic progression (p=0.029) is an important benefit of treatment”

In the workshop A medication RCT Methodolgy, RAMMbo Results – two different types Your own Clinical Query / group project A two step intervention More complex, maybe more real-life More technical aspects of trial design