1. Meta-analysis of split-mouth studies
Helen Worthington & Tanya Walsh

2. Cochrane minimum standards for clustered RCTs
Rationale
Cluster-randomised trials, cross-over trials, studies involving measurements on multiple body parts, and other designs need to be addressed specifically, since a naive analysis might underestimate or overestimate the precision of the study.

3. Cochrane minimum standards for clustered RCTs
Rationale
Failure to account for clustering is likely to overestimate the precision of the study - i.e. to give it confidence intervals that are too narrow and a weight that is too large.
Failure to account for correlation is likely to underestimate the precision of the study - i.e. to give it confidence intervals that are too wide and a weight that is too small.

4. Inclusion/exclusion criteria
Example of Randomised Controlled Trial
Population
Inclusion/exclusion criteria
Randomisation
Treatment A
Treatment B
Outcome
Outcome

5. Crossover RCT Crossover RCT
Compares two or more interventions in which the participants, upon completion of the course of one treatment, are switched to another. The order of treatment is assigned randomly.
Gp A
Gp B
Treatment
Control
Control
Treatment
Wash-out period

6. Suitability of cross-over designs
Appropriate for:
Chronic, stable conditions
Short term outcomes
Sufficient wash-out period
Possible problems affecting the second intervention period:
Irreversible outcomes (pregnancy)
Period effects (degenerative conditions)
Carryover effects (long acting intervention)
Consider using first period only as a parallel group trial
This information is rarely and selectively reported.
Material provided by Cochrane training unit

7. Suitability of split-mouth studies
Appropriate for:
Short and long-term outcomes;
No carry-over effect for intervention or outcome

8. Split-mouth studies
Common within dentistry – “patient acts as their own control”
Different designs:
Jaws or quadrants randomized and teeth/surfaces within jaw/quadrant or contra- lateral receive same intervention
Two teeth/surfaces per person randomized to different groups
The use of a split-mouth study design is common within dentistry.  For example fissure sealants are applied to the occlusal surface of all four molar teeth to prevent decay.  Randomised controlled trials with a split-mouth design may allocate the two pairs of surfaces to different sealant materials, or sealants versus no sealants.   The trials would then examine whether each surface became decayed or not over a 2/3 year period.  In systematic reviews the meta-analysis of trials such as these are frequently combining both parallel group studies (patient the unit of randomisation), and split-mouth studies.
If split-mouth studies are correctly reported then the meta-analysis is relatively straightforward using generic inverse variance (GIV) with effect estimates and standard errors.  However these trials are frequently not well analyzed or reported.  This talk looks at methods to include split-mouth studies in meta-analyses when the data are not fully available.

9. RCTs in Dentistry Frequently encounter the following problems:
Inappropriate design;
Incorrect analysis;
Poor reporting.
These problems are more common in split- mouth studies.
Recent publication of Pandis should help with reporting and perhaps encourage people to think more carefully about the design and analysis……….
CONSORT 2010 statement: extension checklist for reporting within person randomised trials
BMJ 2017; 357 doi: (Published 30 June 2017)
Cite this as: BMJ 2017;357:j2835

10. This talk is not about the appropriateness of split-mouth designs, although this is often an issue with this study design;
There is frequently a carry-over effect from one part of the mouth to the other;
Today we are assuming that a split-mouth study design is an appropriate design for the research question.

11. Meta-analysis Pooling split-mouth studies;
analysed and reported correctly;
analysed and reported incorrectly;
Pooling split-mouth and parallel group studies within the same meta-analysis.

12. Basic split-mouth design
Single pair of sites/teeth within an individual
Maybe add somewhere that a common analysis for correctly analysing paired data is the McNemar test. But this doesn’t give an effect estimate that can be used in a m-a
Similarly we don’t tend to use risk differences

13. Reporting of outcome
Placebo treatment
Total
Success
Failure
Active treatment s t a u v c b d n
A common analysis for correctly analysing paired data is the McNemar test
But this doesn’t give an effect estimate that can be used in a meta-analysis.
Similarly we don’t tend to use risk differences

14. Odds Ratio for paired outcome data
Conditional approach such as Mantel-Haenszel OR is the usual effect estimate for paired dichotomous outcomes;
Becker-Balagtas (BB) marginal method relies on marginal probabilities;
BB approach is preferred when summarising results across study designs in a meta-analysis;
Elbourne et al ’Meta-analyses involving cross-over trials: methodological issues’ (Beware of error in appendix!)
BB for 2 treatment 2 period cross-over trials

15. Odds Ratio for paired outcome data
Placebo treatment
Total
Success
Failure
Active treatment s t a u v c b d n
ORMH Conditional = t / u
ORMarginal = ad / bc
Var (lnORMarginal) = 1/a + 1/b + 1 c + 1/d /2n

16. Odds Ratio for paired outcome data
Placebo treatment
Total
Success
Failure
Active treatment s t a u v c b d n
= n2(ns-ab)/abcd n = s + t + u + v
Binary correlation ρ = ns- ab
coefficient √(abcd)

17. Pooling data
Once we have the lnOR and its SE for each study then we can enter this information into meta-analysis software (Revman) using Generic Inverse Variance (GIV) method

18. Incorrect analysis / reporting
Outcome
Active treatment
Placebo treatment
Success a b
Failure c d
Total n
Note: the n value will be the same for both groups
Frequently, data from split-mouth studies are incorrectly
reported in the form of parallel group studies.

19. Use of BB for split-mouth studies
For correctly reported studies;
For studies incorrectly reporting data;
Need to provide the correlation coefficient;
Sometimes can be estimated from other correctly analysed / reported studies;
Can impute a series of correlation coefficients and undertake a sensitivity analysis to consider the effects of different values.

20. Resin-based sealants versus no sealant, caries at 24 months

21. Resin-based sealants versus no sealant, caries at 36 months

22. Incorrect analysis- 36 months

23. Comparison between two analyses
Correct analysis using BB methods
OR 0.17 (95%CI 0.11, 0.27)
Incorrect analysis ignoring pairing
OR 0.17 (95%CI 0.10, 0.28)

24. More complex designs
Within a mouth, multiple sites are allocated to treatment x and multiple sites are allocated to treatment y;
These designs are rarely analyzed or reported correctly;
When reported as a parallel group we use a pragmatic approach
If the number of sites /intervention is <= 2 we use the BB method and ignore the clustering
We very rarely have access to IPD so for meta-analysis must make the best of what is reported;
This sometimes limits what we are able to do…..

25. Study designs in meta-analysis: different scenarios - 1
Design - Parallel group studies where the unit of randomisation is the individual and outcome measured on only one tooth/site
M-A - Standard approach where the unit of assessment is the individual;
M-A - Enter summary statistics or effect estimate and s.e.

26. Study designs in meta-analysis: different scenarios - 2
Design - Parallel group studies where the unit of randomisation is the individual and outcome is measured on more than one tooth/site;
M-A - Unit of analysis is the tooth/site. Standard errors of the effect estimates should be adjusted to take into account the multiplicity of recorded outcomes;
M-A - Unit of analysis may be the individual and some individual level outcome used on the analysis e.g. gingival bleeding scores.

27. Study designs in meta-analysis: different scenarios - 3
Design - Split-mouth studies studies where the individual can be considered as the cluster, and the unit of randomisation is the tooth / surface, and where the outcome is measured on only one tooth / surface per intervention;
M-A - Measurements will be correlated within an individual. BB method takes this into account;
M-A - Enter effect estimate and s.e. for paired data.

28. Study designs in meta-analysis: different scenarios - 4
Split-mouth studies studies where the individual can be considered as the cluster, and the unit of randomisation is the tooth / surface, and where the outcome is measured on more than one tooth / surface per intervention;
M-A - enter effect estimate and s.e. if correctly analysed and reported;
M-A - may be limited in carrying out M-A for studies incorrectly analysed / reported.
M-A - enter effect estimate and s.e. if correctly analysed and reported;
M-A - may be limited in M-A for studies incorrectly analysed.

29. Cochrane minimum standards for clustered RCTs
Rationale
Cluster-randomised trials, cross-over trials, studies involving measurements on multiple body parts, and other designs need to be addressed specifically, since a naive analysis might underestimate or overestimate the precision of the study.

30. Thank you