1. Estimating Incremental Cost- Effectiveness Ratios from Cluster Randomized Intervention Trials M. Ashraf Chaudhary & M. Shoukri

2. CREATE Biostat Core Meeting, Cape Town Sep 28, 20052 Incremental Cost-Effectiveness Ratio Assuming numerator and denominator positive, R is the cost per additional outcome achieved by the treatment.

3. CREATE Biostat Core Meeting, Cape Town Sep 28, 20053 Statistical Properties Biased Consistent Positively skewed Limiting distribution is normal Very sensitive to changes in the denominator No exact method of estimating variance

4. CREATE Biostat Core Meeting, Cape Town Sep 28, 20054 Methods Parametric: 1) Taylor Series Expansion 2) Fieller's Method Non-Parametric Bootstrap: 3) Percentile Method 4) BCa

5. CREATE Biostat Core Meeting, Cape Town Sep 28, 20055 Cluster Randomized Trials For fixed cluster size, so that.

6. CREATE Biostat Core Meeting, Cape Town Sep 28, 20056 Coefficients of Variation

7. CREATE Biostat Core Meeting, Cape Town Sep 28, 20057 Taylor Series Method Large sample normal approximation yields, Inaccurate if is far from normal or the sample is not large enough. Affected by extreme values Interval is symmetrical even if is not.

8. CREATE Biostat Core Meeting, Cape Town Sep 28, 20058 Fieller's Method » Not symmetrical if is not » Assumes bivariate normality » Assumes unbiasedness of » Imaginary roots for certain samples » Volatility of making a negative quantity leading in correct intervals.

9. CREATE Biostat Core Meeting, Cape Town Sep 28, 20059 Non-Parametric Bootstrap 1.a- resample observations or b- clusters or c- two stage bootstrap? a – not appropriate as observations within cluster are correlated b – theoretically appropriate c – mathematically preferable but assumes no correlation within clusters 2.We use approach b and resample clusters retaining all observations in resampled clusters 3.A cluster level summary data is prepared with mean cost and mean effect in each cluster 4.boot package in R was used to implement bootstrap stratified by study arm and estimate intervals 5. bootstrap replications =1999

10. CREATE Biostat Core Meeting, Cape Town Sep 28, 200510 Percentile Method If is normal, agrees with delta method Interval not symmetrical if is not Transformation respecting Range preserving Robust to extreme replication No adjustment for bias due to for asymmetry

11. CREATE Biostat Core Meeting, Cape Town Sep 28, 200511 BCa  All the advantages of percentile method  Adjustment for bias due to asymmetry  More accurate in terms of coverage

12. CREATE Biostat Core Meeting, Cape Town Sep 28, 200512 Simulation (1) –Balanced CRT with two treatment groups clusters of fixed size varying but equal ICC in cost in two groups and zero for the effectiveness –Cost - Random effects model framework used separately for each arm, between and within cluster effects assumed normal –Effect - A correlated normal variable generated within each cluster and dichotomized – 0.20 in control and 0.40 in treatment and cost- effect correlation 0.30 in each group –Mean cost in control and treatment set at $20.00 and $30.00 –R = $50.00 –Box-Cox transformation of normal for positively skewed cost data

13. CREATE Biostat Core Meeting, Cape Town Sep 28, 200513 Simulation (2) –54 Scenarios: number of clusters (12, 24, 48) cluster sizes (25, 50, 100) ICC (0.25, 0.10, 0.01) Normal/positively skewed cost data –Sum of between and within components of variance in cost data constrained to be 100 –2000 simulation replications of data under each scenario –All four types of intervals computed for each replication –The programming for simulation and analysis in R

14. CREATE Biostat Core Meeting, Cape Town Sep 28, 200514 Skewness & Variability

15. CREATE Biostat Core Meeting, Cape Town Sep 28, 200515 Skewness & Variability –The coefficient of skewness estimated as 3(mean- median)/sd –Distributions of the simulation replications of are generally positively skewed. –Increasing the number of clusters and the cluster sizes leads to more symmetrical distribution of –Distribution of is more skewed with smaller ICC – with more within cluster variability –The order of the numerator and denominator coefficients of skewness is about the same as observed in most cost-effectiveness studies.

16. CREATE Biostat Core Meeting, Cape Town Sep 28, 200516 Histograms

17. CREATE Biostat Core Meeting, Cape Town Sep 28, 200517 95% Confidence Intervals

18. CREATE Biostat Core Meeting, Cape Town Sep 28, 200518 Results (1)  Coverage: Proportion of intervals containing R  Width:  Shape:  ICER is highly unstable with small effect difference.  If |R| < 0.0001 then R= +/- 0.0001.  Replications of R are bounded by Chebychev’s inequality - the probability of having a more extreme replication < 1/100.  Possibly no effect on bootstrap confidence intervals.

19. CREATE Biostat Core Meeting, Cape Town Sep 28, 200519 Results (2)  In terms of coverage all the four methods seem to perform equally well  Generally the coverage falls below the nominal value of 0.95  The coverage does not seem to improve with increase in sample size  The lower levels of ICC seem to be associated with better coverage of the intervals.  The width of the intervals shrinks with the increase in the number of clusters and the cluster size  The shape of the intervals tends to be more even with large cluster and of bigger size.

20. CREATE Biostat Core Meeting, Cape Town Sep 28, 200520 Results (3)  The shapes of the Fieller’s, Percentile and the BCa intervals are similar and reflect the direction of asymmetry in the distribution of ICER.  The distribution of simulation replications of R is more symmetrical as the number of clusters and the size of the clusters increase. This translates to Fieller’s, Percentile and BCa intervals to be more even around the R. On top of this trend, these intervals are more symmetrical with smaller ICC. The same trend is evident in the width of the intervals. Further the width of the intervals shrinks with reduced levels of ICC.  It is evident that confidence intervals ignoring the ICC will be shorter in length and would not provide the desired coverage probability and would be misleading.

21. CREATE Biostat Core Meeting, Cape Town Sep 28, 200521 Results (4)  The big question – why all the methods perform equally well in terms of coverage when the distribution of replications of R is clearly positively skewed?  The overall sample is big in each combination?  The cost data are assumed normally distributed?  The cluster specific means are used in the analysis leading to normality by CLT?

22. CREATE Biostat Core Meeting, Cape Town Sep 28, 200522 References 1.Chaudhary, M.A., and Stearns, S.C., "Estimating Confidence Intervals for Cost Effectiveness Ratios: An Example from a Randomized Trial", Statistics in Medicine, Vol. 15, 1447-1458 (1996) 2.O'Brien, B.J., M.F. Drummond, R.J. LaBelle, A. Willan 'In Search of Power and Significance: Issues in the Design and Analysis of Stochastic Cost-Effectiveness Studies in Health Care', Medical Care, 32(2):150-163 (1994) 3.Mullahy, J. and W. Manning 'Chapter Eight: Statistical Issues in Cost-Effectiveness Analyses', in Costs, Benefits, and Effectiveness of Pharmaceuticals and Other Medical Technologies, edited by Frank Sloan, Cambridge University Press. (1995) 4.Cochran, W. G. Sampling Techniques, John Wiley and Sons. N.Y. (1977) 5.Efron, B., and Tibshirani, R.J. An Introduction to the Bootstrap, Chapman and Hall, N.Y. (1993)