Simulation methods for calculating the conditional power in interim analysis: The case of an interim result opposite to the initial hypothesis in a life-threatening disease.

Somatostin plus Isosorbide-5-Mononitrate vs Somatostatin in the control of acute gastro-oesophageal variceal bleeding: a double blind, randomized, placebo- controlled clinical trial. Junquera F, et al. GUT 2000; 46 (1)

Design Disease –Acute variceal bleeding in cirrhotic patients Objective –To test whether the addition of oral Isosorbide 5-Mononitrate (Is-5-Mn) improved the efficacy of Somatostatine (SMS) alone in the control of bleeding.

Design Treatments –Group 1: SMS + PLB (Control) –Group 2: SMS + Is-5-Mn (Experimental) Working hypothesis –The rate of success would increase from 60% to 90%.

Sample size: Pre-determination n=n per group  2 = variance  = effect size f( ,  ) = function of type I and II errors n =  2 /  2 * f( ,  )

Statistical errors: f( ,  )

Fixed sample size ALPHA = 0.05 POWER = 0.90 P1 = 0.90 P0 = 0.60 Case sample size for uncorrected chi-squared test: 42

Introduction: interim analyses Often ethical concerns on these situations, specially in life-threatening diseases. Sometimes, pre-defined working hypothesis may not adjust to reality. –Treatments may be better than expected –Treatments may be worse than expected (safety and/or efficacy) Long studies or big sample sizes make advisable some kind of interim control.

Introduction At some fixed times, cumulated data can be analysed and decisions may be taken in base to the findings. Multiple analysis can lead to statistical errors and mistaken clinical decisions. Several methods deal with multiplicity issues.

Design For ethical reasons the design allows an interim analysis, when half of the sample size is recruited. Pocock’s group sequential method (1977)  = 0.05  = 0.1(power 90%) p 0 = 60%,p 1 =90%

Group Sequential Methods

 adjusted sample size ALPHA = POWER = 0.90 P1 = 0.90 P0 = 0.60 Case sample size for uncorrected chi-squared test: 48

Digestive System Research Unit Liver Unit Pharmacist Statistician Clinical Pharmacologist Internal ParticipantsMonitoring Comittee

50% Sample size with evaluated outcome Statistical analysis:  50 patients finalised Data for Interim analysis

Interim analysis Chi-square=2.427, p-value=0.119 OR 1 (observed):3.11 (0.72 –13.51) OR r (design): 0.17

Problem statement Evidence from interim analysis against working hypothesis Although no statistical evidence supporting study termination, clinical criteria suggested so. Search for objective support to clinical intuition.

50% Sample size with evaluated outcome Statistical analysis:  50 patients finalised Data for Interim analysis Recruitment:  10 patients

Possible solutions 1) Group sequential methods 2) Alpha spending function approach 3) Repeated confidence intervals 4) Stochastic curtailing methods 5) Bayesian methods 6) Boundaries approach (likelihood function)

Conditional power Negative results: –CAST (I-II) study. NEJM (1989 & 1992) Group sequential testing using permutation distribution & stochastic curtailment methods –HPMPC trial, Ann Intern Med 1997 –ACTG Study 243. NEJM 1998

Conditional power Positive results: –CRYO-ROP Arch Ophthalmology,1988 –Grable el al. Am J Obstet Gynecol, 1996 Extension of trial: –Proschan MA, Biometrics, 1995

Stochastic curtailment Lan, Simon y Halperin (1982) Stop if in i inspection:  0, P(reject H 0 |  ) is high at the end  0, P(reject H 0 |  ) is small at the end

Application to real data design: p(ctr) = 60% p(exp) = 90% 1st Inspection (50 patients) : p(ctr) = 87.5% p(exp) = 69.2% Probability of proving the working hypothesis at the end (100 patients) projecting the results from this inspection

Methods: OR design: 0.17 =>  r = log(OR) = Simulations: –Fortran 90 – 1,000,000 studies =>precision < 0.01% –15 possibilities ranging from –1.5 x  r to +1.5 x  r

Effect Size x  r +1.5 x  r  x  r  x  r Observed Design  /  r OR r design: 0.17  r = log(OR) = -1.79

H0 Obs H1

Conditional power calculation

 1 (1 st inspection)  r (design)

P(  <  1 |  /  r = 1.00) = 53/1,000,000 P(  <  1 |  /  r = 1.25) = 2/1,000,000 P(  <  1 |  /  r = 1.50) = 0/1,000,000

Interim analysis after completion of 10 more patients Chi-square=4.794, p-value=0.029 OR 1’ (observed): 4.00 OR r (design): 0.17

Final Interpretation The study was interrupted not based in the sequential pre-defined rule. The clinical intuition was confirmed by the conditional power calculation. The study was finished due to: –The low likeliness of the working hypothesis –The high probability of a worse outcome with the experimental treatment

Conclusions Simulations may be a very useful tool in some design and analysis situations, as it has been shown in this case of the conditional power calculation.