Examination of Analysis Methods for Positive Continuous Dependent Variables: Model Fit and Cost Saving Implications Brian P SmithMaria De Yoreo Biostatistics DirectorDepartment of Applied Mathematics UC Santa Cruz May 22, 2013 Midwest Biostatistics Workshop; Muncie, IN

2 Personal Motivation Compositional Data Analysis Using Liouville Distributions … - Forgettable Ph.D. Dissertation by BP Smith Compositional Data – Multivariate Data That Sum to 1 Clay – 0.2, Silt , Sand John Aitchison – The Statistical Analysis of Compositional Data ln odds – ln (x1/x3), ln(x2/x3) – Bivariate Normal

3 Basic principle Underlying distribution should match the sample space of the data If using multivariate normal, then must transform compositional data from Simplex  Multivariate Reals Could use Dirichlet or Liouville

4 How to follow principle with positive valued data? log transformation – Positive reals to reals Yet, colleagues were using natural scale or percent change from baseline Why? –That was what had always been done –Central limit theorem protection for type 1 error Easy to show with simulation if true distribution is log-normal and use normal distribution to analyze then there is a power loss

5 What do the critics think? Real data is not log-normal or normal So what factor Arguing a theoretical argument for a real world problem

6 Personal Motivation Part 2 It is generally accepted among statisticians that in a clinical trials the simple use of baseline as a covariate provides more power More than once with scientist – “What is this analysis of covariance, we should just do percent change from baseline.” “That is the analysis Jennings did in their paper...” Or “this is what Goodguy Pharmaceuticals did in their NDA” Me – “But you will lose power” but I have already lost this argument There appears to me to be a higher appreciation that good design can affect power than good analysis.

7 What Do I (and Maybe Some of You, if you are like minded) need? Research that not only suggests that log-transformation is better for positive data But also quantifies how much better Research that not only suggests analysis of covariance is better But also quantifies how much better This should exist, right? Not that I can find

8 What Did We Do? 70 Continuous Endpoints Analyzed 10 Analyses Endpoints Each –4 Phase 1 Studies –1 Phase 2 Study –1 Phase 3 Study 10 Endpoints Chosen from 3 Preclinical Studies

9 What Did We Do? (cont) Chose primary or secondary endpoints if continuous 1-3 per study Remaining 7-9 randomly selected from –ECGs – Vitals –Laboratory Measurements Variety of endpoints from range of studies chosen in non-subjective manner

10 The Analyses All endpoints had repeated observations over time Used Mixed Effect Model –Random subject effect –Fixed Effects Treatment Time Treatment by Time Interaction –If Cross-over study, additional random effects added 8 models examined for each endpoint

11 Eight Models IdentifierResponseCovariate for BL? UNYno LNLn(y)no URy-BLno LRLn(y/BL)no PR100∙(y-BL)/BLno UCyYes; BL LCLn(y)Yes; ln(BL) PC100∙(y-BL)/BLYes; BL

12 Three Means of Comparison For ANCOVA Only –P-value of Covariate For Log Scale –Compare Likelihoods For All Analyses –Compare Costs

13 How to Compare Costs? Compare Standard Errors of Estimates for Treatment Effect Determine change in sample size that would be needed under one model to obtain a standard error equivalent to that of another model Scaling Issue due to log-transformation If no scaling issue and two models (se1/se2) 2 is how many fold more subjects that analysis 1 would need to have the same standard error as analysis 2

14 Dealing with the Scaling Issue Natural Scale Log Scale – Consider If start with log scale and work towards natural scale

15 Which to use? If data is skewed right then Geometric Mean < Mean Use of the mean favors the natural scale (most conservative) Use of geometric mean more consistent with data We do both but Prefer Geometric Mean

16 Back to comparing cost Is the fold increase in subjects needed for the natural scale to be equivalent to the log-scale Similar argument for scaling for percent change from baseline

17 The Case for ANCOVA Comparison% p-value < 0.05 ANCOVA versus No Baseline Adjustment Natural Scale90 Log Scale90 ANCOVA versus “Change from Baseline” Natural Scale60 Log Scale65 % Change from Baseline57.5

18 The Case for ANCOVA Cont. ComparisonAverage Fold-Increase In Sample Size ANCOVA versus No Baseline Adjustment Natural Scale3.32 Log Scale3.72 ANCOVA versus “Change from Baseline” Natural Scale1.25 Log Scale1.48 % Change from Baseline1.29

19 The Case for Log Ratio over Percent Change from Baseline Comparison% Likelihood Log Ratio > Likelihood Percent Change from Baseline No Covariate80 Covariate80

20 Likelihood Plots

21 The Case for Log Ratio over Percent Change from Baseline (Cont) ComparisonAverage Fold-Increase In Sample Size With MeanWith Geometric Mean No Covariate Covariate

22 The Case for Log over Natural Scale Comparison% Likelihood Log Ratio > Likelihood Percent Change from Baseline No Baseline Adjustiment80 “Change from Baseline”79 ANCOVA82

23 Likelihood Plots

24 The Case for Log Ratio over Natural Scale (Cont) ComparisonAverage Fold-Increase In Sample Size With MeanWith Geometric Mean No Baseline Adjustiment “Change from Baseline” ANCOVA

25 Conclusions Don’t just trust us, do it yourself If these results continue to replicate can conclude –If a baseline is available, use of baseline as a covariate should always be undertaken –Although we recommend exploration of data from previous studies, percent change from baseline analyses should not be undertaken unless there is strong empirical evidence that for that endpoint it is preferred –Again with the caveat that nothing replaces exploration of data from previous studies, log-transformation ought to be the default analysis of positive data unless exploration of previous data provides convincing evidence that the natural scale is preferred.