1. Mark Rothmann U.S. Food and Drug Administration September 14, 2018
Applying Hierarchical Models When Evaluating Treatment Effects Across Regions
Mark Rothmann
U.S. Food and Drug Administration
September 14, 2018

2. Acknowledgements Nairita Ghosal Kiya Hamilton Gene Pennello
James Travis
Yun Wang

3. Outline What is shrinkage estimation and why use it?
Hierarchical Models
Two examples
One-way hierarchical model
One-way hierarchical model that includes a known effect modifier
4. Some remarks

4. Challenges in analyses across regions
Multiplicity; random highs and random lows
Lack of power for testing within a given region
Satisfying multiple regulatory authorities
Providing sufficient information applicable to each region

5. What is Shrinkage Estimation?
A shrinkage estimate of a parameter for a subgroup
a weighted average of sample estimate and overall estimate (stratified by subgroup).
Could be a posterior mean in a Bayesian setting
Sample estimate is “shrunk” towards the overall estimate

6. Linking
The parameters across subgroups are linked. This linking of the parameters makes their estimation linked. This is usually done through the assumption of exchangeability.

7. Exchangeability
Treatment effects across subgroups are exchangeable if their possible orderings are considered equally likely a priori (i.e., before seeing data).
Put another way, if subgroup-specific treatment effects were revealed, but their labels were not, then effects would not be helpful predicting their labels.
The assumption of having treatment effects drawn randomly from the same distribution implies that treatment effects are exchangeable.

8. Why use shrinkage estimation for subgroup analysis?

9. Two components of variability in sample estimates across subgroups
The total variability in the sample estimates is sum of
the within subgroup variability of the sample estimator and
the across subgroups variability in the underlying/true parameter values

10. Purpose of Shrinkage Estimation
To address within subgroup variability in the estimation
Obtain estimates where the unaccounted variability is primarily across subgroups variability
The difference between the sample estimate and the shrinkage estimate is the estimated sample deviation (no longer estimating the sample deviation as zero)

11. Properties of Shrinkage Estimation
Greater precision
narrower 95% CIs
Addresses random highs and random lows
95% CIs have 95% coverage after the data are known

12. Hierarchical Models
Statistical model written in multiple levels (hierarchical form).

13. Hierarchical Models
Incorporate shrinkage estimation through (Bayesian) hierarchical models.
Examples
Over regions for a time to event endpoint
Over regions for HbA1c where baseline HbA1c is an effect modifier

14. Hierarchical Models applied to regions
Underlying treatment effects across regions are linked. Underlying treatment effects are regarded as exchangeable.
Does not estimate treatment effect within a region in isolation of all other regions
Estimation of the treatment effects across regions are linked. For the estimating treatment effect in a given region, data from all regions are relevant, with data from the given region more relevant.

15. Example 1: LEADER trial Cardiovascular outcome trial
Liraglutide vs. placebo
Time to first major adverse cardiac event
Rule out a hazard ratio greater than 1.3
Overall Result: HR = % CI (0.78, 0.97)

16. Subgroup Analyses Results Region HR (95% CI) Asia 0.62 (0.37, 1.04)
Europe
0.82 (0.68, 0.98)
North America
1.01 (0.83, 1.22)
The Rest of The World
0.83 (0.68, 1.03)

17. Modeling Assumptions 1/2 is distributed Gamma (.001, .001)
 is distributed Normal mean 0, variance 16
i is distribution Normal mean , variance 2 i =1, 2,3 ,4
For i = 1, 2,3, 4 Yi represents the observed subgroup log hazard ratio
Yi is distributed Normal mean i variance i2 where
12 = 0.069, 22 = , 32 = and 42 = 0.11

18. Shrinkage Analysis Sample estimate Bayes Shrinkage estimate Region HR
Sample estimate
Bayes Shrinkage estimate
Region HR
95% CI
Asia
0.62
(0.37, 1.04)
0.80
(0.59, 1.09)
Europe
0.82
(0.68, 0.98)
0.84
(0.71, 0.98)
North America
1.01
(0.83, 1.22)
0.94
(0.79, 1.12)
The Rest of the World
0.8323
(0.68, 1.03)
0.85
(0.72, 1.00)

19. Shrinkage Analysis Sample estimate Bayes Shrinkage estimate Region HR
Sample estimate
Bayes Shrinkage estimate
Region HR
95% CI
Asia
0.62
(0.37, 1.04)
0.80
(0.59, 1.09)
Europe
0.82
(0.68, 0.98)
0.84
(0.71, 0.98)
North America
1.01
(0.83, 1.22)
0.94
(0.79, 1.12)
The Rest of the World
0.8323
(0.68, 1.03)
0.85
(0.72, 1.00)

20. Shrinkage Analysis Sample estimate Bayes Shrinkage estimate Region HR
Sample estimate
Bayes Shrinkage estimate
Region HR
95% CI
Asia
0.62
(0.37, 1.04)
0.80
(0.59, 1.09)
Europe
0.82
(0.68, 0.98)
0.84
(0.71, 0.98)
North America
1.01
(0.83, 1.22)
0.94
(0.79, 1.12)
The Rest of the World
0.8323
(0.68, 1.03)
0.85
(0.72, 1.00)

21. Example 2: HbA1c change Add-on study
Baseline HbA1c is an effect modifier
Mean Baseline HbA1c varies across regions

22. Different types of effects
Subgroup effects – does not adjust for effect modifiers (is based on the observed distributions of effect modifiers)
Main effects for subgroup – remaining effect after accounting for known effect modifiers

23. Sample results Region Mean Baseline HbA1c
Sample Estimate (%) of Treatment Effect
Sample Standard Error A
8.6
-0.34
0.13 B
8.9
-0.36
0.29 C
9.1
-0.38
0.19

24. Model Model on Patient Level Data: Priors: 𝛽 1 ~𝑁(0.5, 100)
𝑦 𝑖 = 𝛽 0 + 𝛽 1 ∗𝑇𝑅𝑇+ 𝛽 2 ∗ (𝐵𝑎𝑠𝑒 𝑖 − 𝐵𝑎𝑠𝑒 )+ 𝛽 3 ∗𝑟𝑒𝑔𝑖𝑜𝑛𝐵+ 𝛽 4 ∗𝑟𝑒𝑔𝑖𝑜𝑛𝐶+𝛽 5 ∗
𝑇𝑅𝑇∗ (𝐵𝑎𝑠𝑒 𝑖 − 𝐵𝑎𝑠𝑒 )+ 𝛽 6 ∗𝑟𝑒𝑔𝑖𝑜𝑛𝐵∗𝑇𝑅𝑇+ 𝛽 7 ∗𝑟𝑒𝑔𝑖𝑜𝑛𝐶∗𝑇𝑅𝑇+ 𝛽 8 ∗𝑆𝑇𝑅𝐴𝑇𝐴
Likelihood: 𝒚 𝒊 |𝜷 ~ 𝑁 ( 𝑿𝜷, 𝜎 2 𝑰)
Priors:
𝛽 1 ~𝑁(0.5, 100)
𝛽 5 ~𝑁(0.3, 100)
𝛽 𝑖 ~𝑁 0, 1 , 𝑖 ≠1,5
𝜎~ 𝐻𝑎𝑙𝑓−𝑁𝑜𝑟𝑚𝑎𝑙

25. Shrinkage estimates of main effects
Sample estimate
Shrinkage estimate (%)
Standard error
Region B vs. A
-0.02
-0.01
0.3
Region C vs. A
-0.04
0.2

26. Concluding Remarks Greater precision
narrower 95% CIs
Addresses random highs and random lows
95% CIs have 95% coverage after the data are known
When exchangeability assumptions do not hold, can possibly do even better than the shrinkage estimates based on exchangeability

27. Remarks on some choices for shrinkage estimation
Model Assumptions
𝛿 𝑖 ~𝑁(𝛿, 𝜏 2 )
𝛿 𝑖 ~𝑁( 𝛿 𝑖 , 2𝜎 2 / 𝑁 𝑖 )

28. Posterior mean of 𝛿 𝑖 𝑤 (𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒 𝑜𝑓 𝛿 𝑖 )+(1−𝑤)𝛿 where 𝑤= 𝜏 2 𝜏 2 + 2𝜎 2 / 𝑁 𝑖 . 𝜏 2 , 𝜎 2 , 𝛿 are known. Do not get best estimator of 𝛿 𝑖 by 𝜏 2 𝜏 𝜎 2 / 𝑁 𝑖 𝛿 𝑖 +(1− 𝜏 2 𝜏 𝜎 2 / 𝑁 𝑖 ) 𝛿 𝑅𝐸𝑀 Not estimating 𝑎+𝑏𝛿 where a, b≠0 are constant

29. Var( 𝛿 𝑖 ) = 𝐸( 𝛿 𝑖 −𝛿) 2 where 𝐸 𝛿 𝑖 =𝛿. Interested in
Not interested in
Var( 𝛿 𝑖 ) = 𝐸( 𝛿 𝑖 −𝛿) 2 where 𝐸 𝛿 𝑖 =𝛿.
Interested in
𝐸( 𝛿 𝑖 − 𝛿 𝑖 ) 2 where 𝐸 𝛿 𝑖 𝛿 𝑖 = 𝛿 𝑖

30. Chapter from a new book
Pennello G., Rothmann M. Bayesian Subgroup Analysis with Hierarchical Models, in Biopharmaceutical Applied Statistics Symposium Volume 2: Biostatistical Analysis of Clinical Trials, Eds. Karl E. Peace, Ding-Geng Chen, Sandeep Menon, Springer

31. Thank You!

32. Backup Slide
If trial subjects in ith region were random sample from all patients in that region having condition of interest, then 𝛿 𝑖 is the underlying treatment effect for that region.
If regions are collectively exhaustive, there are no additional regions to generalize results
For the treatment effect from multiple studies
Main effects for study exchangeable
Main effects for region exchangeable
Interaction effects for study by region exchangeable