1. Richard K Burdick Elion Labs MBSW Meetings May 2016
Using the Confidence Interval on Effect Size to Demonstrate Analytical Similarity Between Reference and Biosimilar Products
Richard K Burdick
Elion Labs
MBSW Meetings
May 2016
Amgen Corporate Template

2. Collaborators Harry Yang-MedImmune, LLC Steven Novick-MedImmune, LLC
Neal Thomas, Pfizer, Statistical Research and Consulting Center
Aili Cheng, Pfizer, Pharmaceutical Sciences and Manufacturing Statistics

3. Introduction
The FDA Briefing Document for the Oncologic Drugs Advisory Committee (ODAC) Meeting held on January 7, 2015, as well as recent presentations and information provided by Chow (2014) describes the statistical procedure recommended for demonstration of analytical similarity for biosimilar applications.
This approach uses a three-tier system based on attribute criticality with a different statistical procedure applied for each tier.

4. Tier 1 Testing
Tier 1 testing is based on a statistical test of equivalence.
A statistical test of equivalence must “prove” the difference in averages between the reference product (RP) and test product (TP) is less than a pre-defined equivalence margin. (Burden of proof is on sponsor of TP)
The equivalence margin is 1.5sRP where sRP is the true standard deviation of the RP.

5. Variation Inherent in Tier 1 Testing
Tier 1 testing occurs under two possible conditions that impact test size:
Condition A: The value of sRP is unknown and must be estimated with sampled RP lots.
Condition B: RP lots may be correlated if sourced from the same drug substance or formulated bulk (an unknown condition for TP sponsors)
A test size exceeding the desired level, typically 5%, is problematic because it shifts a greater than desired risk to patient.
That is, the risk of passing a TP product that is not analytically similar to the RP is greater than the desired level of 5%.

6. Estimation of Effect Size
One approach to mitigate these conditions is to construct a confidence interval on the effect size directly.
Effect Size

7. Estimation of Effect Size
By changing hypotheses in this manner, sRP is now contained in the parameter of interest, and the uncertainty in its true value is properly accounted for in the confidence interval.
The estimated parameter is now compared to a constant rather than another estimate.
This format also allows appropriate adjustment for correlation when the correlation structure can be identified.

8. Performing Calculations
Equal variances
Exact closed-form solution is based on the inversion confidence interval principle and the non-central t-distribution (see, e.g., Hedges (1981)).
The general approach for constructing a confidence interval on the effect size is to first form a confidence interval on the non-centrality parameter of the t-distribution associated with the test statistic
This non-centrality parameter is

9. Performing Calculations
To determine a two-sided 90% confidence interval on l, determine the lower bound lL such that
and the upper bound lU by such that
Here is the SAS code using the function tnonct to compute the confidence interval:
Converts from l to
effect size

10. Performing Calculations
Unequal variances
An “exact” procedure can be performed using a generalized confidence interval (GCI).
The solution is simulation based, requiring simulation of independent normal and chi-squared random variables.
SAS code is available from the author, but can be written in any other software package including Excel.
Power is impacted by relative sizes of sRP and sTP, but test size is “exact”.

11. Unequal Variances
Using a recipe provided in Appendix B.2 of Burdick, Borror, and Montgomery (2005) perform the following steps:

12. Unequal Variances

13. Example

14. 90% CI

15. Unequal Variances With Correlation
Reference drug product lots may be correlated if sourced from the same drug substance or formulated bulk.
As noted by Ramirez and Burdick (2015), correlation among RP lots can lead to an inflated Type I error rate for the TOST.
Since a sponsor has information concerning TP lots, sourcing of these lots will be known so that independently sourced lots can be selected in the analysis.
In contrast, the lack of sponsor knowledge concerning sourcing of the RP lots makes this risk problematic.

16. Unequal Variances With Correlation
If the correlation structure can be identified, then a GCI can be constructed.
For example, assume the correlation structure generated by a one-factor random model
where there are RP lots created by sourcing r DP lots each from n DS lots.

17. One-Factor Random Model
is a chi-squared random variable with n-1 df
is a chi-squared random variable with n(r-1) df
Here, two DP lots sourced from the same DS lot have a correlation of

18. Unequal Variances With Correlation
Replace ESsim in previous recipe with the quantity

19. Simulation Study TOST1=Unequal variance CI on difference
TOST2=Pooled variance CI on difference
GCI1=Unequal variance CI on effect size (ESsim)
GCI2=Pooled variance CI on effect size
GCI3=Unequal variance CI on effect size with correlation adjustment (ES2sim)
Number of DP lots per DS lot (r=1,2,5)
NRP=10,20 (NTP=10)
10,000 simulations with GCI based on 2,000 iterations.
Correlation is 0.8

24. Identification of Correlated Lots
Repeated measures made on RP lots
DS1
DS2
DS3
Lot A
Lot B
Lot C
Lot D
Lot E
Lot F
Mean
99.893
99.933
Standard deviation
0.335
0.346
0.323
0.268
0.36
Sample size 30
Fisher’s LSD Test
Lot
Mean
Grouping E A F B D
99.933 C
99.893

25. Recommendations to Sponsors
Purchase RP lots over an extended time frame to decrease the likelihood of obtaining DP lots sourced with same DS. This strategy will also provide an opportunity for the lot-to-lot variation to fully manifest and provide a better estimate of the RP standard deviation.
It might be possible for the sponsor to identify the source material of RP lots using a stable isotope profile. (e.g. Apostol et al. (2001)).
An empirical examination of repeated measurements of RP DP lots can be useful in identifying lots sourced with the same DS.
If identifiable, do not use DP lots sourced from the same DS. Although the data can be modeled using the GCI approach, it will decrease power.

26. References
Apostol, I., Brooks, P. D., and A. J. Mathews. (2001). “Application of high-precision isotope ratio monitoring mass spectrometry to identify the biosynthetic origins of proteins”, Protein Science , July 10(7), pages
Burdick, R. K., Borror, C.M., and Montgomery, D. C. (2005). Design and analysis of gauge R&R studies: Making decisions with confidence intervals in random and mixed ANOVA models, SIAM, PA.
Burdick, R. K. and Ramírez, J. G. (2015) “Statistical issues in biosimilar analytical assessment: Perspectives on FDA ODAC analysis, DIA Conference, Washington, D. C., April.
Chow, S.-C. (2014). “On assessment of analytical similarity in biosimilar studies”, Drug Designing 3: 119. doi: / e124 (accessed March 2, 2015).
Hedges, L. V. (1981). “Distribution theory for Glass’s estimator of effect size and related estimators”, Journal of Educational Statistics, Vol. 6 (2), pages
FDA Briefing Document for the Oncologic Drugs Advisory Committee (ODAC) Meeting held on January 7, 2015 (