1. Design and statistical analysis of method transfer studies for biotechnology products
Meiyu Shen, Lixin Xu
Center for Drug Evaluation and Research,
U.S. Food and Drug Administration
Good afternoon, I would like to thank Dr. Tsong for a nice introduction and thank the organizers for providing me this opportunity. my name is Meiyu Shen. Today, I am going to present design and statistical analysis of method transfer studies for biotechnology products. This material has been recently published in Bioanalysis. This is the FDA standard disclaimer.
This presentation reflects the views of the author and should not be construed to represent FDA’s views or policies

2. Outline Method development and its life cycle management
Purpose of analytical method transfer studies
What parameters compared in analytical method transfer studies
Testing materials
Analysis
Conclusion
This slide shows the outline of my talk. I start with a brief introduction of method development and its life cycle management. Then I will discuss the purpose of analytical studies followed by what parameters compared in analytical method transfer studies. Furthermore, I will discuss some design elements and statistical analyses for method transfer studies.

3. New analytical method development
Parameters evaluated
Specificity
Linearity
Accuracy
Precision
Limits of detection (LOD)
Limits of Quantitation (LOQ)
Range
During a new analytical method development, the following parameters should be evaluated: specificity, linearity, accuracy, precision, limits of detection, limits of quantitation, and linear range.

4. Life cycle management of analytical procedures
Including, but not limited to
Trend analysis on method performance at regular intervals
to optimize the analytical procedure
to revalidate all or a part of the analytical procedure
Development and validation of a new or alternative analytical method
A new impurity
Method transferred to a new testing site
After a new analytical procedure has been developed, its life cycle management should include, but not limited to: trend analysis on method performance at regular intervals to optimize the analytical procedure and to revalidate all or a part of the analytical procedure’ development and validation of a new or alternative analytical method for a new impurity; and method transferred to a new testing site.

5. Analytical method transfer studies
The purpose of method transfer studies (internal)
To determine if the two laboratories provide comparable results across the range of interest.
If so, then to transfer a fully validated analytical method from the originating lab to a new lab (receiving lab)
Once transferred, the method is suitable for its intended use and can be used to ensure process consistency and meet product specifications
The purpose of method transfer studies is to determine if the two laboratories provide comparable results across the range of interest.
If so, then to transfer a fully validated analytical method from the originating lab to a new lab (receiving lab)
Once transferred, the method is suitable for its intended use and can be used to ensure process consistency and meet product specifications

6. Analytical method transfer studies
How to achieve the goal?
Obtaining the comparative data from method transfer studies
Checking the receiving lab’s bias (difference between the true value and the mean of the receiving lab)
Determining success of implementation of the fully validated analytical method in the receiving lab
How can we achieve analytical method transfer? First we obtain the comparative data from method transfer studies;
Check the receiving lab’s bias (difference between the true value and the mean of the receiving lab);
and determining success of implementation of the fully validated analytical method in the receiving lab

7. Important Factors in Method transfer studies
Suppose that the same type of instrument from the same manufacturer, same reagents, same experimental conditions, and same testing procedure, we investigate the following factors:
operators
days
runs
Replicates
lots
Next, I will discuss design factors in method transfer studies.
Suppose that the same type of instrument from the same manufacturer, same reagents, same experimental conditions, and same testing procedure, we investigate the following factors:
operators
days
runs
Replicates
lots

8. Key parameters in method transfer studies
Mean shift (often incorrectly cited as accuracy)
Comparing means of two labs
Precision
Comparing the standard deviations of two labs
Bias (accuracy)
Key parameters in method transfer studies are Mean shift (often incorrectly cited as accuracy)
--Comparing means of two labs ; Precision--Comparing the standard deviations of two labs; and Bias (accuracy)

9. Testing materials
Is the reference standard appropriate material from which comparative data is obtained for method transfer studies? No
Since the method is used to ensure process consistency and meet product specifications
What materials should be tested? Is the reference standard appropriate material from which comparative data is obtained for method transfer studies?
No since the method is used to ensure process consistency and meet product specifications

10. Testing materials
Multiple lots of a drug product if the assay is used for drug releasing tests
Multiple lots of a drug substance if assay is used for measuring the content in drug substance
Forced degradation samples or samples of a drug substance or a drug product containing pertinent product-related impurities if the transferred assay is stability indicating
Multiple lots of a drug product should be tested if the assay is used for drug releasing tests
Multiple lots of a drug substance should be tested if assay is used for measuring the content in drug substance
Forced degradation samples or samples of a drug substance or a drug product containing pertinent product-related impurities should be tested if the transferred assay is stability indicating

11. Literature review: statistical analysis
Many proposals, just name a few here
Significance testing approach
Comparing the means of two labs by the p-value of rejecting H0: μR=μS
Comment: Discouraging the sponsors to use a large sample size and to obtain more precise measurement
Quality control method
Checking individual values against the control limit
Comment: Not quantitative criteria for decision
Now let us look at the literature review on statistical analyses. There are many proposals. Here I just name a few. One option is the significance testing approach which Compares the means of two labs by the p-value of rejecting H0: μR=μS
My comment is that the p-value approach Discourages the sponsors to use a large sample size and to obtain more precise measurement
Another approach is Quality control method
Which checks individual values against the control limit
My comment: Not quantitative criteria for decision

12. Literature review: statistical analysis
β-expected tolerance approach
Calculating the tolerance interval in which a proportion (β) of the receiving laboratory population is expected to fall within,
Compares the above tolerance interval to acceptance limits around the mean estimate of the sending laboratory
Comments: challenge to define the acceptance limits.
β-content tolerance interval approach to assure more than 100P% of the individual difference (or percent difference, d) between individual results obtained in the sending laboratory and the receiving laboratory are within the predefined boundary (L, U) with 100(1 - α)% confidence level.
Two-sided tolerance interval
Two one-sided tolerance interval
Comments: challenge to define L, U, and P
Another approach is β-expected tolerance approach
In Which we calculate the tolerance interval in which a proportion (β) of the receiving laboratory population is expected to fall within,
And then compare the above tolerance interval to acceptance limits around the mean estimate of the sending laboratory
challenge is how to define the acceptance limits.
Another is β-content tolerance interval approach to assure more than 100P% of the individual difference (or percent difference, d) between individual results obtained in the sending laboratory and the receiving laboratory are within the predefined boundary (L, U) with 100(1 - α)% confidence level.
Two-sided tolerance interval
Two one-sided tolerance interval
Comments: challenge to define L and U

13. Our proposal: Equivalence test for comparing means of two laboratories
Denote the means of the response variable of interest by μR and μS , respectively, for the receiving laboratory and the sending laboratory.
(Equation 1)
Here δ is a pre-specified constant, also called an equivalence margin.

14. Challenge of setting equivalence margin for equivalence approach
Fixed margin
Based on the experts’ knowledge
Different margin for a different assay
1% , a reasonable margin for HPLC
Too stringent for bioassay
2.5%, a reasonable margin for a specific bioassay
Too liberal for HPLC
Wider than specification 2% for drug substance assay
Challenge: It is hard to have a number
We face the following challenge for setting equivalence margin for equivalence approach.
The first option is fixed margin based on the experts’ knowledge
But it is different margin for a different assay
For example, 1% is a reasonable margin for HPLC but Too stringent for bioassay.
2.5%, a reasonable margin for a specific bioassay but Too liberal for HPLC
Since it is wider than specification 2% for drug substance assay
In summary, it is hard to have a number

15. Challenge of setting equivalence margin for equivalence approach
Non-fixed margin: a function of assay variability
Unified rule for many assays
Based on statistical power for rejecting the null hypothesis in the equivalence hypothesis test with a limit number of observations (not exceeding hundreds)
All margins sits well within the assay specification.
One option is setting up Non-fixed margin: a function of assay variability
It can be developed under unified rule for many assays
It is developed based on statistical power for rejecting the null hypothesis in the equivalence hypothesis test with a limit number of observations (not exceeding hundreds)
All margins sits well within the assay specification.

16. How to obtain the assay variability
Long term quality control data
Not appropriate, e.g.,
If there is a stability trend over the time
If there is a drift from assay instruments over the time
Only good if there is no other confounding factor except operators, days, and runs
Hard to meet this criteria
Comparative method transfer studies
We may estimate the assay variability from studies
Next I will discuss how to obtain the assay variability.
Can we use Long term quality control data
Not appropriate, e.g.,
If there is a stability trend over the time
If there is a drift from assay instruments over the time
Only good if there is no other confounding factor except operators, days, and runs
Hard to meet this criteria
Well, we may estimate the assay variability from Comparative method transfer studies

17. Statistical analysis for the mean difference of two labs
Hypothesis testing (1):
H0: μS – μR ≤ - cσS or μS – μR ≥ cσS
Ha: - c σS <μS – μR <c σS
where μS and μR are the mean responses of the sending lab and receiving lab, respectively, and c > 0 is the constant.
Equivalence margin
c σS
Value of c
Determined from power function of rejecting the above hypothesis if σS is known
Let us look Statistical analysis for the mean difference of two labs. Again the hypothesis is as set up as: the null is not equivalent, the alternative is equivalent within equivalence margin +- csigma_S. We use power function to determine c based on practical sample sizes.

18. Power function for the two one-sided tests procedure
Let be the probability of rejecting H0 under Ha in Hypothesis testing (1) when σS =σR= σ.
The power function is:
-θ1=θ2=cσS
n1: # of obs. in receiving lab
n2: # of obs. in sending lab
: normal cumulative function
This is the power function under these assumptions.

19. Determination of δ=CσS
Power function: n C
Power=0.80
Power=0.85 20
0.94
0.99 22
0.90
0.95 24
0.86 26
0.82
0.87 28
0.79
0.83 30
0.76
0.80
C=0.85 is reasonably chosen such that we can achieve about 85% power with a sample size in the range 20 to 30 per laboratory.
The table on the left is to show the relationship between n and c under different power column. From this table, you can see that C=0.85 is reasonably chosen such that we can achieve about 85% power with a sample size in the range 20 to 30 per laboratory.
Margin: δ=0.85 σS

20. An example: internal transfer to a new site
All equipment moved to the new site
Personnel transferred to the new site
At least 2 lots
>2 analysts and >2 days
Reasonable sample size per lab: ~20-30
Margin: e.g., 0.85σS
Power to pass equivalence test is about 85% under no true mean difference
Here All equipment moved to the new site
Personnel transferred to the new site
At least 2 lots
>2 analysts and >2 days
Reasonable sample size per lab: ~20-30
Margin: e.g., 0.85σS
Power to pass equivalence test is about 85% under no true mean difference
is an example: internal transfer to a new site.

21. Statistical analysis for the mean difference of two labs
Option 1:Treating as a constant
Estimating from the sending lab
Define and
Concluding equivalence criteria is met if and , where is the 1-α quantile of t-distribution with degrees of freedom ν, α is the nominal significance level (e.g., 0.05).
Inflate both type 1 and 2 error rates
The first option is treating 0.85 sigma_s_hat as a constant. We estimate sigma_s_hat from the sending lab. We define T1 and T2 as these. We assume T1 and T2 follow t-distribution and conclude equivqlence criteria is met if T1>talpha(mu) and T2<-talpha(mu). However, this simple method is showed to inflate both type 1 and type 2 error rates.

22. Statistical analysis (continued)
Option 2:
Considering as a random variable
Define and
Where
Concluding equivalence criteria is met if and , where is the 1-α quantile of standard normal distribution, α is the nominal significance level (e.g., 0.05).
The second option is treating 0.85sigm+s_hat as a random variable. Define T1_prime and T2_prime as these. We conclude equivalence criteria is met if T1_primer is greater than z_alpha and T2_prime us less than negative z_alpha.

23. Head-to-head approach for comparing precisions obtained in two labs
Hypothesis H0: σR≤σS
Hypothesis testing: small powers to reject H0 for small samples.
Check the point estimate
Regarding the precisions of two labs, we can do hypothesis test as defined here. The power to reject H0 in this may be smaller than the power to reject H0 in (1) since the variability of a variance estimate may be much larger than the variability of a mean estimate. In order to reject H0 in this, we need to increase sample sizes used for hypothesis test (1). Without increasing sample sizes, the hypothesis testing for (4) would result in concluding that the precision of the data from the receiving laboratory is better than that from the sending laboratory due to insufficient power to reject the null hypothesis.
We may conduct the equivalence test for comparing the precision of the data from two laboratories, but then we must face the challenge of selecting an appropriate margin and finding the correct test statistics.
Considering all challenges discussed above, we recommend comparing the sample estimate of the variability of the data from the receiving laboratory ( ) with the sample estimate of the variability of the data from the sending laboratory. We conclude the precision obtained from the receiving laboratory is at least as good as that obtained from the sending laboratory if sigma_r_hat is smaller than sigma_s_hat.

24. Receiving lab’s bias verification
Important to check bias since
the equivalence margin can be large enough such that 90% confidence interval in mean differences falls within the equivalence margin
but the receiving lab’s mean fails the bias criteria.
Method performance consistency should be checked with at least one more batch. We must point out that we should check the accuracy using a known reference standard to determine if the mean of data from the receiving laboratory meets the accuracy criteria.

25. References
1. U.S. Food and Drug Administration. Draft Guidance on Analytical Procedures and Methods Validation for Drugs and Biologics (2014). 2. International Society for Pharmaceutical Engineering (ISPE). The Good Practice Guide: Technology Transfer. ISBN-13: (2003). 3. United States Pharmacopeia. Transfer of Analytical Procedure. 37-National Formulary Ermer J, Limberger M, Lis K, Wätzig H. The transfer of analytical procedures. Journal of Pharmaceutical and Biomedical Analysis. 85: 262–276(2013). 5. Briggs .J, Nicholson R, Vazvaei F, et al. Method Transfer, Partial Validation, and Cross Validation: Recommendations for Best Practices and Harmonization from the Global Bioanalysis Consortium Harmonization Team. The AAPS Journal (2014). 6. Wieling J. Robust, fit-for-purpose method transfer: why we should apply equivalence testing. Bioanalysis. 7(7). 807–814 (2015). 7. Lin Z, Li W, Weng N. Capsule review on bioanalytical method transfer: opportunities and challenges for chromatographic methods. Bioanalysis. 3(1). 57–66 (2011). 8. Chambers D, Kelly G, Limentani G, Lister A, Lung KR, Warner E. Analytical Method Equivalency--An Acceptable Analytical Practice. Pharmaceutical Technology (2005). 9. Dewé W, Govaerts B, Boulanger B, Rozet E, Chiap P, Hubert P. Using Total Error as Decision Criterion in Analytical Method Transfer. Chemometrics and Intelligent Laboratory Systems –268 (2007). 10. Kaminski L, Schepers U, Wätzig H. Analytical method transfer using equivalence tests with reasonable acceptance criteria and appropriate effort: Extension of the ISPE concept. Journal of Pharmaceutical and Biomedical Analysis –1129 (2010). 11. Frömke C, Hothorn LA, Sczesny F, Onken J, Schneider M. Analytical method transfer: Improving interpretability with ratio-based statistical approaches. Journal of Pharmaceutical and Biomedical Analysis – 193 (2013). 12. Zhong JL, Lee K, Tsong Y. Statistical Assessment of Analytical Method Transfer. Journal of Biopharmaceutical Statistics. 18(5) (2008). 13. Schwenke JR, O'Connor DK. Design and Analysis of Analytical Method Transfer Studies. Journal of Biopharmaceutical Statistics. 18 (5) (2008). 14. Altan S, Shoung JM. Block designs in method transfer experiments. Journal of Biopharmaceutical Statistics.18(5) (2008). 15. Agut C, Caron A, Giordano C, Hoffman D, Ségalini A. Transfer of analytical procedures: A panel of strategies selected for risk management, with emphasis on an integrated equivalence-based comparative testing approach. Journal of Pharmaceutical and Biomedical Analysis – 303 (2011). 16. Krause S. Validation of Analytical Methods for Biopharmaceuticals: A Guide to Risk-Based Validation and Implementation Strategies. PDA/DHI, River Grove, IL, ISBN-10: (2007). 17. Satterthwaite FE. An Approximate Distribution of Estimates of Variance Components. Biometrics Bulletin –114 (1946). 18. Chen Y, Weng Y, Dong X, Tsong Y. Wald Tests for Variance-Adjusted Equivalence Assessment with Normal Endpoints. Journal of Biopharmaceutical Statistics (2016) Okamoto M. Assay validation and technology transfer: Problems and solutions. Journal of Pharmaceutical and Biomedical Analysis – 312 (2014).

26. Acknowledgement Dr. Yi Tsong, CDER/OB Dr. Juhong Liu, CDER/OBP
Dr. Chikako Torigoe, CDER/OBP