1. Mitigating Risk of Out-of-Specification Results During Stability Testing of Biopharmaceutical Products Jeff Gardner Principal Consultant 36 th Annual Midwest Biopharmaceutical Statistics Workshop May 21, 2013

2. What Is an OOS Result? Business Consequences of OOS Results - Regulatory requirements - Marketing/business impact Risks of OOS Associated with Statistical Approaches to Estimating Shelf Life - ICH Q1E Guidance - Mixed model (random slopes) estimation Recommendations for Managing OOS Risk Outline

3. From 21 CFR 314.81(b)(1)(ii): “Any failure of a distributed batch to meet [i.e. conform to] any of the specifications established in an application.” From ICH Q6A/Q6B: “’Conformance to specifications’ means that the drug substance and/or drug product, when tested according to the listed analytical procedures, will meet the listed acceptance criteria.” Expectation is that any test result, not just the average of results or majority of results, meets the specified acceptance criterion. What Is an OOS Result?

4. Regulatory Requirements Field alert required within 72 hours of identification Commercial Impact Potential product recall Possible reduction of shelf-life Business Impact of OOS Results

5. Illustration As seen here, a portion of the population of results extends into the OOS region, meaning that some risk exists of seeing OOS results.

6. “An appropriate approach to… shelf life estimation is to analyze a quantitative attribute… by determining the earliest time at which the 95 percent confidence limit for the mean intersects the proposed acceptance criterion.” The above applies in either of two cases (contingent on assumption of common slope using “poolability” test): Linear regression analysis is performed on data from a single batch In this case, mean = batch mean Linear regression analysis is performed on data from multiple batches In this case, mean = mean of all batches in analysis This mean is then taken to be an estimate of the product mean ICH Q1E Guidance Numerous critiques exist in literature which address “flaws” or limitations of the statistical approach outlined in ICH Q1E.

7. Methodology treats batch as a fixed effect, thereby limiting inference to only the sample of batches being evaluated. Calculation of a (1- α)% prediction interval about the mean is only useful in defining the region of likely results from the batch(es) being evaluated. This is despite the stated aim of the guidance which is to provide a “high degree of confidence” that all future batches will remain within acceptance criteria. ICH Q1E Limitations A statistical methodology is needed that more closely aligns with stated objectives of the ICH guidance documents.

8. Mixed Effects Model Inclusion of random effects a i and b i in the model allows for calculation of prediction limits which more closely align with stated objective of ICH guidance.

9. Prediction Intervals Using Mixed Model The first equation above is “philosophically” closest to ICH Q1E in that it governs the distribution of individual batch means at each stability testing interval. Reflects test result (i.e. analytical method) variability

10. Prediction Intervals Using Mixed Model For sake of discussion, we will use the first equation as the basis for determining shelf life since it aligns closest with ICH Q1E.

11. Determination of Shelf Life Shelf life and P j (OOS) were determined as described above for n = 2 5 = 32 experimental settings in order to examine the relationship between the two.

12. Experimental Settings

13. Example 1: P(OOS) at 24-Month Expiry AB Month ( x j ) Mean (A + B x j ) P(OOS) 00.33-0.0150.0080.1500.00-0.5430.0029 00.33-0.0150.0080.153-0.05-0.5890.0043 00.33-0.0150.0080.156-0.09-0.6390.0064 00.33-0.0150.0080.159-0.14-0.6910.0096 00.33-0.0150.0080.1512-0.18-0.7450.0144 00.33-0.0150.0080.1518-0.27-0.8620.0306 00.33-0.0150.0080.1524-0.36-0.9880.0594 Since parameters are pre-defined (known), z statistics were used in lieu of t for these calculations.

14. Probability of OOS at Expiry

15. Determining Cumulative P(OOS) Determining cumulative probability of OOS provides a direct measure of business risk of field alert/product recall. Note: this probability significantly increases if testing is performed more frequently later in shelf life Therefore the cumulative probability is most useful when calculated only for those intervals that are specified when establishing the stability testing protocol.

16. Example 2: Cumulative P(OOS) AB Month ( x j ) Mean (A + B x j ) P(OOS) Cum. P(OOS) 00.33-0.0150.0080.1500.000.0029 00.33-0.0150.0080.153-0.050.00430.0072 00.33-0.0150.0080.156-0.090.00640.0135 00.33-0.0150.0080.159-0.140.00960.0230 00.33-0.0150.0080.1512-0.180.01440.0371 00.33-0.0150.0080.1518-0.270.03060.0666 00.33-0.0150.0080.1524-0.360.05940.1220 Note: same model parameters used here as in Example 1.

17. Cumulative Probability of OOS This chart illustrates that determining shelf life based on mean alone (x-axis) does not address the risk of OOS results.

18. Application

19. Consider the following: Example 3: 12 Month Assay Data

20. Example 3 cont’d: Q1E Approach Per ICH Q1E, p-value for the month*lot effect (<0.25) requires determination of individual confidence limit for each lot. Shelf life is based on the earliest time at which any lot’s confidence limit intersects the lower acceptance limit (shown in previous graph). In this case, the data supports a maximum shelf life of 15 months which per ICH Q1E would have to be verified with real-time data.

21. Example 3 cont’d: Mixed Model Approach A B

22. Example 3 cont’d Based on a nominal α =0.05, the cumulative P(OOS) suggests a shelf life of 12 months.

23. Summary

24. Estimation of shelf-life is based on the earliest point at which a batch’s true mean is likely to intersect a specification limit. Substantial risk exists for observing individual OOS analytical results during the expiry period OOS risk is greatest toward the end of product shelf-life Summary of ICH Guidance

25. A prediction interval based on a mixed effects model can be constructed with either of two aims: philosophical alignment with ICH guidance (i.e. including only variance components for batch intercept & slope variability) minimizing risk of OOS results (by including residual variance component as well as batch variance components) Either of the above methods are an improvement upon the statistical approaches outlined in ICH guidance. Estimating Expiry Using Mixed Model

26. It is recommended that both prediction interval calculations are employed for preliminary shelf life estimation. Difference in shelf life estimates = impact of analytical method variability on shelf-life Additional analysis may be considered for evaluating how this impact can be lessened via increased replication Such understanding may provide guidance toward optimal method validation Impact of Analytical Method Variability

27. It’s recommended that a cumulative probability of OOS occurrence accompanies each preliminary shelf life estimate 1.calculate P(OOS) at each scheduled stability testing interval 2.calculate cumulative P(OOS) across all stability testing intervals It may be worth considering whether the final proposed shelf life should be based on first determining a maximum cumulative P(OOS) rather than by focusing on lower bound on the population of batch slopes. Formally Assessing Risk of OOS

28. Using experimental settings defined earlier, examine the sampling distributions of model parameter estimates via Monte Carlo simulations: How does sampling error contribute to over-estimation or under-estimation of P(OOS)? What impact does analytical method variability have on shelf life estimates if P(OOS) is used instead of the distribution of batch means at a given testing interval? Future Research

29. End of Presentation