1. Useful Block Designs for Gage R&R Studies for Measuring Total Analytical Variability Jyh-Ming Shoung*, Areti Manola, Yan Shen, Stan Altan *jshoung@its.jnj.com Midwest Biopharmaceutical Statistics Workshop May 19, 2015

2. Outline Measurement System Analysis – Gage R&R – Design and Outputs – ICH Q2(R1) Case Study I – Dissolution Case Study II – Content Uniformity Case Study III – In Vitro Release Assay Summary

3. Measurement System Analysis – Gage R&R Gage R&R design is a well known statistical design embraced by Six Sigma practitioners – Describes quantitatively how a process is performing. – To achieve the designation of a Six Sigma process, it must not produce more than 3.4 defects per million opportunities. The typical Gage R&R (Repeatability and Reproducibility) study consists of a factorial type design with blocking used to measure the total variability in a measurement system and its important components. Other designs can be used also, examples shown in the case studies. Gage R&R studies have been used for investigating chemical analytical methods for the active pharmaceutical ingredient, uniformity across dosage units and dissolution.

4. ICH Q2(R1) Repeatability: Repeatability expresses the precision under the same operating conditions over a short interval of time. Repeatability is also termed intra-assay precision Intermediate Precision (IP): Intermediate precision expresses within-laboratories variations: different days, different analysts, different equipment, etc. Reproducibility: Reproducibility expresses the precision between laboratories (collaborative studies, usually applied to standardization of methodology).

5. Measurement System Analysis – Gage R&R Total Variability = Process Variability + Gage Variability (Measurement Error) Two Components of Gage Variability:  Repeatability – Short Term Variability Inherent precision of the device itself  Reproducibility – Long Term Variability Variability due to different laboratories, operators, time periods, environments or in general, different conditions  Intermediate Precision (IP): Reproducibility without laboratories

6. CASE STUDY I - DISSOLUTION

7. DoE for Dissolution Test DayDissRunAnalystAppV 1V 2V 3V 4V 5V 6 HPLC Run 1 821 BCA---1 ---BAC2 1012 ABC---2 ---ACB1 2 112 ABC---4 ---ACB3 921 CAB---3 ---CBA4 3 1211 BCA---5 ---BAC6 322 CAB---6 ---CBA5 4 622 BCA---8 ---BAC7 411 ABC---7 ---ACB8 5 211 BCA---9 ---BAC10 522 CAB--- ---CBA9 6 721 ABC---12 ---ACB11 12 CAB--- ---CBA12 Design Parameters: 1.3 Batches: A, B, C 2.4 sites/labs: G, I, L, T 3.2 apparatus and 2 analysts per site Experiment to be run in 6 days On each day, each analyst will carry out 1 dissolution run, total 12 dissolution run per lab/site Samples are assigned to 2 HPLC runs according to the table, total 12 HPLC runs per lab/site Responses: Dissolution measurements at 5, 10, 15, 20, 30, 45, 60 and 90 minutes are collected with analysis focusing on Q time of 30 minutes

8. Data Plot for 30 Minutes Dissolution

9. Mixed Effects Model for Dissolution at 30 min  Fixed effects: Batch, Site  Random effects : Day(Site), DissolutionRun(Day), HPLCRun*DissolutionRun(Site*Day) y n(ijklm) = dissolution of the n th vessel for j th batch tested in i th Site for l th dissolution run at k th day with m th HPLC run,  i = i th Site effect,  j = j th batch effect, D k(i) = random effect of k th day within i th Site:  N(0,  D 2 ), E l(k) = random effect of l th dissolution run in k th day:  N(0,  Ei 2 ), H lm(ik) = random effect of m th HPLC run and l th dissolution run within the kth day and i th Site  N(0,  H 2 ),  n(ijklm) = residual error (vessel-to-vessel):  N(0,  ei 2 ). IP =  D 2 +  E 2 +  H 2 +  e 2 and Repeatability =  e 2 Uncertainty (se) term associated with one reportable value = mean of six vessels = SQRT(  D 2 +  E 2 +  H 2 +  e 2 /6)

10. Table 1: Variance Components Estimates Variance Components SiteEstimateLower95%Upper95% Day(Site)0.260.045.04E+05 DissRun*HPLC (Site*Day) 0.960.414.25 DissRun(Day) Site G6.352.7228.06 Site I0.830.2080.83 Site L10.304.6239.63 Site T0-- Residual Site G5.774.058.90 Site I3.082.134.87 Site L8.606.1212.98 Site T6.724.8010.10

11. Table 2: Repeatability and Intermediate Precision (IP) SiteSourceEstimate% of TotalLower95%Upper95% 2*se for a Reportable Value G Repeatability5.77434.058.90 5.84 IP (Total)13.351007.225.04E+05 I Repeatability3.08602.134.87 3.21 IP (Total)5.141002.785.04E+05 L Repeatability8.60436.1212.98 7.20 IP (Total)20.1310011.195.04E+05 T Repeatability6.72854.8010.10 3.06 IP (Total)7.951005.255.04E+05

12. Site as random effect and homogeneous across sites Table 3: Variance Components Estimates Table 4: Repeatability and Reproducibility Variance ComponentsEstimateLower95%Upper95% Site8.732.70147.71 Day(Site)1.230.449.89 DissRun*HPLC (Site*Day) 2.651.545.60 DissRun(Day)1.570.619.56 Residual6.215.137.67 SourceEstimate% of TotalLower95%Upper95% 2*se for a Reportable Value Repeatability6.21305.307.67 7.80 Reproducibility (Total) 20.3910010.42180.44 Uncertainty (se) term associated with one reportable value = mean of six vessels = SQRT[ (Site+Day+Disso_run+HPLC run) + Repeatability/6 ]

13. CASE STUDY II – PARTICLE SIZE

14. Design for Laser Diffraction Test Method for Particle Size 14 Site/LabAnalystInstrument ID AJLMS-2 BDHMM2 CKC EOP049 EOP224 DLL PSE002 LD005 M300 Four laboratories, 4 analysts and 7 instruments Six Batches Design of Experiments performed by each combination of analyst and instrument within each lab BatchBatch Info L1Target batch L2Larger particle size L3Smaller particle size L4Registration batch at target L5100L Target batch L6100L Smaller particle size BatchDay 1Day 2Day 3 L1Syringe 1-2-3-4Syringe 5-6/ L2Syringe 1-2/Syringe 3-4-5-6 L3/Syringe 1-2-3-4Syringe 5-6 L4Syringe 1-2-3-4Syringe 5-6/ L5Syringe 1-2/Syringe 3-4-5-6 L6/Syringe 1-2-3-4Syringe 5-6 Response: particle size d50 (50% of particle size distribution, m)

15. Data Plot for d50

16. Mixed Effects Model for Particle Size d50 Site_App - combination of Site-Analyst and Instrument (7 combinations), Fixed effects : Site_App and Batch Random effects : Day(Site_App), Day*Batch(Site_App) and residuals. y l(ijk) = d50 of the l th syringe of j th batch tested by i th Site_App at k th day  i = i th Site_App effect  j = j th batch effect D k(i) = random effect of k th day within i th Site_App:  N(0,  D 2 ) E kj(i) = random interaction effect of j th batch by k th day within i th Site_App:  N(0,  E 2 )  l(ijk) = residual errors:  N(0,  e 2 ). IP =  D 2 +  E 2 +  e 2 and Repeatability =  e 2 Uncertainty (se) term associated with one reportable value = SQRT(  D 2 +  E 2 +  e 2 )

17. Table 5: Variance Components Estimates Variance Components EstimateLower95%Upper95% Day(SITE_APP)0-- BATCH*Day (SITE_APP) 0.000470.000063.61E+109 Residual0.060.050.08 Table 6: Repeatability and Intermediate Precision (IP) SourceEstimate% of TotalLower95%Upper95% 2*se for a Reportable Value Repeatability0.0627990.051410.07818 0.5027 IP (Total)0.06321000.051473.61E+109

18. Site_App as random effect, Site_Apps homogeneous Table 7: Variance Components Estimates Table 8: Repeatability and Reproducibility Uncertainty (se) term associated with one reportable value = SQRT((Reproducibility) Variance ComponentsEstimateLower95%Upper95% SITE_APP0.001850.000450.19354 Day(SITE_APP)0-- BATCH*Day(SITE_APP)0.000379.60E-058.85E+172 Residual0.062780.051490.07827 SourceEstimate% of TotalLower95%Upper95% 2*se for a Reportable Value Repeatability0.063970.050.08 0.51 Reproducibility (Total)0.0651000.058.85E+172

19. CASE STUDY III - BIOASSAY

20. DoE for Bioassay Design Parameters: 1.2 Analyst: 1, 2 2.5 assay/run per Analyst 3.2 Plates within each assay 4.6 concentration levels (%)  25, 50, 67, 100, 150 and 200  3 different concentrations within each plate Response: Potency AnalystRunPlate Concentration (%) 255067100150200 1 1 1XXX 2XX X 2 3X X X 4X X X 3 5X XX 6 XX X 4 7 X XX 8 X XX 5 9 XX X 10 XXX 2 1 1 XXX 2 X XX 2 3 X X X 4 XX X 3 5 XXX 6X XX 4 7X X X 8X XX 5 9XX X XX X

21. Data Plot for Bioassay

22. Mixed Effects Model for Potency Fixed effects : Analyst, Concentration Random effects : Run(Analyst), Plate(Run*Analyst) and residual (within-plate) y m (ijkl) = log of m th potency for j th concentration tested by i th Analyst in l th Plate at k th Run  i = i th Analyst effect  j = j th Concentration effect R k(i) = random effect of k th Run by i th Analyst:  N(0,  R 2 ) P l(ik) = random effect of l th plate within k th Run by i th Analyst:  N(0,  P 2 )  m(ijkl) = residual errors:  N(0,  e 2 ). IP =  R 2 +  P 2 +  e 2 and Repeatability =  e 2 Reportable Value defined as mean of 3 runs, 2 plates/run /analyst Uncertainty = SQRT(  R 2 /3 +  P 2 /6 +  e 2 /6) (Format IP)

23. Variance Components, Repeatability, Intermediate Precision Table 9: Variance Components Estimates (in log scale) Table 10: Repeatability and Intermediate Precision (in log scale) Variance ComponentsEstimateLower95%Upper95% Run(Analyst)0.00150.00031.4679 Plate(Analyst*Run)0.00390.00170.0159 Residual0.00190.00120.0032 SourceEstimate% of TotalLower95%Upper95% Repeatability0.0019260.00120.0032 IP (Total)0.00731000.00321.4871

24. Variance Components, Repeatability, Intermediate Precision Table 11: Repeatability and Intermediate Precision (in linear scale) Source % Geometric Variance % of TotalLower95%Upper95% 2*%GCV Reportable Value Repeatability19.92613.034.0 7.8 IP (Total)76.410034.015627.6

25. Remarks 1.Application of blocking and Statistical DoE is an efficient way to study analytical method performance for the purpose of assessing reproducibility, intermediate precision and repeatability of chemical analytical methods. 2.DoEs permit a logical decomposition of the total variability into meaningful component parts. 3.Results are straight forward to interpret and have scientific meaning. 4.Inherent precision of the device itself cannot always be estimated because of the destructive nature of the test – confounded with dosage unit variation. 5.Some sources of variability may be estimated with insufficient precision. – Not possible at the development stage to estimate process variability due to small number of lots (make it a fixed effect). 6.Various statistical designs can form the basis for a Gage R&R study to gather information about additional factors. 7.Improves understanding and ability to support product specifications. 25