2. 1 Searching for the Holy Grail of a Single PSD Profile Comparator Douglas S. Lee, Ph.D Nonclinical Statistics & Biostatistical Applications Pfizer Global Research & Development

3. DSLee 860-441-07452 Introduction The FDA is interested in the problem of determining the equivalence of aerodynamic particle size distributions (PSDs) between a generic and innovator product as well as part of a suite of tests to demonstrate chemical and manufacturing control. PSD Profile Comparison Working Group ACPS Product Quality Research Institute Future FDA guidance/recommendations for statistical analysis FDA (2003)Current Draft Guidance: Bioavailability and Bioequivalence Studies for Nasal Aerosols and Nasal Sprays for Local Action. Proposed statistic removed while the PQRI process identifies the best science Bioequivalence Studies for Nasal Aerosols and Nasal Sprays FDA (1999) Draft Guidance: Bioavailability and for Local Action.

4. DSLee 860-441-07453 Introduction Aerodynamic Particle Size Distribution – though involving measurement by impaction or impingement - is defined by the agency on a categorical basis and includes sites of deposition where the particles are not physically sized (e.g. valve/actuator, throat, pre-separator, and case).

5. DSLee 860-441-07454 Introduction To infer that the mean test profile is suitably equivalent to the mean reference profile, the null hypothesis that the mean profiles are different has to be tested and rejected...

6. DSLee 860-441-07455 Introduction... taking stage to stage variation – as well as correlation in variances between stages - into account. “Spaghetti” Plot of Individual PSDs

7. DSLee 860-441-07456 Introduction There is a strong desire for a “one-size-fits-all” test that results in a single test value that can be compared against a single critical value. The “one-size-fits-all” aspect of the desired test is the Holy Grail...

8. DSLee 860-441-07457 Outline This presentation... Outlines the algorithm proposed in the 1999 draft guidance. Describes the results of an investigation on the performance of the proposed procedure in the equal formulation or equal aerosol performance case (e.g. where the sets of reference and test PSD profiles are equal). Highlights technical issues with the algorithm as originally proposed.

9. DSLee 860-441-07458 Disclaimer The observations include work executed in support of the efforts of the PQRI working group but may not necessarily reflect the final consensus opinion of the group. The PQRI process is currently on- going. !!!??*# ##@!! &%!!@ @%!!

10. DSLee 860-441-07459 FDA Algorithm Obtain three lots each of a candidate (test) and comparator (reference) product. Select 10 inhalers from each lot at random and collect a PSD profile from each. Pool the 30 test product profiles into one group and pool the 30 reference product profiles into another group.

11. DSLee 860-441-074510 FDA Algorithm Select one profile at random from the test group and two profiles from the reference group (such that R  R’) to create a TRR’ triplet. Calculate the difference between the test profile and the average of two reference profiles as a  2 statistic: ReferenceTest

12. DSLee 860-441-074511 FDA Algorithm Also, calculate the difference between the two reference profiles as a  2 statistic: Calculate the ratio (rd) of the two  2 statistics as:

13. DSLee 860-441-074512 FDA Algorithm Repeat the process of selecting TRR’ triplets from the Test and Reference pools, 500 times with replacement and then calculate the mean ratio (Rd avg. ) from the 500 individual (rd) ratios.

14. DSLee 860-441-074513 FDA Algorithm Repeat the process of selecting 500 triplets and calculating Rd avg. 300 times (again with replacement) to obtain the 95 th percentile of the distribution of Rd avg.. Compare this last test statistic against a critical value to test the null hypothesis of non-equivalence.

15. DSLee 860-441-074514 FDA Algorithm The intent of the algorithm is to obtain an estimate of the variation in the profile difference of the test and reference groups scaled by the variation in the reference group. Large values of the ratio imply that the distance between the mean profiles of the test and reference groups is large and/or the profile to profile variation of the test group is greater than that of the reference group.

16. DSLee 860-441-074515 PQRI Activities Initially, the agency was interested in validating a critical value to which the test statistic could be compared. After discussion, the working group decided to start at the beginning and examine the performance of the algorithm in the equal formulations case - where there is no difference in the mean profiles of the test and reference groups - with respect to: general profile shape, pattern of stage to stage variation, and number of stages The objective was to begin characterization of the statistical properties of the algorithm in order to determine the feasibility of defining critical values based on test parameters – rather than upon aspects of the samples themselves.

17. DSLee 860-441-074516 Experimental Design approach to examining the statistical performance of the FDA algorithm for the equal formulation case We defined three key parameters for defining sets of simulated rank-ordered PSD profiles in terms of shape and pattern of variation: Expected (target) percent on rank-ordered stage defined by beta distribution parameters. First rank-ordered stage standard deviation. CV slope for rank-ordered profiles. In addition, we have a fourth variable – number of stages. How did we come up with these metrics?

18. DSLee 860-441-074517 Rank-ordered PSD profiles actually represent families of PSD profiles... The 95 th percentile of Rd avg. by the FDA algorithm for the set of reference and test profiles with the mean PSD profile below is 3.93.

19. DSLee 860-441-074518 Rank-ordered PSD profiles actually represent families of PSD profiles... This set of mean reference and test profiles yields a value of 3.93 for the 95 th percentile of Rd avg. (using same randomization seed as the last example).

20. DSLee 860-441-074519 Rank-ordered PSD profiles actually represent families of PSD profiles... This set of reference and test mean profiles also yields a value of 3.93 for the 95 th percentile of Rd avg. (using same randomization seed as the last two examples).

21. DSLee 860-441-074520 Rank-ordered PSD profiles actually represent families of PSD profiles... The same 95 th percentile of Rd avg. is obtained from the three example sets because they share the exact same rank ordered profile (as well as rank order specific stage variances). The 95 th percentile of the  2 ratio statistic equals 3.93 for each example profile as well as the rank ordered profile. (same simulation seed for each determination) ABC

22. DSLee 860-441-074521 Rank-ordered PSD profiles actually represent families of PSD profiles... This simplifies the definition of profile “shape” when simulating data sets of PSD profiles. We can cover the “PSD profile “water front” using easily defined distribution parameters. Percent on Stage Rank-Ordered Stage (a, b) = (1, 1) (a, b) = (1, 2) (a, b) = (1, 4)

23. DSLee 860-441-074522 Underlying patterns of profile variation... We now want to also systematically define and deal with stage to stage differences in standard deviations for rank-ordered profiles.

24. DSLee 860-441-074523 Underlying patterns of profile variation... Examination of the IPAC-RS PSD database revealed that generally, the coefficient of variation (e.g. relative standard deviation) is either relatively constant across the stages or increases more or less linearly with decreasing rank-order of the stages (e.g. declining mass recovered on stage). As a consequence, the stage specific standard deviation is generally proportional to the percent retained on stage and decreases with declining percent on stage at a rate equal to or less than the rate at which the mean percent on stage decreases.

25. DSLee 860-441-074524 Underlying patterns of profile variation... Corresponding ranges for first rank ordered stage standard deviation and CV slope used to calibrate simulation data sets:

26. DSLee 860-441-074525 Experimental Design approach to examining the statistical performance of the FDA algorithm for the equal formulation case Design repeated across four levels of stage: 4, 7, 10, 13 Response surface design – 500 sets of Reference and Test profiles per design point.

27. DSLee 860-441-074526 Experimental Design approach to examining the statistical performance of the FDA algorithm for the equal formulation case Patterns of rank-ordered profiles/variation at extremes of the design space.

28. DSLee 860-441-074527 Experimental Design approach to examining the statistical performance of the FDA algorithm for the equal formulation case For each 500 pairs of test and reference profile sets, the distribution of all possible (13,050) values of rd were calculated and the 95 th percentiles rd’s were calculated. The median 95 th percentile rd was then tabulated for each design point. The point estimate of the 95 th percentile of the distribution of rd was contrasted with the critical value from an F-distribution at  = 0.05 with the appropriate dfs. The ratio of two independent central chi-squares will be distributed as a central F with (S-1, S-1) df’s.

29. DSLee 860-441-074528 Experimental Design approach to examining the statistical performance of the FDA algorithm for the equal formulation case If the value of the difference (median rd 95 – F crit ) is less than zero, then the distribution of rd from which Rd avg is estimated in the FDA algorithm has a shorter tail than expected if rd is distributed as an F. If the difference is greater than zero, then the distribution of rd has a longer tail than otherwise expected.

30. DSLee 860-441-074529 Experimental Design approach to examining the statistical performance of the FDA algorithm for the equal formulation case The contrast value was modeled over the design space. To demonstrate stability of rd in the equal formulation case, the contrast value should not be sensitive to location in the design space.

31. DSLee 860-441-074530 Results

32. DSLee 860-441-074531 Results

33. DSLee 860-441-074532 Results

34. DSLee 860-441-074533 Results

35. DSLee 860-441-074534 Results

36. DSLee 860-441-074535 Discussion These are not particularly surprising results... The assumption of independence for d TRR’ and d RR’ is violated because the same reference profiles appear in both the numerator and denominator of each individual rd. Consequently, the distribution of rd can only be only F-like. Setting the question of lack of independence aside, d TRR’ and d RR’ are only asymptotically distributed as  2 ’s. The distribution of rd will only approximate an F-like distribution when the ratio for the smallest observed average percent on stage, divided by the number of stages in the profile, exceeds a value of one and/or the number of stages increases. Consequently, the distribution of rd is particularly sensitive to rank- ordered profile shape and number of stages.

37. DSLee 860-441-074536 Discussion These simulation results point out several difficulties with respect to the desire for a “one-size-fits-all” test... It suggests that equivalence test values will need to be defined on a product by product basis. The “recipe” for defining equivalence test values without resorting to inclusion of properties of the samples themselves, may not be tractable. Depending upon the rank-ordered profile shape and number of stages, pooling of stages might be required.

38. DSLee 860-441-074537 Discussion Other technical considerations... The mean Rd is not always defined... and is not necessarily the 95 th percent upper confidence bound for Rd avg.

39. DSLee 860-441-074538 Discussion Other technical considerations... The FDA algorithm doesn’t treat the creation of triplets in a balanced fashion – it admits triplets for which both reference profiles may come from the same lot. This is easily remedied by the sampling approach of Cheng and Shao where lot-triplets are first defined. However, this does not address the fundamental distributional issues.

40. DSLee 860-441-074539 Discussion The working group is continuing work on the problem. To get over the hump, it will be critical to clearly define what is meant by equivalence of profiles.

41. DSLee 860-441-074540 Acknowledgements Pfizer: Mark Berry Jeff Blumenstein Yukun Ren Greg Steeno PQRI PSD Profile Working Group: David Christopher (chair) Svetlana Lyapustina Wallace Adams, Craig Bertha, Peter Byron, Bill Doub, Craig Dunbar, Walter Hauck, Jolyon Mitchell, Beth Morgan, Steve Nichols, Yinou Pang, Guirag Poochikian, Gur Singh, Terry Tougas, Yi Tsong, Ron Wolff, Bruce Wyka, DPTC Liaisons: Jeff Blumenstein Michael Golden

42. DSLee 860-441-074541 Back up slides...

43. DSLee 860-441-074542 Underlying patterns of profile variation... Blinded Profiles 3 – 14 (from top left to bottom right by line)

44. DSLee 860-441-074543 Underlying patterns of profile variation... Blinded Profiles 15 – 26 (from top left to bottom right by line). Profile pattern 16 CV for last stage is off-scale (~480) and the overall pattern of CV change with descending rank-order of stages is better characterized as exponential.

45. DSLee 860-441-074544 Spaghetti plots of IPAC-RS profiles Blinded Profiles 1 – 4 (from top left to bottom right by line)

46. DSLee 860-441-074545 Spaghetti plots of IPAC-RS profiles Blinded Profiles 5 – 8 (from top left to bottom right by line)

47. DSLee 860-441-074546 Spaghetti plots of IPAC-RS profiles Blinded Profiles 9 – 12 (from top left to bottom right by line)

48. DSLee 860-441-074547 Spaghetti plots of IPAC-RS profiles Blinded Profiles 13 – 16 (from top left to bottom right by line)

49. DSLee 860-441-074548 Spaghetti plots of IPAC-RS profiles Blinded Profiles 17 – 20 (from top left to bottom right by line)

50. DSLee 860-441-074549 Spaghetti plots of IPAC-RS profiles Blinded Profiles 21 – 24 (from top left to bottom right by line)

51. DSLee 860-441-074550 Spaghetti plots of IPAC-RS profiles Blinded Profiles 25 – 27 (from top left to bottom right by line)