PQRI PSD Profile Comparison Working Group: Draft Simulation Results on Stability of the Chi-Square Ratio Statistic Under the Null Case of Identical Sets of Reference and Test Profiles.

Douglas S. Lee Discussion Outline Quick review of product profiles as spaghetti plots rather than scatter plots. Review of simulation design and 4, 7, 10, & 13 stage simulation results.

Douglas S. Lee Spaghetti plots of profiles Blinded Profiles 1 – 4 (from top left to bottom right by line)

Douglas S. Lee Experimental Design approach to examining  2 ratio stability for null reference-test data Simulated profiles are shown by selected design points to illustrate the how the combinations of simulation parameters relate to each other.

Douglas S. Lee 10 Stage Simulation Results: Median Q50 Final Equation in Terms of Centered and Scaled Factors: (Median Q50) -2 = * coded Rank-ordered profile shape (b parameter value) * coded SD first rank-ordered stage * coded CV slope * (coded CV slope) * coded Rank-ordered profile shape * coded SD first rank-ordered stage * coded Rank-ordered profile shape * coded CV slope R-Squared Adj R-Squared Pred R-Squared Adeq Precision back-transformed centered Median Q50: 0.96

Douglas S. Lee Putting it all together... Q95 50 th to 99 th Percentile Distance uniform: linear decline: “exponential” decline: 4 stages7 stages10 stages13 stages Z scale 0 to 2Z scale 0 to 20

Douglas S. Lee Summary Observations We executed a face-centered central composite design for the combination of rank-ordered profile (using the fitting coefficients for the beta distribution: b running from 1 to 2 to 4 with a = 1), first rank- ordered stage standard deviation (1 to 5.5 to 10), and CV ramp (0 to 7.5 to 15). The selected simulation parameters reasonably covers the patterns of profiles observed in the database. In general, the chi-square ratio statistic shows some sensitivity with respect to rank-ordered profile shape, differences in the rate at which stage specific standard deviations decrease (CV slope), and number of stages. With respect to the objective of examining the stability of the chi- square ratio statistic in the null case (identical sets of Reference and Test profiles), the stability of the statistic increases as the number of stages increases and the rank-ordered profile shape is more linear than “exponential”.