By Trusha Patel and Sirisha Davuluri

“An efficient method for accommodating potentially underpowered primary endpoints” ◦ By Jianjun (David) Li and Devan V. Mehrotra” 2

1. Introduction 2. Procedure 1: PAAS Method 3. Procedure 2: 4A Method a.Independent Primary Endpoints b.Correlated Primary Endpoints 4. Performance of both Procedures (Probability of achieving positive trial) 5. Conclusion 6. Appendix 7. References 8. Questions? 3

Clinical trials generally classify the endpoints into primary, secondary and exploratory types. Primary Endpoints:  Primary endpoints address primary objectives of the trial. They are usually few but are clinically most relevant to the disease and the treatment under study. They assess the main clinical benefits of the treatment. 4

Secondary Endpoints:  Secondary endpoints characterize extra benefits of the treatment under study after it has been demonstrated that the primary endpoints show clinically meaningful benefits of the treatment. Explanatory Endpoints:  Exploratory endpoints are usually not prospectively planned and are generally not rigorously evaluated like primary and secondary endpoints. 5

Different approaches have been used to specify a clinical decision rule for trials that have more than one primary endpoint. Significant results can be required for each of several primary endpoints to consider a trial ‘‘positive.” 6

 To demonstrate the treatment’s superiority on at least one endpoint.  If each multiple endpoint is independently clinical relevant, the multiple endpoint problem can be formulated as a multiple testing problem, and the trial is declared positive if at least one significant effect is detected.  When significant results are required for more than one but not all of multiple primary endpoints for a trial to be considered positive, correction for multiplicity is also necessary, and this must take into account the total number of endpoints and the number required for the trial to be considered positive. 7

 Weighted Bonferroni procedure  Prospective Alpha Allocation Scheme (PAAS Method)  Adaptive Alpha Allocation Approach (4A Method)  Bonferroni-type parametric procedure  Fallback-type parametric procedure For this project, we are mainly focusing on PAAS Method and 4A Method by considering two Primary Endpoints (A : well powered, B : potentially underpowered). 8

Procedure:  Moye’(2000) proposed the ‘‘prospective alpha allocation scheme” for preserving Type I error rates at acceptable levels when there are multiple endpoints.  Assume that p-values for the individual endpoints are independent.  The experiment-wise type I error rate = α should be capped, say at 0.05, and the fraction of the Type I error rate α allocating α 1 * = α − ε, ε= 0.01 to endpoint A α 2 * = 1 - (1−α)/(1−α 1 *) to endpoint B 9

Advantages:  PAAS is a simple and appealing method.  Prospectively allocating alpha in this method preserves the experiment-wise Type I error rate and makes it possible to consider a treatment efficacious when the null hypothesis is not rejected for the primary endpoint but is rejected for one or more of the secondary endpoints. Disadvantage: The prospective alpha allocation scheme preserves the experiment-wise Type I error rate at a higher rate than is customarily accepted. 10

Procedure:  Li and Mehrotra (2008) proposed a multiple testing procedure, which they referred to as the adaptive alpha allocation approach or 4A procedure.  Consider a clinical trial with m endpoints and assume that the endpoints are grouped into two families. The first family includes m 1 endpoints that are adequately powered and the second family includes m 2 potentially underpowered endpoints (m 1 + m 2 = m). 11

Let’s consider 2 multiple points for illustration of the procedure:  Let p A and p B denote the p-value for endpoint A and endpoint B, respectively.  In the article, they assume that p-values are two-tailed (to mimic common practice) and that ‘statistically significant’ results are in the direction of interest; however all the methods discussed can also be used with one-tailed p-values.  In their proposed adaptive alpha allocation approach, endpoint A is tested at the pre specified level α 1 = α − ε, and endpoint B is tested at the adaptive level. 12

 If A achieves statistical significance (i.e. p A ≤α 1 ), then B is tested at level α 2 (p A,α 1, α) = α;  If A fails to achieve statistical significance (i.e. p A >α 1 ), then B is tested at level 0 <α 2 (p A,α 1,α) ≤ α 1, which is close (or equal) to α 1 if p A is not much greater than α 1, but approaches zero as p A increases.  Sample values of α 2 as a function of observed p A are provided in Table I (ρ = 0 column)  Table I shows the alpha allocation for Primary endpoint B. 13

14 Table I

 Let there be M primary end points out of which the first m are assumed to be powered by type A endpoints and the rest of the M-m are underpowered or type B endpoints.  Here, they suppose, the p-values for the type A endpoints are tested for statistical significance at an overall level α 1 = α – ε using Hochberg’s method.  The p-values for the type B primary endpoints are then measured at and overall adaptive level α 2 (p (m), α 1, α,m) using Hochberg’s method 15

Note : (3) and (4) are generalizations of (1) and (2); setting m=1 in (3) and (4) leads to (1) and (2), respectively. Advantage: The remaining endpoints are tested at a generally higher significance level, which improves their power. 16 Extending 4A to the scenario of three or more independent primary endpoints

Procedure:  If the endpoints are correlated, then formula (1) and (3) cannot be used directly because the FWER may be inflated. Therefore, an adjustment may be needed in the formulas presented for correlated endpoints.  The 4A procedure is readily implemented using Table I if ρ is known.  For the case of unknown ρ, See Appendix-I. Extending 4A to the relatively uncommon scenario of M>2 primary endpoints is notably more challenging in the case of correlated endpoints compared with independent endpoints. 17

Let’s compare the performances of the PAAS and 4A methods for our motivating scenario of two primary endpoints: A-well powered and B-underpowered. Goal: To improve the probability of correctly achieving a positive trial, i.e. of rejecting at least one of the two null hypotheses, while ensuring that the family wise type I error rate is at most α. Table A summarizes the probabilities of achieving a positive trial using the PAAS and 4A methods when the two endpoints are either independent (ρ=0) or correlated (ρ=0.5); results are based on 10,000,000 simulations. (See Table III in Appendix-II) 18

19 Table A

Case 1-7: Primary endpoint A: Adequately Powered Primary endpoint B: Underpowered Case 8-17: Primary endpoint A: Underpowered Primary endpoint B: Underpowered For PAAS method, α 1 * = 0.04 and α 2 * = 0.010; and For 4A method, α 1 = 0.04 and α 2 ≡α 2 (p A,α 1,α,ρ) is calculated adaptively as described before. 20

 Use of 4A method increased the probability of achieving a positive trial compared with PAAS. o For Independent endpoints, the absolute gain in power ranged from 0.9 % (case 7; 77.9 versus 78.8%) to 3.7% (case 8; 78.8 versus 82.5%) compared with PAAS. o For Correlated endpoints A and B (ρ=0.5); the power for 4A was numerically higher than that for PAAS, For example, the power gains of 4A ranged from 0.1 per cent (case 7; 77.4 versus 77.5 per cent) to 2.9 per cent (case 8, case 14) compared with PAAS. 21

 The probability of achieving a positive trial by pre specifying both A and B as primary endpoints and using 4A for the analysis is about as good as (cases 7 and 13) or notably better than (cases 1 and 8) the marginal power for endpoint A. o In other words, 4A enables us to accommodate the underpowered endpoint B in the primary family while essentially preserving or substantially enhancing the likelihood of achieving a positive trial compared with the strategy of using only A as the primary endpoint with α=

 There is no optimal strategy for the use of significance testing with multiple endpoints, especially when dealing with clinically persuasive endpoints that may be underpowered, but based on the simulation results that are provided in the article, we can conclude that 4A method performs better than PAAS method. o The power advantage of 4A over PAAS was greater when the marginal power for endpoint B was lower (higher), regardless of whether the endpoints were independent or correlated. 23

24

25

  Multiple Testing Problems in Pharmaceutical Statistics, by Alex Dmitrienko, Ajit Tamhane and Frank Bretz. 26

27