2. Webinar Overview – Host
HOSTED BY: Ronan Fitzpatrick
Head of Statistics
FDA Guest Speaker
nQuery Lead Researcher
Guest Lecturer

3. Webinar Overview – Content
Sample Size and Sensitivity Analysis
Introducing Assurance with nQuery Demo
Assurance Discussion and Recommendations
Content>
nQuery Assurance Plans

4. Part 1
Sample Size & Sensitivity Analysis

5. Sample Size Determination Review
Sample Size Determination (SSD) is the process for finding the appropriate sample size for your study.
Common metrics: statistical power, interval width or cost
SSD seeks to balance ethical and practical issues
This has made it Standard Trial Design Requirement
SSD is crucial to arrive at valid conclusions
Yet Many Studies Still have Insufficient Sample Size
Biggest SSD challenge is dealing with uncertainty

6. What Is Statistical Power?
The most commonly used metric for sample size determination is statistical power
The power is the probability that the study will be able to detect a true effect of a specified size.
In other words, Power is the probability of rejecting the null hypothesis when it is false
Traditional statistical power and assurance are intimately related concepts

7. 5 Essential Steps for Sample Size1
Formulate the study
Study question, primary outcome, statistical method 2
Specify analysis parameters
Standard deviation, ICC, dispersion 3
Specify the Effect Size for Test
Expected/targeted difference or ratio 4
Compute Sample Size or Power
N for specified power or power for fixed N 5
Perform Sensitivity analysis
Some parameters more uncertain and/or influential

8. Issues With Sensitivity Analysis
Sensitivity analysis provides an overview of the effect of varying the effect size and/or analysis parameters
Important to understand robustness of sample size estimate and dispel over-confidence in fixed estimates
But traditionally only involves assessing a small number of potential alternative scenarios
No rules for choosing scenarios and how to pick between them
How to improve upon or assist with this process?

9. Part 2
Assurance Introduction & Demo

10. Bayesian Sample Size Sample Size for Bayesian Methods
Sample size for specific values of Bayesian parameters
e.g. Bayes Factors, Credible Intervals, Utility/Cost function
Bayesian Approaches to Improve Sample Size
Integrating Bayesian methods into current methods to add greater context for parameter uncertainty
e.g. Assurance, Predictive Power, Adaptive
One Sample Credible Interval with Known Precision
One Sample Credible Interval with Unknown Precision
One Sample Mixed Bayesian Likelihood Criterion
Two Sample Credible Interval with Known Precision
Two Sample Credible Interval with Unknown Precision
Assurance for Superiority Trial Comparing Two Means
Assurance for Equivalence Trial Comparing Two Means
Assurance Non-inferiority Comparing Normal Means

11. Assurance for Clinical Trials
Assurance (sometimes called “Bayesian power”) is the unconditional probability of significance given a prior
Focus on methods proposed by O’Hagan et al. (2005)
Assurance is the expectation of the power averaged over a prior distribution for the effect and/or parameters
Often framed the “true probability of success” of a trial
Will focus on simple two-sample normal case in our demonstration

12. Assurance and Sensitivity Analysis
In a sensitivity analysis, a number of scenarios are chosen by the researcher and assessed individually for power or N
This gives a clear indication of the merits of the individual cases highlighted but no information on other scenarios
With assurance, we have the average power over all plausible values by assigning a prior to one/more parameters
This provides a summary statistic for the effect of parameter uncertainty but less information on specific scenarios.

13. Introducing nQuery
Over 20 Years of Experience in Sample Size Determination and Power Analysis for Clinical Trials
Latest Release had Methods for >200 Trial Designs
Used by 45/50 Top Pharma and Biotech Companies
nQuery’s over 20 Years of Success is Based on:
Being Easy to Use and Accessible to All Users
Being Fully Validated and an Industry Standard
Quickly Integrating Innovations and Customer Requests

14. Assurance Example 1
“The outcome variable … is reduction in CRP after four weeks relative to baseline, and the principal analysis will be a one-sided test of superiority at the 2.5% significance level. The (two) population variance … is assumed to be … equal to … the test is required to have 80% power to detect a treatment effect of 0.2, leading to a proposed trial size of n1 = n2 = 25 patients … For the calculation of assurance, we suppose that the elicitation of prior information … gives the mean of 0.2 and variance of If we assume a normal prior distribution, we can compute assurances with m = 0:2, v = 0.06 … With n = 25, we find assurance = 0.595
Source: Wiley.com
Parameter
Value
Significance Level (One-Sided)
0.025
Prior Mean Difference
0.2
Prior Difference Variance
0.06
Posterior Standard Deviation
√0.0625=0.25
Sample Size per Group 25

15. Assurance Example 2
“We estimated that we would need to enrol patients, given an expected mean (±SD) annual decline in the FVC of 9±16 percent of the predicted value and a dropout rate of 15 percent, to achieve a two-sided alpha level of 0.05 and a statistical power of 90 percent.”
Source: nejm.com (2006)
Parameter
Value
Significance Level (Two-Sided)
0.05
Mean Difference (%) -9
Standard Deviation (%) 16
N per Group (post drop-out)
0.85(163/2)= 68
Target Power
90%

16. Part 3
Assurance Discussion and Recommendations

17. Technical Issues With Assurance
How to decide on a prior for the assurance calculation?
Previous studies, educated guess, expert elicitation
Sheffield Elicitation Framework (SHELF) has industry interest
Able to use a non-normal prior?
Yes, but no analytical solution. Requires usage of simulation
More flexible methods available in nQuery in Spring 2018
Assurance for alternative data types or decision criteria?
Methods in literature for proportions and survival (Spring 2018)
Assurance concept technically generalizable to any criteria

18. Recommendations For Assurance
Assurance instead of power to find the original sample size?
Would usually recommend “no” due to the nature of chosen effect size
If the effect size is the “expected” size it might make sense but are we interested in non-clinically relevant powers?
Does assurance measure for the question of interest or just the p-value?
Does Assurance replace sensitivity analysis completely?
Has many advantages but sensitivity analysis more “tactile”?
Assurance is vital contextual tool in the planning toolbox
Places uncertainty at the heart of the sample size determination process

19. Part 4
nQuery Assurance Plans

20. nQuery Assurance Plans
Additional assurance methods for more complex probability distributions for prior (parametric, empirical)
Assurance methods for other data types (proportions, TTE etc.) and statistical hypotheses (equivalence etc.)
Assurance methods for joint probability distributions for multiple parameters and where multiple hypotheses exist
Greater access and plotting options to explore assurance results

21. nQuery Features & Capabilities
nQuery 100s of sample size tables to cover your trial design Installation & Operational Qualification Tools Expert support available from our Sample Size Experts
Bayes
Bayesian Assurance – Get an informative answer on how likely it is to see a “positive” result from a trial
Posterior Credible Intervals – Use prior information from pilot studies and other sources to make quicker and better decisions

22. New nQuery Tables Survival (time-to-event) Analysis
The two major area of focus in this release is on:
Survival (time-to-event) Analysis
Bayesian Statistics
New Survival Tables
New Bayes Tables
Bonus Tables - n-of-1 Trials - Gamma Regression

23. nQuery Advanced
For further details about any item discussed today, us at
Thanks for listening!

24. References
O'Hagan, A., Stevens, J. W., & Campbell, M. J. (2005). Assurance in clinical trial design. Pharmaceutical Statistics, 4(3), Dallow, N., Best, N., & Montague, T. (2017). Better Decision Making in Drug Development Through Adoption of Formal Prior Elicitation. arXiv preprint arXiv: O'Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., ... & Rakow, T. (2006). Uncertain judgements: eliciting experts' probabilities. John Wiley & Sons. Ren, S., & Oakley, J. E. (2014). Assurance calculations for planning clinical trials with time‐to‐event outcomes. Statistics in medicine, 33(1),