1. Advanced Statistical Methods for Translational Research
Context, Process, Purpose

2. Disclosure Statement of Financial Interest
Within the past 12 months, I or my spouse/partner have had a financial interest/arrangement or affiliation with the organization(s) listed below.
Affiliation/Financial Relationship
Company
My employer provides testing and consulting services to medical device companies
Multiple
All TCT 2018 faculty disclosures are listed online and on the App.

3. We are driven to make a scientific contribution to every medical device in the world

4. Why do we need “advanced” statistical methods?
Better answers
Faster answers
Cheaper answers

5. Dangers of “advanced” statistical methods?
Wrong answers
Delayed answers
Expensive answers

6. Dangers of “advanced” statistical methods?
Wrong answers
Delayed answers
Expensive answers
“Being a statistician is like being in Alcoholics Anonymous.
You try as hard as you can to not fool yourself”
Paraphrasing Jim Hodges

7. Examples of advanced methods
Multiple imputation for missing data Propensity score adjustment for non-randomized comparisons Finkelstein-Schoenfeld / “Win-ratio” for composite endpoints Hiearchical models Bayesian approaches: sample size re-estimation, using historical data

8. How do we evaluate a new statistical method?
Does the novel method address a practical problem?
What problems does a novel method raise?
Role of pre-specification
Operating characteristics
Urgency of public health need
𝑞(𝜃| 𝐷 0 , 𝐷, 𝛼 0 )∝ 𝐿(𝜃|𝐷)𝐿(𝜃| 𝐷 0 ) 𝛼 0 𝜋 0 (𝜃)

9. Does the method address a practical problem?
If not, why not use a traditional / non-advanced method?
How serious is the practical problem?
How well does the method address it?

10. What problems does a novel method raise?
Are the assumptions of the method reasonable at the design stage?
Are they met for the data we obtained?
How thoroughly has the new method been evaluated?
Has it been evaluated for the specific circumstances at hand?

11. Role of pre-specification
When we see a new method applied, was it pre-specified?
If not, consider the use of the method an exploratory case-study
If it was pre-specified, was the study design aligned?
Sample size calculations should be based on the planned analysis method
Crossover studies, longitudinal studies, etc., require special methods

12. Operating characteristics
Higher power (low type II error rate)
Low chance of false-positive (low type I error rate)
Under what conditions is the new method optimal/non-optimal?

13. Urgent public health need
Sometimes we need faster answers
HIV/AIDS crisis, Ebola vaccine, opioid crisis
Device improvements to solve urgent device-related problems
Need for high power is the driving consideration
What tradeoffs of other considerations can be made?
Considerations not purely statistical, e.g. shorter term endpoints

14. Context, Process, and Purpose
P-values…
“Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold”
From: The American Statistical Association Statement on P-values
Context, Process, and Purpose

15. A mock example We have a study with P<0.05!!!
Based on a small study
Clinically meaningful effect size
Analysis based on KM analysis
KM stands for Kushner-Manafort test
Just kidding – it’s Kaplan-Meier/log-rank test, but the test was not pre-specified

16. A more realistic example: Kaplan-Meier analysis was once new
Addressed practical problems
Better handling of time to event data
Better handling of censored data (losses-to-follow-up)
New problems?
Difficulty interpreting? This improved with usage
Glosses over issue of informative censoring

17. Another example: Evaluating methods for “borrowing data”
Bayesian methods can be used to “borrow” strength from prior work
Formalizes incorporation of existing knowledge
Posterior = current data x prior data
How much strength should we borrow?

18. Hierarchical Models

19. Hierarchical Models Reasonable operating characteristics
Only applicable when we have a hierarchical structure
Somewhat challenging to describe how much we’re “borrowing”

20. Prior counts as much as current data
Bayesian power prior
Traditional Bayesian inference:
Nfinal = Ncurrent + Nprior
Power prior inference
Nfinal = Ncurrent + a*Nprior
Prior counts as much as current data

21. For a <1, prior has less weight
Bayesian power prior
Traditional Bayesian inference:
Nfinal = Ncurrent + Nprior
Power prior inference
Nfinal = Ncurrent + a*Nprior
For a <1, prior has less weight

22. How should we weight historical data?
Historical and current data similar
(Easy case)
Historical and current data different
Weight equally?
Ignore prior?
Posterior
Historical data
Current data

23. Can the weight be dynamic? adaptive?
Allow weight to vary as a function of agreement?
More agreement = more weight
Less agreement = less weight

24. Advantages of dynamic prior
Better type I error rate
Less bias

25. When should we prefer fixed vs. dynamic borrowing?
While both approaches rely on theory and data… Fixed borrowing can be considered theory driven We believe a priori that historical and current data should be similar Dynamic borrowing can be considered data driven We allow that historical and current data may or may not be similar

26. When should we prefer fixed vs. dynamic borrowing?
Prefer Theory Driven
Prefer Data Driven
Reason to believe historical data
is representative
Questionable if historical data
Is representative
Identical device, patient population, protocol
Small differences in device, patient population, or protocol

27. Criticisms of dynamic borrowing and responses
Data are “used twice”; could introduce bias
Shows less bias than other methods of borrowing
Other statistical methods use data “twice” (e.g. propensity scores, multiple imputation, interim analyses)
Amount of borrowing should depend on study design and clinical considerations
Use pre-specification
Set specifics of model based on these considerations
Lack of straightforward clinical interpretation
Criticism also applies to other borrowing methods
Common tradeoff with adaptive methods
Dynamic borrowing is data driven borrowing
MDIC Workshop on Virtual Patient Methodology

28. For advanced statistical methods, consider…
Not just the p-value
but the Context, Process, and Purpose

29. Thank you!