Simplicity, Complexity and Modelling Stephen Senn (c) Stephen Senn1.

Slides:



Advertisements
Similar presentations
Appraisal of an RCT using a critical appraisal checklist
Advertisements

What is a review? An article which looks at a question or subject and seeks to summarise and bring together evidence on a health topic.
Designing an impact evaluation: Randomization, statistical power, and some more fun…
Standardized Scales.
Understanding p-values Annie Herbert Medical Statistician Research and Development Support Unit
Comparing Two Proportions (p1 vs. p2)
Objectives 10.1 Simple linear regression
Statistical Analysis and Data Interpretation What is significant for the athlete, the statistician and team doctor? important Will Hopkins
Meta-analysis: summarising data for two arm trials and other simple outcome studies Steff Lewis statistician.
LSU-HSC School of Public Health Biostatistics 1 Statistical Core Didactic Introduction to Biostatistics Donald E. Mercante, PhD.
This Outcome report is based on data from clients who completed a Pain Management Programme at the RealHealth Treatment Centre in Coventry between May.
Basic Design Consideration. Previous Lecture Definition of a clinical trial The drug development process How different aspects of the effects of a drug.
Estimation of Sample Size
Topic 6: Introduction to Hypothesis Testing
Critical Appraisal for MRCGP Jim McMorran Coventry GP GP trainer Editor GPnotebook (
Point and Confidence Interval Estimation of a Population Proportion, p
Common Problems in Writing Statistical Plan of Clinical Trial Protocol Liying XU CCTER CUHK.
Basic Business Statistics, 10e © 2006 Prentice-Hall, Inc. Chap 8-1 Chapter 8 Confidence Interval Estimation Basic Business Statistics 10 th Edition.
Meta-analysis & psychotherapy outcome research
Journal Club Alcohol, Other Drugs, and Health: Current Evidence January–February 2010.
Sample Size Determination Ziad Taib March 7, 2014.
This Outcome report is based on data from patients who completed a Pain Management Programme at the RealHealth Treatment Centre in Coventry between November.
Are the results valid? Was the validity of the included studies appraised?
Inference in practice BPS chapter 16 © 2006 W.H. Freeman and Company.
Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 8-1 Chapter 8 Confidence Interval Estimation Basic Business Statistics 11 th Edition.
Confidence Interval Estimation
Power and Sample Size Determination Anwar Ahmad. Learning Objectives Provide examples demonstrating how the margin of error, effect size and variability.
Research Skills Basic understanding of P values and Confidence limits CHE Level 5 March 2014 Sian Moss.
Statistical Decision Making. Almost all problems in statistics can be formulated as a problem of making a decision. That is given some data observed from.
This Outcome Report is based on data from patients who completed a Functional Restoration Programme (FRP) at the RealHealth Treatment Centre in Coventry.
IMMPACT(c) Stephen Senn On the interpretation of responder analyses and NNTs Stephen Senn.
+ The Practice of Statistics, 4 th edition – For AP* STARNES, YATES, MOORE Unit 5: Hypothesis Testing.
+ The Practice of Statistics, 4 th edition – For AP* STARNES, YATES, MOORE Chapter 9: Testing a Claim Section 9.1 Significance Tests: The Basics.
Biostatistics Case Studies 2008 Peter D. Christenson Biostatistician Session 5: Choices for Longitudinal Data Analysis.
Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 4 Gathering Data Section 4.3 Good and Poor Ways to Experiment.
Estimating Causal Effects from Large Data Sets Using Propensity Scores Hal V. Barron, MD TICR 5/06.
1 OTC-TFM Monograph: Statistical Issues of Study Design and Analyses Thamban Valappil, Ph.D. Mathematical Statistician OPSS/OB/DBIII Nonprescription Drugs.
1 THE ROLE OF COVARIATES IN CLINICAL TRIALS ANALYSES Ralph B. D’Agostino, Sr., PhD Boston University FDA ODAC March 13, 2006.
Examination of Analysis Methods for Positive Continuous Dependent Variables: Model Fit and Cost Saving Implications Brian P SmithMaria De Yoreo Biostatistics.
Lecture 9: Analysis of intervention studies Randomized trial - categorical outcome Measures of risk: –incidence rate of an adverse event (death, etc) It.
Federal Institute for Drugs and Medical Devices The BfArM is a Federal Institute within the portfolio of the Federal Ministry of Health (BMG) The use of.
Medical Statistics as a science
EBM --- Journal Reading Presenter :呂宥達 Date : 2005/10/27.
Compliance Original Study Design Randomised Surgical care Medical care.
Statistical inference Statistical inference Its application for health science research Bandit Thinkhamrop, Ph.D.(Statistics) Department of Biostatistics.
Biostatistics Basics: Part I Leroy R. Thacker, PhD Associate Professor Schools of Nursing and Medicine.
The Impact Factor (IF): What Is It Good For? Richard M. Rocco, PhD October
+ The Practice of Statistics, 4 th edition – For AP* STARNES, YATES, MOORE Unit 5: Hypothesis Testing.
Course: Research in Biomedicine and Health III Seminar 5: Critical assessment of evidence.
Statistics (cont.) Psych 231: Research Methods in Psychology.
The inference and accuracy We learned how to estimate the probability that the percentage of some subjects in the sample would be in a given interval by.
Date of download: 9/20/2016 Copyright © 2016 American Medical Association. All rights reserved. From: Effects of a Palliative Care Intervention on Clinical.
Simulation setup Model parameters for simulations were tuned using repeated measurement data from multiple in-house completed studies and baseline data.
An Alternative to Data Imputation in Analgesic Clinical Trials David Petullo, Thomas Permutt, Feng Li Division of Biometrics II, Office of Biostatistics.
Stats Methods at IC Lecture 3: Regression.
Copyright © 2013 American Medical Association. All rights reserved.
Statistics in Clinical Trials: Key Concepts
The Importance of Adequately Powered Studies
Unit 5: Hypothesis Testing
Neal B, et al. Diabetes Care 2015;38:403–411
Chapter 8: Inference for Proportions
Common Problems in Writing Statistical Plan of Clinical Trial Protocol
Significance Tests: The Basics
Significance Tests: The Basics
Psych 231: Research Methods in Psychology
Psych 231: Research Methods in Psychology
Handling Missing Not at Random Data for Safety Endpoint in the Multiple Dose Titration Clinical Pharmacology Trial Li Fan*, Tian Zhao, Patrick Larson Merck.
Simvastatin in Patients With Prior Cerebrovascular Disease: HPS
Medical Statistics Exam Technique and Coaching, Part 2 Richard Kay Statistical Consultant RK Statistics Ltd 22/09/2019.
Presentation transcript:

Simplicity, Complexity and Modelling Stephen Senn (c) Stephen Senn1

Basic Thesis There is an increased enthusiasm for modelling in the pharmaceutical industry A great deal of interesting work is being done However many simple things that we have known about for decades are not being done It is time we changed this (c) Stephen Senn2

Examples Improvements Mixed models replacing summary measures Better approaches to missing data More use of non-linear models Continued bad practice Obsession with the log rank test Change from baseline – Percentage change from baseline Dichotomisation – Responder analysis Ordinal measures treated as categorical Fear of linearity Failure to implement decision analysis (c) Stephen Senn3

Information and Inference 1 A Simple Model (c) Stephen Senn4

Information and Inference 2 A Simple Model (c) Stephen Senn5  Y  Y

Simulation (An Aside) In performing simulations be very careful! Simulations (generally) run forward from model to data But inference runs backward from data to model Are you sure you know the difference? Are you sure that your simulation is relevant? (c) Stephen Senn6

Consequences Expect to be disappointed – Senn’s law ‘you can expect to be disappointed even if you have taken account of Senn’s law’ The less information you gather the more disappointed you will be Reducing the cost of the information you have to gather is much more valuable than reducing the amount of information you need to gather (c) Stephen Senn7

Flexible Designs Good Flexibility per se is always valuable Killing what is useless is good – Treatments – Doses – Measures? Bad Can lead to less information on treatments eventually chosen What optimises power for a controlled type I error rate is bad for many other criteria Can lead to unnatural weighting of information (c) Stephen Senn8

How to do better for free Use original values rather than dichotomies Make more use of covariates Study past trials to find out what works and what doesn’t Build planning data-bases Be sensible about dose-response (c) Stephen Senn9

10

(c) Stephen Senn11 A common definition of response in hypertension Y = outcome, X = baseline If (Y 95)  (Y < 0.9X) patient ‘responds

A Thought Experiment Imagine a cross-over trial in hypertension Patients randomised to receive ACE II inhibitor or placebo in random order Then we do it again Each patient does the cross-over twice We can compare each patient’s response under ACE II to placebo twice (c) Stephen Senn12

Design First Cross-overSecond Cross-over Period Sequence1234 IABAB IIBABA IIIABBA IVBAAB (c) Stephen Senn13

(c) Stephen Senn14

(c) Stephen Senn15

(c) Stephen Senn16

(c) Stephen Senn17

Significant differences in favor of tiotropium were observed at all time points for the mean absolute change in the SGRQ total score (ranging from 2.3 to 3.3 units, P<0.001), although the differences on average were below what is considered to have clinical significance (Fig. 2D). The overall mean between-group difference in the SGRQ total score at any time point was 2.7 (95% confidence interval [CI], 2.0 to 3.3) in favor of tiotropium (P<0.001). A higher proportion of patients in the tiotropium group than in the placebo group had an improvement of 4 units or more in the SGRQ total scores from baseline at 1 year (49% vs. 41%), 2 years (48% vs. 39%), 3 years (46% vs. 37%), and 4 years (45% vs. 36%) (P<0.001 for all comparisons). (My emphasis) From the UPLIFT Study, NEJM, 2008 Tiotropium v Placebo in Chronic Obstructive Pulmonary Disease (c) Stephen Senn18

(c) Stephen Senn19

(c) Stephen Senn20

In summary…this is rather silly If there is sufficient measurement error even if the true improvement is identically 2.7, some will show an ‘improvement’ of 4 The conclusion that there is a higher proportion of true responders by the standard of 4 points under treatment than under placebo is quite unwarranted So what is the point of analysing ‘responders’? (c) Stephen Senn21

Who are the authors? 1.Tashkin, DP, Celli, B, Senn, S, Burkhart, D, Kesten, S, Menjoge, S, Decramer, M. A 4-Year Trial of Tiotropium in Chronic Obstructive Pulmonary Disease, N Engl J Med Personal note. I am proud to have been involved in this important study and have nothing but respect for my collaborators. The fact that, despite the fact that two of us are statisticians, we have ended up publishing something like this shows how deeply ingrained the practice of responder analysis is in medical research. We must do something to change this. (c) Stephen Senn22

The “Wisdom” of the CHMP (c) Stephen Senn23 It is recommended to define responders, for an assessment of proportions of patients with a clinical relevant reduction in pain score. Subjects with a 30% to 50% reduction in assessment scale as compared to baseline are considered responders. The between treatment groups comparison of responder rates is recommended as primary end-point of the confirmatory studies. (My italics.)

An Example of What I Mean (c) Stephen Senn24 Baron R, Mayoral V, Leijon G, Binder A, Steigerwald I, Serpell M. (2009) The study was a two-stage adaptive, randomized, controlled, open-label, multicentre trial. The pre-planned primary study end-point was the rate of treatment responders, defined as completing patients experiencing a reduction from baseline of >= 2 points or an absolute value of <= 4 points on the 11-item numerical rating scale of recalled average pain intensity over the last 3 days (NRS-3), after 4 weeks of treatment.

Statements I Hate ‘Because the hazard might not be proportional we will use the log rank test’ ‘Change from baseline is more relevant clinically’ ‘We can’t use ANCOVA because there might be a baseline by treatment interaction’ ‘Physicians find responder analyses easier to understand’ (c) Stephen Senn25

In Summary There are many new complex things we can do to improve efficiency of clinical trials – These things have required theoretical and computational advances – They are either relatively new or it is relatively new that we have software to make them easy However, there are many simple things we could do which would bring an even bigger dividend (c) Stephen Senn26

A Modest Proposal Every time a medical advisor proposes to have a simple responder analysis based on a dichotomy as the primary approach two sample size calculations should be done – One using the dichotomy – The other using the continuous outcome and including prognostic covariates The medical advisor should then be required to write an essay justifying the extra cost of his or her preferred approach (c) Stephen Senn27