Why we don’t like protocol violations Yuko Y. Palesch, PhD Medical University of South Carolina.

Slides:



Advertisements
Similar presentations
CQ Deng, PhD PPD Development Research Triangle Park, NC 27560
Advertisements

FDA/Industry Workshop September, 19, 2003 Johnson & Johnson Pharmaceutical Research and Development L.L.C. 1 Uses and Abuses of (Adaptive) Randomization:
Objectives (BPS chapter 9)
LSU-HSC School of Public Health Biostatistics 1 Statistical Core Didactic Introduction to Biostatistics Donald E. Mercante, PhD.
Statistical Issues in Interpreting Clinical Trials D. L. DeMets Journal of Internal Medicine 255: “Lies, Damn Lies, and Clinical Statistics”
Systematic Review of Literature Part XIX Analyzing and Presenting Results.
Analysis & Expressing Resultd in Clinical Trials Dr. Khalili.
Shana Schwarz.  Type I (i.e. false positive) ◦ Occurs by rejecting the null hypothesis even when it is true. ◦ In other words, this is the error of accepting.
More About Type I and Type II Errors. O.J. Simpson trial: the situation O.J. is assumed innocent. Evidence collected: size 12 Bruno Magli bloody footprint,
天 津 医 科 大 学天 津 医 科 大 学 Clinical trail. 天 津 医 科 大 学天 津 医 科 大 学 1.Historical Background 1537: Treatment of battle wounds: 1741: Treatment of Scurvy 1948:
Lab exam when: Nov 27 - Dec 1 length = 1 hour –each lab section divided in two register for the exam in your section so there is a computer reserved for.
ODAC May 3, Subgroup Analyses in Clinical Trials Stephen L George, PhD Department of Biostatistics and Bioinformatics Duke University Medical Center.
Design and Analysis of Clinical Study 12. Randomized Clinical Trials Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia.
How does the process work? Submissions in 2007 (n=13,043) Perspectives.
Journal Club Alcohol and Health: Current Evidence July-August 2006.
Common Problems in Writing Statistical Plan of Clinical Trial Protocol Liying XU CCTER CUHK.
Meta-analysis & psychotherapy outcome research
How Science Works Glossary AS Level. Accuracy An accurate measurement is one which is close to the true value.
BS704 Class 7 Hypothesis Testing Procedures
PY 427 Statistics 1Fall 2006 Kin Ching Kong, Ph.D Lecture 6 Chicago School of Professional Psychology.
Sample Size Determination
Cohort Studies Hanna E. Bloomfield, MD, MPH Professor of Medicine Associate Chief of Staff, Research Minneapolis VA Medical Center.
RANDOMIZED CLINICAL TRIALS. What is a randomized clinical trial?  Scientific investigations: examine and evaluate the safety and efficacy of new drugs.
6th June 2004 By Norah A A Al Khathlan M.D. JOURNAL CLUB A Comparison of High-Dose and Standard-dose Epinephrine in Children with Cardiac Arrest NEJM 350;17April.
Accredited Member of the Association of Clinical Research Professionals, USA Tips on clinical trials Maha Al-Farhan B.Sc, M.Phil., M.B.A., D.I.C.
NDA Study MP-US-M01. Division of Oncology Drug Products 2 Federal Food, Drug, and Cosmetic Act of 1962 Substantial Evidence = Adequate and well-controlled.
Educational Research Chapter 13 Post-Analysis Considerations Gay, Mills, and Airasian.
Basic Statistics in Clinical Research Slides created from article by Augustine Onyeaghala (MSc, PhD, PGDQA, PGDCR, MSQA,
1 Efficacy Results NDA (MTP-PE) Laura Lu Statistical Reviewer Office of Biostatistics FDA/CDER.
Chapter 4 Hypothesis Testing, Power, and Control: A Review of the Basics.
Hypothesis Testing A hypothesis is a conjecture about a population. Typically, these hypotheses will be stated in terms of a parameter such as  (mean)
CME Disclosure Statement The North Shore LIJ Health System adheres to the ACCME's new Standards for Commercial Support. Any individuals in a position.
Inference in practice BPS chapter 16 © 2006 W.H. Freeman and Company.
Clinical Trials. What is a clinical trial? Clinical trials are research studies involving people Used to find better ways to prevent, detect, and treat.
Intervention Studies Principles of Epidemiology Lecture 10 Dona Schneider, PhD, MPH, FACE.
The paired sample experiment The paired t test. Frequently one is interested in comparing the effects of two treatments (drugs, etc…) on a response variable.
Inference for a Single Population Proportion (p).
Research Design. Research is based on Scientific Method Propose a hypothesis that is testable Objective observations are collected Results are analyzed.
Power and Sample Size Determination Anwar Ahmad. Learning Objectives Provide examples demonstrating how the margin of error, effect size and variability.
Study design P.Olliaro Nov04. Study designs: observational vs. experimental studies What happened?  Case-control study What’s happening?  Cross-sectional.
Analyzing Randomized Control Trial: ITT vs. PP vs. AT Proceedings from Journal club….. Vikash.
Consumer behavior studies1 CONSUMER BEHAVIOR STUDIES STATISTICAL ISSUES Ralph B. D’Agostino, Sr. Boston University Harvard Clinical Research Institute.
1 Statistical Review Dr. Shan Sun-Mitchell. 2 ENT Primary endpoint: Time to treatment failure by day 50 Placebo BDP Patients randomized Number.
1 Chapter 10: Introduction to Inference. 2 Inference Inference is the statistical process by which we use information collected from a sample to infer.
1 Statistics in Drug Development Mark Rothmann, Ph. D.* Division of Biometrics I Food and Drug Administration * The views expressed here are those of the.
1 THE ROLE OF COVARIATES IN CLINICAL TRIALS ANALYSES Ralph B. D’Agostino, Sr., PhD Boston University FDA ODAC March 13, 2006.
What is a non-inferiority trial, and what particular challenges do such trials present? Andrew Nunn MRC Clinical Trials Unit 20th February 2012.
Biostatistics in Practice Peter D. Christenson Biostatistician LABioMed.org /Biostat Session 4: Study Size and Power.
Biostatistics in Practice Peter D. Christenson Biostatistician Session 4: Study Size and Power.
August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 1 Statistical Considerations for Topical Microbicide Phase 2 and 3 Trial Designs: A Regulatory.
Lecture 17 Dustin Lueker.  A way of statistically testing a hypothesis by comparing the data to values predicted by the hypothesis ◦ Data that fall far.
C82MST Statistical Methods 2 - Lecture 1 1 Overview of Course Lecturers Dr Peter Bibby Prof Eamonn Ferguson Course Part I - Anova and related methods (Semester.
Stroke Hyperglycemia Insulin Network Effort (SHINE) Trial Site Monitoring Shirley Frederiksen Donna Harsh.
1 BLA Sipuleucel-T (APC-8015) FDA Statistical Review and Findings Bo-Guang Zhen, PhD Statistical Reviewer, OBE, CBER March 29, 2007 Cellular, Tissue.
European Patients’ Academy on Therapeutic Innovation The Purpose and Fundamentals of Statistics in Clinical Trials.
Biostatistics in Practice Session 6: Data and Analyses: Too Little or Too Much Youngju Pak Biostatistician
1 Pulminiq™ Cyclosporine Inhalation Solution Pulmonary Drug Advisory Committee Meeting June 6, 2005 Statistical Evaluation Statistical Evaluation Jyoti.
Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.1 Sample Size and Power Considerations.
1 Chapter 6 SAMPLE SIZE ISSUES Ref: Lachin, Controlled Clinical Trials 2:93-113, 1981.
Critical Appraisal Course for Emergency Medicine Trainees Module 3 Evaluation of a therapy.
Copyright © 2009 Pearson Education, Inc. Chapter 25 Paired Samples and Blocks.
Inference for a Single Population Proportion (p)
Methods to Handle Noncompliance
Chapter 33 Introduction to the Nursing Process
Randomized Trials: A Brief Overview
S1316 analysis details Garnet Anderson Katie Arnold
Common Problems in Writing Statistical Plan of Clinical Trial Protocol
Intent-to-treat Analysis of Randomized Clinical Trials.
Evidence Based Practice
Use of Piecewise Weighted Log-Rank Test for Trials with Delayed Effect
Presentation transcript:

Why we don’t like protocol violations Yuko Y. Palesch, PhD Medical University of South Carolina

Humans are the worst experimental units.

Because…  They are very heterogeneous

Because…  They are very heterogeneous  They need to be respected and treated fairly and with dignity

Because…  They are very heterogeneous  They need to be respected and treated fairly and with dignity  They have a mind of their own

In addition, those of us who design and conduct the trial are also humans

To Err = Human

Some Errors in ProTECT  Eligibility / Randomization  TBI mimic (e.g., alcohol intoxication) randomized (??%)  Study drug not given within 4 hours (16%)  Study drug administration  Wrong drug given / crossovers (<1%)  Infusion interruptions (12%)  Taper errors (30%)  Missing data  Lost to follow-up (<1%)  Outcomes assessed beyond the time window (13%)  Data quality and timeliness  Incorrect and/or late data entries

But why do we care about these errors? the ITT Analysis

ITT stands for… (choose one) 1.Incentive to Treat 2.Insure to Treat 3.Intent to Treat 4.International Treatment Trial 5.Incredibly Tedious Trial 6.Both 3 and 5

Scenario 1 (misdiagnosis) A patient, upon arrival in the ER was mis-diagnosed to have a TBI and was randomized and treated with the study drug. Shortly thereafter, it was discovered that she was just intoxicated, and terminated from the study. Should the subject be included in the ITT analysis of the primary outcome? A.Yes B.No

Scenario 2 (time delay) An eligible patient was randomized at 3.5 hours from injury, but the study drug was initiated at 5 hours. He died from his injury within two hours of study drug initiation. Should the subject be included in the ITT analysis of the primary outcome? A.Yes B.No

Scenario 3 (crossover) An eligible patient was randomized but, because of the study drug kit accounting error, the “wrong” study drug was administered. Nevertheless, the subject is followed through 6 months per the protocol. Should the subject be included in the ITT analysis of the primary outcome? A.Yes B.No

Scenario 4 (drug admin error) An eligible patient was randomized but, because of a variety of reasons, including infusion interruptions, only 1/2 of the study drug dose was administered. The subject was lost-to-follow- up at 3 months. Should the subject be included in the ITT analysis of the primary outcome? A.Yes B.No

Scenario 5 (non-EFIC Trial) An eligible patient is randomized and treated with study drug, but it is discovered upon site monitoring 2 months later that a signed and dated Informed Consent was not obtained prior to randomization. Should the subject be included in the ITT analysis of the primary outcome? A.Yes B.No

Scenario 6 (consent withdrawal) An eligible patient is randomized at 2 hours from injury and IC obtained from the LAR within the hour. But the family asks for DNR before the study drug is administered and withdraws consent. Should the subject be included in the ITT analysis of the primary outcome? A.Yes B.No

Definition of ITT Analysis Analysis that includes all randomized patients in the groups to which they were randomly assigned, regardless of their adherence with the entry criteria, regardless of the treatment they actually received, and regardless of subsequent withdrawal from treatment or deviation from the protocol. Fisher et al. In: Statistical issues in drug research and development. New York:Marcel Dekker, pp , 1990.

Effect of errors on the statistical test and interpretation of results

Primary outcome is dichotomous - good vs bad: * Using alpha (two-sided) = 0.05 and power = 85%; accounting for two planned interim analysis. Group Hypothesized % good outcome N* PRG60%462 PLC50%462 Tx effect∆=10% ProTECT Trial 1º Hypothesis

A Hypothetical Scenario Subject Types PRG GroupPLC Group ABCDEF How many? % Good outcome among them Col. A x Col. B How many? % Good outcome among them Col. D x Col. B Misdiagnose alcohol intoxication 1%90%0.0091%90%0.009 Time to study drug initiation >4 hrs 6%50%0.036%50%0.03 Crossovers1%50%0.0051%60%0.006 Drug infusion errors10%50%0.0510%50%0.05 Remainder (i.e., protocol compliers) 82%60% %50%0.41 Observed good outcome 58.6% 50.5% Observed difference 8.1% (instead of 10%) Hence, dilution of treatment effect Assume true PRG and PLC % good outcome are 60% and 50%, respectively

 With N=924, we only have 67% power to detect a difference of 8.1%, even if the true PRG effect is 10% better than PLC.  OR we’d need an additional 482 subjects to keep the original 85% power (or a total of N=1,404) (NOTE: Max total N planned in protocol = 1,140 to account for some dilution effect)  Therefore, because of the errors, we may fail to achieve statistical significance, and the ProTECT Trial will be deemed negative / neutral study, even if PRG is an effective treatment.  Dilution of Tx Effect

Scenario 4: Lost to follow-up Scenario 6: Withdrawal of consent  Missing data  Need to impute (make up) data for ITT  Will also contribute to the dilution of the tx effect  May cause biased results, especially if Missing NOT At Random (MNAR) Bias Effect

“…no statistical analyses can ever adequately adjust for missing data, despite many techniques that attempt to do so.” DeMets DL. Journal of Internal Medicine 2004; 255:529–537 Bias Effect (cont’d)

So why do the ITT analysis? Why not do Per Protocol analysis?

Problem with PP Analysis  Assumes that the non-compliers do not differ in their state of health from those who comply.  And that the decision to comply is not itself influenced by treatment.  If assumptions are incorrect, we have subset selection bias which causes increase in false positive errors.  In practice, EXTREMELY difficult to determine who belongs in PP analysis.

Anturane Infarction Trial Example Temple R, Pledger GW. The FDA's critique of the Anturane Reinfarction Trial. New Engl J Med 1980; 303: 1488–92. Compared sulfinpyrazone vs placebo in post-heart attack patients. Original study results only used those deemed “eligible” post-hoc with p-value=0.07. Results of re-analysis by the FDA:

Anturane Infarction Trial Example Temple R, Pledger GW. The FDA's critique of the Anturane Reinfarction Trial. New Engl J Med 1980; 303: 1488–92. Compared sulfinpyrazone vs placebo in post-heart attack patients. Original study results only used those deemed “eligible” post-hoc with p-value=0.07. Results of re-analysis by the FDA:

Conclusion To ensure correct statistical inference from the study:  Minimize randomization and implementation errors.  Avoid missing data.  Enter data in a timely manner so that the interim analysis can be performed with complete and accurate data.