A Comparison of Some Methods for Detection of Safety Signals in Randomised Controlled Clinical Trials Raymond Carragher Project Supervisors: Prof. Chris.

Slides:



Advertisements
Similar presentations
THE USE OF HISTORICAL CONTROLS IN DEVICE STUDIES Vic Hasselblad Duke Clinical Research Institute.
Advertisements

Bayesian Hierarchical Models for Detecting Safety Signals in Clinical Trials H. Amy Xia and Haijun Ma Amgen, Inc. MBSW 2009, Muncie, IN March 20, 2009.
1 Health Warning! All may not be what it seems! These examples demonstrate both the importance of graphing data before analysing it and the effect of outliers.
Statistical Modeling and Data Analysis Given a data set, first question a statistician ask is, “What is the statistical model to this data?” We then characterize.
Analysis of dose-response microarray data using Bayesian Variable Selection (BVS) methods: Modeling and multiplicity adjustments 7th meeting of the Eastern.
Bayesian posterior predictive probability - what do interim analyses mean for decision making? Oscar Della Pasqua & Gijs Santen Clinical Pharmacology Modelling.
Analysis of variance (ANOVA)-the General Linear Model (GLM)
Copyright © 2013, SAS Institute Inc. All rights reserved. LEVERAGE THE CDISC DATA MODEL TO STREAMLINE ANALYTICAL WORKFLOWS KELCI J. MICLAUS, PH.D. RESEARCH.
1 An Overview of Multiple Testing Procedures for Categorical Data Joe Heyse IMPACT Conference November 20, 2014.
RELATIVE RISK ESTIMATION IN RANDOMISED CONTROLLED TRIALS: A COMPARISON OF METHODS FOR INDEPENDENT OBSERVATIONS Lisa N Yelland, Amy B Salter, Philip Ryan.
ODAC May 3, Subgroup Analyses in Clinical Trials Stephen L George, PhD Department of Biostatistics and Bioinformatics Duke University Medical Center.
Estimation and Reporting of Heterogeneity of Treatment Effects in Observational Comparative Effectiveness Research Prepared for: Agency for Healthcare.
Elements of a clinical trial research protocol
Statistics for Managers Using Microsoft® Excel 5th Edition
Chapter 11: Sequential Clinical Trials Descriptive Exploratory Experimental Describe Find Cause Populations Relationships and Effect Sequential Clinical.
Differentially expressed genes
Statistics Micro Mini Threats to Your Experiment!
Knowledge Update Clinical documentation: from preclinical studies to drug registration Split, 12 September 2008.
Réunion Ambulatoires SAS,  Similarly, a statistically significant MADRS reduction over time was found (F=156.2, p 800 mg/day) and low (
Thoughts on Biomarker Discovery and Validation Karla Ballman, Ph.D. Division of Biostatistics October 29, 2007.
POSC 202A: Lecture 1 Introductions Syllabus R Homework #1: Get R installed on your laptop; read chapters 1-2 in Daalgard, 1 in Zuur, See syllabus for Moore.
Use of the False Discovery Rate for Evaluating Clinical Safety Data Joseph F. Heyse Devan V. Mehrotra Clinical Biostatistics – Vaccines Merck Research.
Epidemiology The Basics Only… Adapted with permission from a class presentation developed by Dr. Charles Lynch – University of Iowa, Iowa City.
Research Methods in Computer Science Lecture: Quantitative and Qualitative Data Analysis | Department of Science | Interactive Graphics System.
Applying False Discovery Rate (FDR) Control in Detecting Future Climate Changes ZongBo Shang SIParCS Program, IMAGe, NCAR August 4, 2009.
Background to Adaptive Design Nigel Stallard Professor of Medical Statistics Director of Health Sciences Research Institute Warwick Medical School
Empowering Evidence: Basic Statistics June 3, 2015 Julian Wolfson, Ph.D. Division of Biostatistics School of Public Health.
Antidepressants and Suicidality in Adults: Statistical Evaluation Mark Levenson, Ph.D.* and Chris Holland, M.S. Statistical Safety Reviewers Quantitative.
General Register Office for S C O T L A N D information about Scotland's people General Register Office for Scotland 2006 Census Test – Evaluation Methodology.
Understanding real research 4. Randomised controlled trials.
False Discovery Rates for Discrete Data Joseph F. Heyse Merck Research Laboratories Graybill Conference June 13, 2008.
A A R H U S U N I V E R S I T E T Faculty of Agricultural Sciences Introduction to analysis of microarray data David Edwards.
1 Statistics in Drug Development Mark Rothmann, Ph. D.* Division of Biometrics I Food and Drug Administration * The views expressed here are those of the.
Back to basics – Probability, Conditional Probability and Independence Probability of an outcome in an experiment is the proportion of times that.
DSBS Discussion: Multiple Testing 28 May 2009 Discussion on Multiple Testing Prepared and presented by Lars Endahl.
Multiple Testing Matthew Kowgier. Multiple Testing In statistics, the multiple comparisons/testing problem occurs when one considers a set of statistical.
Using Predictive Classifiers in the Design of Phase III Clinical Trials Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute.
The Multiple Comparisons Problem in IES Impact Evaluations: Guidelines and Applications Peter Z. Schochet and John Deke June 2009, IES Research Conference.
August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 1 Statistical Considerations for Topical Microbicide Phase 2 and 3 Trial Designs: A Regulatory.
Statistical Testing with Genes Saurabh Sinha CS 466.
Power & Sample Size Dr. Andrea Benedetti. Plan  Review of hypothesis testing  Power and sample size Basic concepts Formulae for common study designs.
© Imperial College LondonPage 1 A method for estimating the cost of reducing the false alarm rate in multi- institution performance monitoring using CUSUM.
Bayesian Statistics & Innovative Trial Design April 3, 2006 Jane Perlmutter
The False Discovery Rate A New Approach to the Multiple Comparisons Problem Thomas Nichols Department of Biostatistics University of Michigan.
Organization of statistical research. The role of Biostatisticians Biostatisticians play essential roles in designing studies, analyzing data and.
IMPORTANCE OF STATISTICS MR.CHITHRAVEL.V ASST.PROFESSOR ACN.
A shared random effects transition model for longitudinal count data with informative missingness Jinhui Li Joint work with Yingnian Wu, Xiaowei Yang.
Postgraduate books recommended by Degree Management and Postgraduate Education Bureau, Ministry of Education Medical Statistics (the 2nd edition) 孙振球 主.
 An exposure-response (E-R) analysis in oncology aims at describing the relationship between drug exposure and survival and in addition aims at comparing.
BIOSTATISTICS Lecture 2. The role of Biostatisticians Biostatisticians play essential roles in designing studies, analyzing data and creating methods.
John W. Tukey’s Multiple Contributions to Statistics at Merck Joseph F. Heyse Merck Research Laboratories Third International Conference on Multiple Comparisons.
Marshall University School of Medicine Department of Biochemistry and Microbiology BMS 617 Lecture 6 –Multiple hypothesis testing Marshall University Genomics.
A Quantitative Overview to Gene Expression Profiling in Animal Genetics Armidale Animal Breeding Summer Course, UNE, Feb Analysis of (cDNA) Microarray.
Guidelines for Multiple Testing in Impact Evaluations Peter Z. Schochet June 2008.
2 years versus 1 year of adjuvant trastuzumab for HER2-positive breast cancer (HERA): an open-label, randomised controlled trial Aron Goldhirsch, Richard.
Estimating the False Discovery Rate in Genome-wide Studies BMI/CS 576 Colin Dewey Fall 2008.
The Importance of Adequately Powered Studies
8. Causality assessment:
Confidence Intervals and p-values
Milwaukee, Wisconsin(USA)
Random error, Confidence intervals and P-values
9. Introduction to signal detection
Critical Reading of Clinical Study Results
Strategies for Implementing Flexible Clinical Trials Jerald S. Schindler, Dr.P.H. Cytel Pharmaceutical Research Services 2006 FDA/Industry Statistics Workshop.
Analyzing Intervention Studies
A Two-Stage Microbial Association Mapping Framework with Advanced FDR Control Jiyuan Hu Ph.D. Division of Biostatistics New York University Langone Health.
What Do We Know About Estimators for the Treatment Policy Estimand
Handling Missing Not at Random Data for Safety Endpoint in the Multiple Dose Titration Clinical Pharmacology Trial Li Fan*, Tian Zhao, Patrick Larson Merck.
The 3rd Stat4Onc Annual Symposium
Detecting Treatment by Biomarker Interaction with Binary Endpoints
Presentation transcript:

A Comparison of Some Methods for Detection of Safety Signals in Randomised Controlled Clinical Trials Raymond Carragher Project Supervisors: Prof. Chris Robertson (University of Strathclyde) Dr. Ian Bradbury (Frontier Science (Scotland)) Dr. David Young (University of Strathclyde) The project is EPSRC funded with Frontier Science (Scotland) as the industrial partner.

Overview The main aim of this presentation is to: ●Compare (by way of a simulation) a number of existing approaches for analysing Adverse Events using groupings in clinical trials. ●Discuss the Adverse Event groupings and methods ●Look at a simulation study and the results ●Summary and conclusions

Adverse Events ●Routinely recorded during a trial ● Severity - Common Terminology Criteria for Adverse Events provides a scale from 1–5 (1 = mild,..., 5 = death) ● Time of occurrence and/or duration ● Effect sizes may be small – long follow up / large numbers of patients ● Many different types of Adverse Events – may have multiple hypotheses

Recent Approaches to Analysing Safety Data A number of recent approaches to analysing safety data have grouped what they consider to be related adverse events into body-systems or System Organ Classes. The idea being that if a treatment affects a particular body system then we may expect to see raised adverse event counts for all adverse events in that body-system. Berry, Berry - Accounting for Multiplicities in Assessing Drug Safety: A Three-Level Hierarchical Mixture Model (2004) Mehrotra, Adewale - Flagging clinical adverse experiences: reducing false discoveries without materially compromising power for detecting true signals (2011).

Body-System Hierarchy The grouping by Body-System we consider has a natural hierarchical structure, with the body-system being part of an overall body and the adverse events being associated with particular body-systems:

Methods Package: c212 – under development and untested MethodDescription HIER.BBBerry and Berry model HIER.1aSubset of HIER.BB BHControl of the False Discovery Rate by the Benjamini-Hochberg procedure DFDRDouble False Discovery Rate NOADJUnadjusted testing BONFBonferroni correction GBHGroup Benjamini-Hochberg ssBHSubset Benjamini-Hochberg

Berry and Berry Model B body systems, body-system b containing k b Adverse Events N C – number of patients in the control arm N T – number of patients in the treatment arm X bj – number of adverse events on the control arm Y bj – number of adverse events on the treatment arm AE counts: Log Odds: First Level: is the log odds-ratio for the occurrence of the AE for treatment compared to control. π b is the probability that there are no differences in rates in body-system b.

Simulation Study We used a simulation study to assess how the various methods performed with regard to detecting the raised levels of adverse events between treatment and control. Model used to generate the simulated trial data: where μ is a fixed effect and U is a random effect.

Simulation Study Results from one particular (repeated) simulation: 8 body-systems with between 1 and 11 adverse events in each body-system. 45 adverse events in total. Trials size: Trial 1 – 110 patients in each arm Trial 2 – 450 patients in each arm Trial 3 – 1100 patients in each arm For all trials: The AE rate was raised for body-system 5 for both treatment and control. The AE rate was raised for body-system 3 for treatment only. The AE rate was raised for body-system 2 for two out of 4 AEs for treatment only.

Simulation Study 500 simulations in total. Adverse Event Numbers In each simulation there are 9 Adverse Events which have underlying rate raised in treatment compared to control Adverse Events over the whole simulation Adverse Events with raised rates over the whole simulation. Flagging an Adverse Event: 95% posterior probability for Bayesian methods 5% significance level for the error controlling methods

Simulation Study Bob=zz Body-system5 Body-system3 Body-system2

Simulation Study Berry & Berry Model: HIER.BB (point mass):HIER.1a (no point mass):

Simulation Study – Trial 2 (450 per arm) MethodCorrectIncorrectMissed Berry & Berry (HIER.BB) Berry & Berry without point mass (HIER.1a) Unadjusted Testing (NOADJ) Bonferroni Correction (BONF) Double False Discovery Rate (DFDR) False Discovery Rate (BH) Group Benjamini-Hochberg (GBH) Subset Benjamini-Hochberg

Simulation Study – Trial 3 (1100 per arm) MethodCorrectIncorrectMissed Berry & Berry (HIER.BB) Berry & Berry without point mass (HIER.1a) Unadjusted Testing (NOADJ) Bonferroni Correction (BONF) Double False Discovery Rate (DFDR) False Discovery Rate (BH) Group Benjamini-Hochberg (GBH) Subset Benjamini-Hochberg

Summary The simulations have indicated that where there are relationships between the Adverse Events using groupings (body-systems) do appear to make a difference to the results. The point mass in the Berry & Berry model (HIER.BB) makes a quantitative difference to the results. For the error controlling methods it may be difficult to objectively pick a method of analysing the data before the trial. The body-system described in Berry & Berry looks to be a worthwhile structure to consider when modelling data. The models and data discussed here do not take into account the severity of events. The models and data discussed here do not take into account the timings of events.

References [1] D. V. Mehrotra and J. F. Heyse. Use of the false discovery rate for evaluating clinical safety data. Stat Methods Med Res, 13(3):227–38, [2] S. M. Berry and D. A. Berry, “Accounting for multiplicities in assessing drug safety: A three-level hierarchical mixture model,” Biometrics, vol. 60, no. 2, pp. 418–426, [3] D. V. Mehrotra and J. F. Heyse. Use of the false discovery rate for evaluating clinical safety data. Stat Methods Med Res, 13(3):227–38, [4] D. B. Dunson, A. H. Herring, and S. M. Engel, “Bayesian selection and clustering of polymorphisms in functionally related genes,” Journal of the American Statistical Association, vol. 103, no. 482, pp. 534–546, [5] J. N. S. Matthews, Introduction to Randomized Controlled Clinical Trials, Second Edition. Chapman & Hall/CRC Texts in Statistical Science, Chapman and Hall/CRC, [6] Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1):289–300, [7] J. X. Hu, H. Zhao, and H. H. Zhou. False discovery rate control with groups. J Am Stat Assoc, 105(491):1215–1227, [8] Daniel Yekutieli. False discovery rate control for non-positively regression dependent test statistics. Journal of Statistical Planning and Inference, 138(2):405–415, [9] H. Amy Xia, H. Ma, and B. P. Carlin, “Bayesian hierarchical modeling for detecting safety signals in clinical trials,” Journal of Biopharmaceutical Statistics, vol. 21, no. 5, pp. 1006–1029, 2011.