Data Mining AERS FDA’s (Spontaneous) Adverse Event Reporting System Division of Drug Risk Evaluation Office of Drug Safety Carolyn McCloskey, M.D., M.P.H. Drug Safety and Risk Management Advisory Committee May 18, 2005
Outline Brief History of data mining (DM) activities at the FDA Current Use in the Division of Drug Risk Evaluation (DDRE) – CRADA Application Development - WebVDME Pilot - Examples and Selected Conclusions Other CRADA Activities Future Directions in DDRE Pharmacovigilance
Historical Overview February 1998: Office of Women’s Health Grant (Ana Szarfman) March 2003 – July 2005: Cooperative Research & Development Agreement (CRADA) Research in more advanced methodology (Drug-drug interaction & logistic regression modeling)
CRADA - Cooperative Research and Development Agreement Web Visual Data Mining Environment (WebVDME) With Lincoln Technologies, Inc. March 2003 – July 2005
CRADA Objectives User-friendly application Web-based environment Performance Evaluations by User Groups Training Continued development and refinement
Empirical Bayes Geometric Mean (EBGM) An observed/expected score Adjusts for sampling variation (e.g., sample size) No adjustment for reporting bias Allows data stratification in DM software Standard stratification: gender, age group, year
EB05 – EB95 Interval Interval in which the EBGM score could be found because of sampling variability EB05 is the lower bound EB95 is the upper bound 90% probability of EBGM occurring between EB05 and EB95
Example of Sampling Variability Adjustment (for small numbers) Adverse Event (AE) Observed Count (N) Expected Count (E) Obs/Exp* N/E (RR) EBGM** EB05 EB95 Myalgia 1,665 334 4.99 4.97 4.78 5.18 Spinal Osteoarthritis 17 3 6.16 4.54 3.03 6.60 * Obs = Observed; Exp = expected ** EBGM = Empirical Bayes Geometric Mean - adjusts for sampling variability
CRADA Pre-Pilot Performance Evaluations May 2003 – October 2003 WebVDME record retrieval validation with AERS case retrieval Multiple trade & ingredient nomenclature Drug assignment allocations (suspect & concomitant) Duplicate removal logic OIT system performance evaluations
CRADA Pilot Medical Safety Evaluators Evaluated data mining scores for drugs and biologics Indication vs. new signal Labeled vs. unlabeled Innocent bystanders or concomitant medications Drug names Safety Evaluators’ ease of use
CRADA Pilot Epidemiologists Evaluated Temporal trends Drug name selections Suspect & Concomitant selections Stratification Signal strengths Epidemiologists’ ease of use
Pilot Examples New vs. Old Drug EBGM Rankings & Confidence Intervals
New Drug (1 Year) EBGM (EB05-EB95)
OLDER DRUG (>10 Years) EBGM (EB05-EB95)
Selected Pilot Conclusions - 1 WebVDME DM - Statistical tool assists in identifying unusual patterns with AERS data but ! Patterns Need Interpretation!
Selected Pilot Conclusions - 2 Knowledge of data in database imperative to interpret Clinical & pharmacologic activities of drug Other - reporting disproportionalities which also reflect limitations of AERS data
AERS DATABASE LIMITATIONS
Continuing CRADA Activities - DDRE March 2004 - Present Access by interested Reviewers to WebVDME Training Application Refinements addressing Technical problems identified Customization by user needs
Summary – 1 DM Signals in DDRE Assist in prioritizing evaluations of case series Identify associations, NOT a cause or degree of risk Reflect limitations of data
Summary – 2 DM Signals in DDRE Threshold a compromise between sensitivity and specificity (false positives & negatives) Absence of a DM signal ≠ absence of a drug-event association Magnitude of DM scores ≠ magnitude of risk Always require clinical case report and reporting bias evaluation
Future Directions of DM Prospective signal detection Parallel use with traditional pharmacovigilance methods in DDRE Continued research in more advanced methodology (Drug-drug interaction & logistic regression modeling)
Acknowledgments Division of Drug Risk Evaluation Rita Ouellet-Hellstrom, Ph.D., M.P.H. Mary Willy, Ph.D. Mark Avigan, M.D., C.M.
THANK YOU