1. Sentinel Initiative
A Comprehensive Tool for Monitoring the Safety of Approved Medical Products
Rongmei Zhang, PhD
Food and Drug Administration
Center for Drug Evaluation and Research
Division of Biometrics 7
September 8, 2016

2. Disclaimer
This presentation reflects the views of the author and should not be construed to represent the policies of the U.S. Food and Drug Administration

3. Stages of Post-Marketing Assessments
The boundary is not always well determined, as both often use similar studies and methods.
Signal Generation
Signal Identification/ Refinement
Signal Evaluation
Spontaneous case reports (e.g. FAERS)
Medical literature
Other resources (e.g. pre-clinical information, clinical trials)
Registries
Surveys
Other observational Studies
Protocol based observational studies (e.g. FDA led CMS, DoD, VA studies)
Post-marketing Safety Outcome RCT

4. Use of Sentinel in Post-Marketing Assessments
Signal Generation
Signal Refinement
Signal Evaluation
Data Mining
Tree Scan (under development)
Summary Tables
Modular Programs
Modular Programs with confounding adjustments
Protocol-based Evaluations with customized programs

5. Overview What is Sentinel Initiative? How does Sentinel work?
Example: dabigatran and bleeding events
Integrating Sentinel into regulatory programs – ARIA
Limitations and challenges

6. What is Sentinel Initiative?
National electronic healthcare data system to monitor drugs and medical products after FDA approval (launched in 2008)
Active Surveillance: FDA can initiate safety evaluations of specific products and safety outcomes
Mini-Sentinel: A pilot to test the feasibility of and develop the scientific approaches needed for Sentinel (completed in Feb 2016)

7. How does Sentinel Work?

8. Sentinel Partner Organizations
Lead – HPHC Institute
Data partners
Data Partners Expansion
Scientific partners
Institute for Health

9. Sentinel Distributed Database
Populations with well-defined person-time for which most medically-attended events are known
193 million members*
351 million person-years of observation time
39 million people currently accruing new data
4.8 billion dispensings
5.5 billion unique encounters
33 million people with >1 laboratory test result
*as of August 2015

10. Sentinel Key Features Active Surveillance Distributed Database:
Common Data Model
Data partners maintaining all data
Standardized SAS program executed at Data Partners
Privacy: Aggregated data and summary results are provided to FDA. No ID or individual level data is shared across data partners and with FDA.
Transparency: FDA releases findings, study protocol, reports, and SAS programs to the public.

11. Rapid Query Tools Modular Programs Level 1 Level 2 Level 3
PSM
Propensity Score Matching
GEE
Generalized Estimating Equations
IPTW
Inverse Probability of Treatment Weighting Regression
Binomial maxSPRT
Maximized Sequential Probability Ratio Testing
CIDA
Cohort Identification and Descriptive Analysis
MP4
Concomitant exposure characterization
MP8
Uptake, use, persistence of new molecular entities
MP7
Frequency of codes before/after index date
Cohort Identification
and Descriptive Analysis Tools
Analytic Adjustment Tools
Sequential Analysis
and Signaling Tools
Group Sequential
GEE Signaling
IPTW Signaling
Level 1
Level 2
Level 3
SCRI
Self-Controlled Risk Interval
Unadjusted Analysis
Adjusted Analysis

12. Example: dabigatran and bleeding events

13. Example
Dabigatran is an oral anticoagulant approved in 2010 to reduce the risk of stroke in patients with atrial fibrillation.
Dabigatran 150 BID

14. Dabigatran Example Timeline for investigations and regulatory actions -1
Reports
Of bleeding (FAERS and literature)
Mini-Sentinel
Modular Program level 1 investigation
(10/2012)
Approval of Dabigatran
(10/2010)
Drug Safety Communication (12/2011) and change of labeling (01/2012)
Drug Safety Communication (11/2012) and NEJM publication

15. Dabigatran Example Modular Program Level 1 (10/2012) Descriptive, unadjusted analysis
Source: N Engl J Med 2013; 368:

16. Regulatory Action

17. Dabigatran Example (continued)
FDA led CMS
Protocol based
Investigation
Final statistical Analysis plan (06/2013)
Mini-sentinel
Protocol Based Assessment
(03/2014)
Drug Safety Communication based on findings in CMS Study
(05/2014)
NISS Spring Affliates Meeting, March 15th 2015

18. CMS Protocol Based Assessment
Design Elements
Study Design
New user cohort, retrospective
Data Source
Medicare Part A (inpatient hospital), Part B (outpatients Medical care) and Part D (prescription drugs)
Study Period
10/19/ /31/2012
Study Population
Patients 65+ filled ≥ 1 study drug (dabigatran or warfain) in the study period.
Inclusion/Exclusion Criteria
In look back period 183 days
Includes: atrial fibrillation or flutter
Excludes: dialysis, kidney transplant, VTE, joint replacement, valvular disease, less than 6 months enrollment in Medicare prior to index date
Washout Period
no oral anticoagulants use within 183 days before taking study drug
Primary Outcome
Stroke, major bleeds (gastrointestinal, intracranial), acute myocardial infarction (AMI), and mortality
Graham D et al, Cardiovascular, bleeding, and mortality risk in elderly Medicare patients treated with dabigatgran or warfarin for non-valvular atrial fibrillation Circulation, published online Oct 2014

19. CMS Protocol Based Assessment
Analysis Elements
Propensity Score Analysis
1:1 greedy match with a caliper of 0.2 times the standard deviations of the logit of the propensity score
Confounders: cardiovascular risk factors, bleeding risk factors, medical conditions, health care utilizations, prescriber characteristics, CHADS and HAS-BLED scores
Outcome Analysis
Cox proportional hazard regression
Censoring Criteria
Follow-up: “As treated” (examining outcomes while patients remain exposed to study drug allowing up to 3-day gaps)
Switching study drug, therapy gap, admission to nursing facility/home or transfer to hospice care, start of dialysis or receipt of a kidney transplant, disenrollment from Medicare, study end
Subgroup Analysis (pre-specified)
Sex, age, dose (150mg vs. 75mg twice daily)

20. CMS Protocol Based Assessment
Results
Before matching Ndab =67,494 Nwar=273,920
After matching Ndab = Nwar=67,207

21. Regulatory Action

22. MiniSentinel Protocol Based Assessment
Similar design elements as CMS Protocol Based Assessment (New user cohort, Inclusion/exclusion criteria, Censoring criteria, Confounders, Outcomes)
Data Source: 8 Sentinel Data Partners
Study Period: November 1, May 14, 2014
1:1 Propensity score matching by Data Partner to control for confounding
Primary analysis is time to event using Cox regression stratified by Data Partner

23. Dabigatran Example (continued)
Mini-sentinel Protocol Based Assessment
-manuscript submitted to peer-review journal, under revision (07/2016)
- report is under FDA review (08/2016), going to be posted publicly
FDA led CMS
Protocol based
Investigation
Final statistical Analysis plan (06/2013)
Mini-sentinel
Protocol Based Assessment
(03/2014)
Drug Safety Communication based on findings in CMS Study
(05/2014)
NISS Spring Affliates Meeting, March 15th 2015

24. Integrating sentinel into regulatory programs

25. Integrating Sentinel into regulatory programs at CDER
ARIA (Active Postmarket Risk Identification and Analysis), a subcomponent of Sentinel and an expansion use of the Modular Programs
CDER is working on integrating Sentinel ARIA into the regulatory post-marketing safety review
CBER has fully integrated Sentinel PRISM(Postlicensure Rapid Immunization Safety Monitoring Program ) into its regulatory review process of vaccines.

26. ARIA is Comprised of Modular Programs
Level 3 Sequential Adjusted Analyses with Sophisticated Confounding Control
Level 2 Adjusted Analyses with Sophisticated Confounding Control
(Propensity Score Matching)
Level 1 Descriptive Analyses, Unadjusted Rates
Modular Programs Currently in ARIA
Future ARIA Capabilities
Adapted from Michael Nguyen’s slide in September 2015

27. Limitations and Challenges
Primarily claims data, more electronic medical records and lab data to be included
Ascertainment of exposures and outcomes
Statistical Challenges
Modeling rare outcomes
Unmeasured confounding bias
Confounding bias by indication
Sequential analysis

28. Summary
Sentinel is an active surveillance system complementing existing post-marketing safety assessment tools at FDA
CDER is working on integrating Sentinel ARIA into the regulatory post-marketing safety review
Together with other benefit and risk information, Sentinel enhances FDA’s understanding of safety issue and inform regulatory decisions

29. Acknowledegments Rima Izem (CDER\DB7) Mark Levenson (CDER\DB7)
Michael Nguyen (CDER\OSE)
Marsha Reichman (CDER\OSE)
David Graham (CDER\OSE)

30. Thank you and Questions?

31. Back up

32. Marketing Application and Review of Benefits and Risks
Safety in (New) Drug Development
Marketing Application and Review of Benefits and Risks
Basic Research
Discovery
Preclinical
Randomized Clinical Trials
Post-Market Assessments
Drug Reasonably Safe for use in Humans
Div. of Biometrics Div. of Biometrics 1-5 (efficacy) Div. of Biometrics 7
Div. of Biometrics 7 (safety)
Safety Profile of The Drug
Safety Surveillance
NISS Spring Affliates Meeting, March 15th 2015 32

33. Sentinel Data Sources Administrative data (claims data) EHR data
Enrollment
Demographics
Outpatient pharmacy dispensing
Utilization (encounters, diagnoses, procedures)
EHR data
Height, weight, blood pressure, temperature
Laboratory test results
Registries
Immunization
Birth certificates

34. Rivaroxaban Example Protocol Based Assessment
Adjusted, Sequential Analyses
New user cohort study
Variable ratio propensity score matching by Data Partner to control for confounding
Sequential looks with Pocock stopping boundary
Primary analysis is time to event using Cox regression stratified by Data Partner
NISS Spring Affliates Meeting, March 15th 2015 34

35. Post Marketing Requirement Criteria with ARIA
New Safety Information*
Not Evaluable in FAERS & ARIA
Study Purpose
PMR
Study purpose is one of the following:
To assess a known serious risk related to the use of the drug
To assess signals of serious risk related to the use of the drug
To identify an unexpected serious risk when available data indicate the potential for a serious risk
* Can occur in both pre-approval and post-approval settings
Adapted from Michael Nguyen’s slide in September 2015