1. Use of Registries in Product Safety Assessment Kathryn Starzyk Outcome Sciences

2. AuthorsReviewers Kathryn Starzyk (lead)Jeff Brown Outcome Harvard Medical School Nancy DreyerLeslye Fitterman Outcome Centers for Medicare & Medicaid Services Stella BlackburnJon Merz European Medicines Agency (EMEA) University of Pennsylvania School of Medicine Richard Platt Harvard Medical School Ken Rothman RTI International Authors and Reviewers 1

3. To evaluate the utility of registries and other prospective observational studies in identifying potential safety signals through adverse event (AE) collection – Note: signal detection through existing datasets (e.g., administrative claims data), and the specific technical and statistical methodologies considerations involved, are addressed in detail elsewhere and beyond the scope of this paper Purpose 2

4. Introduction Registries specifically designed for safety assessment – Design considerations – Use of healthcare provider and patient reported outcomes – Effects observed in larger population over time – Defining exposure and risk windows – Special Conditions: Pregnancy Registries Orphan Drugs Controlled Distribution/PLAS Medical Devices Paper Overview 3

5. Registries designed for purposes other than safety Signal detection in registries and observational studies Potential obligations for registry developers in reporting safety issues Paper Overview (cont.) 4

6. Post-approval is important phase for identifying and understanding product safety concerns associated with both acute and chronic use – RCTs are by necessity of limited duration, size and population representativeness Observational and inclusive design may allow for surveillance of a diverse patient population – Spontaneous reporting captures an extremely small percentage of the actual events occurring Some evidence that physicians who report adverse events are not typical of physicians in general Rationale 5

7. Safety signal: – Suggests a new possibly causal association between an intervention and event – Warrants further evaluation Typically assessed based upon: – Strength of the association between exposure and the event – Biological plausibility – Any dechallenge/rechallenge evidence – Any experimental or animal models, and – The nature, consistency and quality of the data source. Challenge, as always, is to separate a potential safety signal from the “noise” Signal Detection 6

8. Product Registries Disease Registries – Provide large, systematic data collection for target populations of interest – Can provide background rate of events of interest – May be more likely to be successful in collecting unbiased, interpretable long-term safety data Considerations: Inclusion/exclusion criteria Internal and external comparator groups Analysis planning incorporating possible sources of bias Staff training and availability of systematic follow-up Disease vs. Product Registries 7

9. Registries well suited to identify effects only observed in large and diverse population over an extended period of time To ensure validity of the inferences made, when designing a registry for the purposes of safety consider: – Size of the registry – Characteristics of the enrolled population – Duration of follow-up Describing safety outcomes from observational studies in statistical terms is not always straightforward – How is the “expected” rate calculated? – “Rule of three” sometimes helpful as a means for “estimating the worse case that is compatible with the observed data” Effects Observed in Larger Populations over Time 8

10. Recruiting a meaningful patient population Evaluating the utility of a registry when the entire population-at-risk has not been included – Assess whether patients in the registry are representative of typical patients and what types of patients may be systematically excluded or not enrolled in the registry General Challenges 9

11. Important to collect information on the timing of events (and treatment) in relation diagnosis Use of time windows Other aspects of product use in the real world: Switching therapies and regimens during follow-up Use of multiple products in combination or in sequence Dose effects Delayed effects Patient compliance (i.e., adherence, persistence) Defining Exposure and Risk Windows 10

12. Prospective data collection remains the best source of meaningful safety data related to pregnancy – Pregnant women are usually not included in clinical trial populations – Computerized claims or billing data for pregnancy safety monitoring hampered by: Patients often not presenting early in pregnancy Lack of relevant data on other exposures (since these are often unrelated to reimbursement) Difficulty linking maternal and infant records Special Conditions: Pregnancy Registries 11

13. Safety profile at authorization extremely limited (due to small patient populations) and post-authorization monitoring is likely required Diseases are often fatal, so natural history when treated is unknown – Difficult to disentangle the effects of the product from those of the ongoing disease Special Conditions: Orphan Drugs 12

14. Registries may be part of REMS/controlled distribution in the U.S. – All patients treated in registry - cannot otherwise obtain access to the treatment – Looking for a known AE or risk factor, collecting data specifically related – Primary focus is usually prescribing controls (not safety), data elements are usually minimal, which can limit their utility for analysis In Europe, differing national legislation makes controlled distribution more difficult - Registries are more often voluntary Special Conditions: Controlled Distribution 13

15. Safety monitoring important because of widespread and long-term use Challenges: – Identifying appropriate practitioners and obtaining medical records for devices not covered by insurance – Studying devices that are easily attached/detached by the patient (i.e. hearing aids) Special Conditions: Medical Devices 14

16. Comparative effectiveness Natural disease history Data collection in support of national coverage decisions or quality improvement efforts May gather and report AE data, but may not be able to reliably detect all events – Must collect a wide range of data from patients to try to catch any possible events May allow ad hoc data pooling Registries Designed for Purposes other than Safety 15

17. Key questions: – What is the role of registries not designed for safety purposes to look for AEs? – What are the obligations to further investigate and report these findings? Registries designed for safety assessment should have a plan in place for detecting the signal of interest, how to handle unanticipated events and what and how any such events will be reported to appropriate authorities An incidental findings plan may be requested by IRB prior to approval All registries should address reporting mechanisms for AE information Potential Obligations in Reporting Safety Issues 16

18. Registries are likely to make strongest contributions through: – Detection of novel AEs (as reported by treating physicians) that constitute a signal necessitating further study – Establishment of registries of pregnant women and other hard to study subpopulations of product users – Linkage of registries with additional data sources such as the Medicare-SEER data linkage Broadening the range of questions that can be addressed beyond the constraints of data collected for a registry – Confirming or validating signals generated in other data, such as from automated signal generation in large claims databases Conclusion 17

19. 18 THANK YOU!