1. By William Wang On Behalf of the ASA Biopharm Safety Working Group
Practical considerations in planning and implementing aggregate safety monitoring
By William Wang
On Behalf of the ASA Biopharm Safety Working Group

2. Agenda Introduction of the ASA Safety Working Group
From the Program Safety Analysis Plan (PSAP) to the Aggregate Safety Assessment Plan (ASAP)
Practical Considerations
Concluding Remarks

3. Safety Working Group: Formation and Mission
The ASA Biopharmaceutical Safety Working Group was initially created in 2014.
It started with a working group on “Safety Strategies and Analysis” in 2014
Safety Monitoring Working Group was created in the fall of 2015, with mission “to help empower the biostatistics community to better enable qualitative and quantitative safety monitoring”
Founding co-chairs were Dr Olga Manchenko and Dr Qi Jiang, current co-chairs are Judy Li and William Wang (effective Oct 2017)

4. ASA Safety Working Group
Co-chairs: William Wang (Merck), Judy Li (Regeneron),
Mission: Empower the biostatistics community and interdisciplinary safety management partnerships to better enable qualitative and quantitative safety evaluation throughout drug development life cycle
Workstream 1a: Regulatory context, industry survey, planning framework
Review paper on the global regulatory landscape and underlying quantitative principles
Aggregate Safety Assessment Planning (ASAP) Points to Consider
Workstream 1b: Safety Evaluation and Identification of Risk
Clarify signal detection and evaluation for clinical development safety monitoring (new taskforce)
WS1: Interdisciplinary Safety Evaluation
(Joint with DIA )
Bayesian vs Frequentist Approach
Blinded vs Unblinded Analyses
Static and dynamic methods
Visual analytics
Meta-analysis
Safety Enabled Benefit Risk Evaluation (new task force)
WS2: Safety Monitoring Statistical Methodology
Statistical and Design Consideration for RWE
Statistical and Design Consideration for RCT
Advanced Analytical and Machine Learning Methodologies for Qualitative or Quantitative Integration of the Multi-Source Safety Data
WS3: Integration and Bridging the RWE and RCT for Safety Decision Making

5. WS1: ASA-DIA Collaboration on Interdisciplinary & Multi-Regional Safety Evaluation
Activities:
Across community collaborations: ASA/ DIA
Interdisciplinary collaborations between statisticians and clinicians : Fun club presentations
Multi-regional collaborations with European colleagues: brought balance between the FDA and the EMA to our paper and have clarified the regulatory landscape (FDA has placed the responsibility on sponsors to judge reasonable possibility and EMA wants to be more involved in the assessments)
2018 Q1-Q2 Deliveries:
World Drug Safety Congress April 9
Introduction of the joint working group and deliverables (including the regulatory landscape and ASAP proposal) to safety community
Industry Survey Paper (accepted by the DIA journal)
Demonstrates how the industry survey can be used to map an action plan for meeting the demand for improved quantitative safety monitoring
10 new members enrolled into our Fan Club to beta-test WS1 deliverables (7 safety physicians and 3 statisticians)

6. WS2:Statistical Methodology Deep Dive
Activities:
Continue deep diving in statistical methodology for safety evaluation
“Safety Enabled Benefit Risk” & “Safety Estimand” initiative
2018 Q1-Q2 Deliveries:
DIA webinar April 19
safety monitoring in aggregation: interdisciplinary collaboration and methodology enablement
MBSW  May 14-16, 2018
Session: Bayesian Monitoring and Benefit-Risk Assessment of Safety Data - Presentations:
Presentation: Programming and Visualization of Safety Data in the World of Big Data: NLP and ML
Round Table Discussion: From Safety Statistics to Safety Science and Drug Safety Analytics
IISA   May , 2018
Session: Safety Monitoring in Clinical trials  
ICSA  June 14-17, 2018
Session: Reasonable Possibility and Statistical Science in Drug Safety Monitoring (benefit risk )
DIA Webinar June 21, 2018
Getting the question right: benefit risk evaluation

7. WS3: RCT, RWE and Advanced Analytics.
Activities:
Charted the team
Identified 3 areas of exploration: RCT, RWE and advanced analytics. Bi-weekly meetings started in March 2018 with regulatory background and case studies presentations shared by team members
Next goal:
Identify and clarify specific areas of WG focus and some short term and long term deliverables by July/August

8. Agenda Introduction of the ASA Safety Working Group
From the Program Safety Analysis Plan (PSAP) to the Aggregate Safety Analysis Plan (ASAP)
Practical Consideration
Concluding Remarks

9. Program Safety Analysis Plan (PSAP)
Safety Planning, Evaluation, and Reporting Team (SPERT) is formed by the PhRMA in to recommend a standard for safety planning, data collection, evaluation, and reporting.
In their key 2009 publication* , they recommend:
Multi-disciplinary Safety Management Teams (SMT), referencing CIOMS VI
Program Safety Analysis Plan (PSAP): A proactive approach for sponsors to plan and review aggregated safety data on a regular and ongoing basis throughout the development program, rather than waiting until the time of submission
* Crowe et al: Recommendations for safety planning, data collection, evaluation, and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team. Clinical Trials 2009; 6:

10. Aggregate Safety Assessment Planning (ASAP)
A process to enable transparent and interdisciplinary collaboration in safety evaluation
A process to accumulate safety knowledge from drug development life cycle (eg pre-clinical, early clinical, late development and post-marketing)
A process to promote proactive planning and execution of safety monitoring
A process to prepare the safety filing

11. Key Components of the ASAP
Characterization of the emerging safety profile of the product (“safety story”)
Ensure a consistent approach to collection and analysis of the safety data across the program
Description of ongoing aggregate safety signal detection and evaluation methods
Preparation for regulatory application filing and periodic reporting

12. ASAP as a integrated part of the Drug Development Life Cycle
Phase 1-2a
Phase 2a
Phase 2b
Early Ph 3
Ph 3; Prep /Completion of Submission Deliverables
Post-marketing
Benefit-Risk Strategic Thinking and Planning
Emerging Safety Profile
Safety Governance / Regulatory Reporting
Master ASAP
Program-level definitions
AESIs defined
Program-level aggregation conventions
Background AE rates
Update Master ASAP
ISS SAP
ISS, CSS/CO
EuRMP
Safety in Label (USPI, SmPc)
PBRER
Ad hoc regulatory queries
Ties to Real World Data sources
Consider:
Target Product Profile
Benefit Risk Value Tree
Preclincal Data
MOA/Related product information
+Regulatory feedback
Modular Approach to Building the Safety Story
Benefit-Risk Thinking and Planning
Begins early (e.g. Phase 1/2a), builds over time
Safety Governance/Regulatory Reporting
Aggregate safety reports
Periodic Safety Update Reports
Regulatory requests
Emerging Safety Profile
Reference Safety Information / Label
Important Risks / RMP
Risk Mitigation
AESI Adverse events of special interest
BR Benefit-Risk
CTD Common Technical Document
DSCI Development Safety Core Information (internal safety doc)
dRMP Development Risk Management Plan (internal safety doc)
DSUR Development Safety Update Report
EuRMP Risk Management Plan (for EMA)
IB Investigator Brochure
IND Investigational New Drug (for FDA)
ISS Integrated Summary of Safety (for FDA)
PBRER Periodic Benefit Risk Evaluation Report
RSI Reference Safety Information
SAP Statistical Analysis Plan
SmPc Summary of Product Characteristics (EMA)
USPI Package Insert (fFDA)
RSI, DCSI, DSUR, dRMP
Monitoring Ongoing Trials
Safety monitoring questions
Output displays
Analytical methods
+ New Information from Clinical Trials or External sources
Ph 2 Study SAPs
Safety aspects

13. Agenda Introduction of the ASA Safety Working Group
From the Program Safety Analysis Plan (PSAP) to the Aggregate Safety Analysis Plan (ASAP)
Practical Considerations
Concluding Remarks

14. Practical Considerations of ASAP Implementation
Getting the partnership right (GTPR)
Getting the question right (GTQR)
Getting the estimands right (GTER)
Getting the data right (GTDR)
Getting the monitoring right (GTMR)
Getting the story right (GTSR)
Getting the tools right (GTTR) ….

15. Practical Considerations of ASAP Implementation: Getting the partnership right (GTPR)
Charter the right team: Statistician(s), Safety Scientist(s), Clinical Scientist(s), Epidemiologist ….
Leverage each function’s unique perspective
Integrate preclinical and early clinical knowledge
Reach shared understanding of safety profile

16. Practical Considerations of ASAP Implementation: Getting the question right (GTQR)
Signal detection vs signal validation?
To rule out harm vs to claim benefit?

17. Practical Considerations of ASAP Implementation: Getting the estimand right (GTER)
Treatment policy
Account for all safety events during the follow-up regardless of intercurrent events, such as treatment discontinuation or rescue med.
Composite
Combine the intercurrent event into a composite safety endpoint,
e.g., subject discontinues due to AE as a component of the composite safety event
Hypothetical
Would the subject develop the adverse events if he/she did not have the intercurrent event.
Principal stratum
Defines subset of patients in whom the intercurrent event occurs in one or both treatments. Causal inference methods on safety evaluation are required.
While on treatment
Only accounts for adverse events prior to the occurrence of an intercurrent event, e.g. before rescue meds.

18. Practical Considerations of ASAP Implementation: Getting the estimand right (cont)
AE Metric
Type of Measure
Features
Crude rate
Count
Short exposure time
Fixed f-u time
Risk is relatively constant
Exposure-adjusted incidence rate
Counts per total person-time at risk
Individual f-u time
Time is continuous
3. Kaplan-Meier
Cumulative
rate
Counts accumulating over time intervals
of exposure
Risk can vary over time
4. Hazard ratio
Ratio of two hazard rates
Covariates
5. Restricted
mean survival
time
Area under the survival curve
Average time to failure up to the specified time
Well defined irrespective of risk pattern over time

19. Practical Considerations of ASAP Implementation: Getting the data right (GTDR)
Quality-by-design (ICH E6 and ICH E8)
Ensure key safety information are built into the design and data collection
Ensure consistent adjudication, coding and risk-based monitoring

20. Practical Considerations of ASAP Implementation: Getting the monitoring right (GTMR)
Fit for purpose: Blinded vs unblinded aggregate monitoring
Proper pooling strategy:
Across studies
Across indications
Across stages (time dimensions)
Proper right tools and statistical methodologies
Meta Analytical approach vs Simpson’s paradox
Bayesian and likelihood principles
Integrating vs bridging RCT and RWE
Visual analytical solutions

21. Practical Considerations of ASAP Implementation: Getting the story right (GTSR)
ASAP to drive the safety preparation for the core company data sheet (CCDS)
Use the ASAP and CCDS to build ISS, RMP, CTD (2.7.4) and SUR

22. Practical Considerations of ASAP Implementation: Getting the leadership right (GTLR)
Enhance our skills not only in dealing with uncertainty , but equally important, in dealing with ambiguity
Enhance our holistic thinking and decision making skills
Enhance our culture intelligence
Choose to lead …