1. PhUSE Standard Analyses and Code Sharing Working Group
All Hands Working Group Meeting
29 November 2018

2. PhUSE Working Groups Standard Analyses and Code Sharing Working Group
Leads: Hanming Tu, Mary Nilsson
From the PhUSE Working Groups home page – The Standard Analyses and Code Sharing WG is one of 6 WGs as part of the FDA/PhUSE collaboration.

3. SACS Working Group Vision
Vision: Leverage crowd-sourcing to improve the content and implementation of analyses for medical research, leading to better data interpretations and increased efficiency in the clinical development and review processes
SACS = Standard Analyses and Code Sharing

4. SACS Working Group Goals
Establish and maintain a publicly available repository for storing program code to be used as analytical tools for medical research
Where gaps exist, develop recommendations for analyses and displays in areas that could benefit from crowd-sourcing
Where gaps exist, develop code for recommended analyses and displays that could benefit from crowd-sourcing (to reside in the repository)
Note: In early 2019, we should decide whether to add goals to cover our additional project teams

5. Current Projects Analysis and Display White Papers (ADW)
Lead: Mary Nilsson
Best Practices for Quality Control and Validation
Leads: Maria Dalton and Jane Marrer
New
Code Sharing (Repository)
Lead: Hanming Tu
Good Programming Practices
Lead: Ninan Luke (Macro Development)
New
Communication, Promotion, Education (CPE)
Leads: Wendy Dobson, Jared Slain
Test Data Factory
Lead: Peter Schaefer
Note: We’re still working out the details around the Good Programming Practices project team. We will likely have more sub-projects.
Note: Most meetings occur at the project team and sub-project team level. We rarely meet as a full Working Group. We do have

6. ADW Project Team Lead: Mary Nilsson FDA Liaison: Nhi Beasley
Description: This project includes the development of white papers that provide recommended Tables, Figures, and Listings for clinical trial study reports and submission documents. The intent is to begin the process of developing industry standards with respect to analysis and reporting for measurements that are common across clinical trials and across therapeutic areas.
ADW = Analysis and Display White Papers

7. ADW Project Team Goals
Define a set of recommended tables, figures, listings and associated analyses
Ensure reviewers receive clinically relevant and meaningful analyses for benefit-risk assessment
Harmonization leads to efficiencies in both creation and review of analyses
Improve expertise in safety analytics across multiple disciplines involved with planning, interpreting, and report safety analyses
Promote good analytical practices, avoid poor practices
Facilitate cross-functional engagement in analytical planning
ADW = Analysis and Display White Papers
Excerpt from the Adverse Events White Paper:
The development of standard TFLs and associated analyses will lead to improved standardization of AE data from collection through data display and analysis. The development of standard TFLs will also lead to improved and harmonized product lifecycle management across therapeutic areas by ensuring that reviewers receive clinically relevant and meaningful analyses of patient safety for benefit-risk assessment. This white paper reflects recommendations that would lead to more consistent TFLs, but the recommendations should not be interpreted as “required” by any regulatory agency.
Another purpose of this white paper (along with the other white papers from the project team) is to improve expertise in safety analytics across the multiple disciplines involved with planning, interpreting, and reporting safety analyses. Statisticians can and should assist cross-disciplinary teams to understand and reduce bias in analysis planning and reporting. This assistance is important even when inferential statistics are not used. The potential for biased comparisons is especially a concern when multiple studies are combined (eg, via poor pooling practices for integrated summaries). Safety physicians have relied on qualitative analyses of case reports, looking at individual or small clusters of events. Recently, there has been an increased emphasis on aggregate reviews of safety data. As noted in Section VI of the CIOMS Working Group VI report [1], while medical judgment remains critical in the interpretation of safety data, descriptive and inferential statistical methods can help medical personnel decide whether chance variation is a possible explanation for what is observed or whether it is more likely that some genuine drug effect has occurred. This requires statisticians to increase their engagement and help cross-disciplinary safety management teams to think more quantitatively, This also requires nonstatistical disciplines to obtain a higher level of analytical expertise. For example, all members of a safety management team should understand that improper pooling can lead to biased and even paradoxical results. See Table 10.1 for an illustrative example of paradoxical results and see Section 10.1 for other areas where increased expertise is needed.

8. Vision: Fill the Gap on Analysis and Display Standards
Data Collection Systems
Observed Datasets
Analysis Datasets
Tables, Figures and Listings
Clinical Data
Flow
Trial Design
PRM
SDTM
ADaM
No Standards Exist
Industry
Standards
Alignment
CDASH
A lot of progress has been made with respect to standardization – mostly in the collection and data space. There’s a gap with respect to analyses and displays. CFAST = Coalition for Accelerating Standards and Therapies
(Therapeutic Areas)

9. ADW – Final Deliverables
All final white papers from the project team are in the Working Group Deliverables Catalog
White Papers section
ADW = Analysis and Display White Papers

10. ADW - Timeline for Future Deliverables
ADW Project Team – Timeline for Future Deliverables

11. ADW Project Team – Get Involved
Co-author a white paper
Medical and statistical participants
Approximately 8 hours a month for 6 months
Participate in reviews
Medical and statistical reviewers across business units
Approximately 2 hours
Help recruit reviewers
Help with action items (follow-up tasks) where relevant
Communicate the availability of the white papers
Promote following the recommendations in the white papers
If you receive push-back from internal colleagues or from a regulator, let us know
Reference white papers in Statistical Analysis Plans
ADW = Analysis and Display White Papers

12. Code Sharing (Repository) Project Team
Lead: Hanming Tu FDA Liaison: Mat Soukup
Description: Establish and maintain a collaboration platform for leveraging crowd-sourcing to improve the content and implementation of analyses for medical research and leading to better data interpretations and increased efficiency in the clinical drug development and review processes.

13. Code Sharing Project Goals
Establish and maintain a publicly available repository for storing program codes to be used as analytical tools for medical research
Develop guidance on managing repositories and script metadata for sharing scripts
Improve frontend for accessing and searching the repository
Review contributed scripts such as FDA scripts
Develop scripts based on white papers

14. Vision: Open Tool Box Shared tools Text Shared Value
Code Sharing (Repository) Project Team

15. Vision: Script Repository
FDA
Industry
Academia
Script Repository in Github
Shared tools
Script Metadata
Test Data
SAS, R, Spotfire, Etc.
White Paper Scripts
Contributed Scripts
Code Test Environment
Code Sharing (Repository) Project Team

16. Code Sharing – Final Deliverables
Script Repository in Github created
Qualification guidelines created
Scripts from other groups have been contributed
FDA, Non-clinical, Data handling, Spotfire safety templates
Scripts from within the project have been created
Tables and figures from the 2013 labs/vitals/ECGs white paper
White paper showing the displays created from the FDA-contributed scripts
FDA:
Non-clinical:
Data Handle:
Spotfire Templates:

17. Code Sharing – Timeline for Future Deliverables
Script Metadata white paper – Currently under public review
The review period closes 13th December
More code creation planned for 2019
Will likely focus on tables and figures that are recommended in a white paper but not part of predominant practice today

18. Code Sharing – Get Involved
Participate in the public review of the Script Metadata white paper
Go to phuse.eu, click on the Resources tab, click on Working Group Deliverables, click on Deliverables Under Review
Share existing code
Create code
Use code from the repository
Help identify areas where crowd-sourcing the creation of a shared tool makes sense
Interactive safety review?
Reduce some common errors in e-submission?

19. Communication, Promotion, Education (CPE) Project Team
Leads: Wendy Dobson, Jared Slain FDA Liaison: Mat Soukup, Susan Duke
Description: The success of working group “Standard Analyses and Code Sharing” relies on the acceptance, input, feedback and further development from/by the user community. A main priority will be to develop a Communication Plan that conceptualizes efficient ways to communicate working group progress and results, e.g. white papers, and the call for scripts. It will define target groups, timing, communication channels, and the presentation.

20. CPE Project Goals Facilitate communication of deliverables
Improve navigation to information
Facilitate presentations/posters at multiple conferences
Keep wiki up-to-date
Facilitate creation of educational video clips
Study-size adjusted percentages completed
Help define places to mention deliverables
Write journal article(s)
Create a Safety Analytics site
Host a Safety Analytics workshop
Crowd-source educational library

21. CPE Project – Final Deliverables
Multiple presentations at various conferences
Educational video on study-size adjusted percentages

22. CPE Project – Timeline for Future Deliverables
Safety Analytics Educational Workshop – CSS 2019
DIA presentation – June 2019
Journal article – targeting end of 2019
Additional educational video clips – throughout 2019 and beyond
Safety Analytics education library - Create front-end in early 2019

23. CPE Project – Get Involved
Help with communication
Help with improving navigation
Volunteer to be a liaison with other groups
Present at professional meetings where PhUSE work can fit
Even a couple of slides would help!
Help with crowd-sourcing educational information
Primarily interested in safety analysis educational information
Share existing educational material
Help create missing material

24. Best Practices for Quality Control and Validation Project Team
New
Leads: Maria Dalton and Jane Marrer FDA Liaison: TBD
Description: Quality and accuracy is essential in the health and life sciences industries. Patients and regulators must be able to trust analyses of clinical data. A white paper will be created to provide an in-depth review of best practices for robust quality control. The scope of the paper will be quality control of analysis programming of clinical data in health and life sciences organisations. It is applicable to the organisations who produce analyses of clinical data, including Contract Research Organisations. The paper will not discuss oversight of outsourced programming.

25. Quality Control and Validation – Get Involved
Join the project team
Co-author sections of the white paper
Review the white paper
Recruit project team members/reviewers

26. Good Programming Practices (GPP) Project Team
New
Lead: Ninan Luke (Macro Development) FDA Liaison: TBD
Description: Macros provide an effective way to automate and reuse code in a standard and consistent manner across SAS programs. This ability to reuse code means that the use of GPP is particularly important in macro code and we think that there is a need to develop a consensus and document good programming prairies specifically for macro programming.
Note: We will likely have more GPP-related sub-projects. Macro development will likely become a sub-project within a broader project team.

27. Good Programming Practices – Get Involved
Join the project team
Co-author deliverable
Review deliverable
Recruit project team members/reviewers
Propose a sub-project related to GPP

28. Test Data Factory (TDF) Project Team
Lead: Peter Schaefer CDISC Liaison: TBD
Description: Several CS Projects develop and specify medical research methods, features, or processes, and some even create software components or subsystems for common tasks in drug development. As part of these efforts, a variety of SDTM or ADaM test datasets are required. The typical fallback position of project teams is to use data from the CDISC pilot project and/or anonymized study data that are provided by project team members. The Test Data Factory project aims at providing test data formatted in SDTM and ADaM that support a more systematic and comprehensive testing of these concepts and scripts.
Note: We will likely have more GPP-related sub-projects. Macro development will likely become a sub-project within a broader project team.

29. Test Data Factory – Deliverables
Initial Approach:
Start with existing CDISC pilot datasets, and update them to comply with more up-to-date SDTM standards
Reasoning: Gain experience, provide some results sooner rather than later
Completed update of SDTM and ADaM CDISC Pilot datasets
The datasets and related documentation are in final review. Should be released before the end of the year
Next step:
Implement scripts/applications that generate Test Data based on user input for specific testing tasks.

30. Test Data Factory – Get Involved
Join the project team
Review SDTM and ADaM test data packages when out for public review
Recruit project team members/reviewers
Volunteer to help with the next phase of the project – simulated test datasets

31. How to Participate Sign up for the PhUSE working group mailings
From phusewiki.org, click “Join a Working Group Now”
Standard Scripts Groups
CSS-WG-Standard-Scripts (Entire Working Group)
CSS-WG-Standard-Scripts-WhitePapers (White Paper Project Team)
CSS-WG-Standard-Scripts-Platform (Script Repository)
CSS-WG-SS-WhitePaperReviewers (Notified when a white paper is ready for review)
See wiki pages for all projects (
Contact any of the WG or project leads
Note: We’ll likely need additional group mailing lists for the new projects