1. ClinicalTrials.gov: A Primer on Results Reporting 1 Christina McCarthy, MS Senior Compliance and Regulatory Affairs Specialist Boston Children’s Hospital christina.mccarthy@childrens.harvard.edu January 14, 2015

2. Outline Introduction – Registration Requirement Re-cap – Results Reporting Requirement – Preparing for Results Reporting The Four Results Modules – Participant Flow – Baseline Characteristics – Outcome Measures – Adverse Events ClinicalTrials.gov QA Review Process 2 A copy of this presentation is available on the BCH intranet: Research > Translational Research > Regulatory Resources

3. Registration Requirement Re-cap 3

4. FDAAA Results Reporting Requirement Required for: FDAAA “Applicable Clinical Trials” that study drugs, biologics, and devices that are approved, licensed, or cleared by FDA (for any use) When: Within 12 months of Primary Completion Date (the final data collection for the primary endpoint) If product not approved by Primary Completion Date but is approved later, then results due 30 days after approval Delays are possible, primarily for manufacturer or under limited special circumstances o Pending publication is NOT considered a good cause for delay http://clinicaltrials.gov/ct2/manage-recs/fdaaa 4

5. Brief interruption to mention changes are coming… 5 Approved or Unapproved

6. An Overview of Results Reporting The process of submitting results information to ClinicalTrials.gov is conceptually similar to preparing a manuscript for publication in a journal. An individual familiar with the study design and data analysis (such as the clinical investigator or study statistician) will need to be involved in order to accurately summarize the results information in the tabular format required by law and to ensure that the results are consistent with the ClinicalTrials.gov review criteria. 6

7. An Overview of Results Reporting Scientific information is submitted as 4 separate modules: 1.Participant Flow 2.Baseline Characteristics 3.Outcome Measures 4.Statistical Analyses and Adverse Events 7

8. How are Results Reported? Tables are constructed by data providers – “Stand alone” tables - must be meaningful to people who are not already familiar with the study. – No narrative conclusions – Columns are study arms – Rows are measures – Type of measure determines specific design of cells 8

9. Quick Reference: Steps for Submitting Results Follow these steps for submitting results:steps for submitting results 1.Learn about the requirements for submitting results. 2.Login to the Protocol Registration and Results System (PRS). 3.Update the Protocol Section and release (submit) the record. 4.Enter the required (and optional) results data elements. – For basic help with using PRS, review the Quick Start Guide found in the Help section of the PRS main menu. – See the Basic Results Data Element Definitions for descriptions of each required data item.Basic Results Data Element Definitions – See the results data preparation checklists and simple results templates for each module, listed in the Scientific Information section above, for the data you will need and a view of the data elements in a tabular format.Scientific Information – See the Helpful Hints (PDF) for tips on entering results data, including three examples of common study models (parallel design, crossover design, and diagnostic accuracy studies) and measure types.Helpful Hints 5.Preview, inspect, and release (submit) the record. 9

10. Preparing for Results Reporting Accurate registration entry is essential for results reporting – Be sure your registration is as accurate and complete before entering results. – Registration inaccuracies will be replicated in the results reporting module as some fields auto populate. 10

11. Preparing for Results Reporting TIP: Populating the “Detailed Description” field in your Registration record may assist in the reader’s understanding of the study and will make your Quality Assurance review more effective. Remember, the ultimate goal is clarity for the public. 11 TIP: If time allows, release your registration one final time to the ClinicalTrials.gov QA review team to ensure you have addressed any errors before inputting results.

12. Preparing for Results Reporting: Recommended Reading How to Submit Your Results Includes preparation checklists and templates, also links to definitions http://clinicaltrials.gov/ct2/manage-recs/how-report Simple Results Templates and Examples http://prsinfo.clinicaltrials.gov/results_table_layout/ResultSimpleForms.html Basic Results Data Elements Definitions http://prsinfo.clinicaltrials.gov/results_definitions.html PRS User Guide Located on Main Menu in database 10 minute webinars for each results module http://clinicaltrials.gov/ct2/manage-recs/present Helpful Hints (with common study designs examples) http://prsinfo.clinicaltrials.gov/ResultsExamples.pdf http://prsinfo.clinicaltrials.gov/ResultsExamples.pdf 12

13. Results Overview 13 This is your home base for inputting results. You can access the 4 Results modules from here. Click on “edit” next to Participant Flow to edit that module.

14. Edit Mode – Simple but important! At the bottom of each page in edit mode: Save: Saves your work and brings you back to the “Results Overview page.” Validate: Activates display of Errors, Warnings and Notes, but does not save entered data. Cancel: Does not save changes and brings you back to the “Results Overview page.” 14

15. Module 1: Participant Flow 15

16. FDAAA: Participant Flow Definition “A table…, including the number of patients who dropped out of the clinical trial and the number of patients excluded from the analysis, if any.” FDAAA[Sec. 282(j)(3)(c)(i)], http://www.gpo.gov/fdsys/pkg/PLAW- 110publ85/pdf/PLAW-110publ85.pdf#page=82http://www.gpo.gov/fdsys/pkg/PLAW- 110publ85/pdf/PLAW-110publ85.pdf#page=82 16

17. Participant Flow Purpose Tabular presentation of progress of research participants through each stage of trial – Arms are “copied” from Protocol Section, but may be modified – Table may consist of a single Period or multiple Periods, to represent different stages of the trial – Each Period must include 2 Milestones: Number STARTED and Number COMPLETED The module should account for all enrolled participants, and make it clear which participants were analyzed. See ClinicalTrials.gov Participant Flow Module Presentation, Rebecca J. Williams, PharmD, MPH, at http://prsinfo.clinicaltrials.gov/webinars/module5/index.htmlhttp://prsinfo.clinicaltrials.gov/webinars/module5/index.html 17

18. Participant Flow Data Elements Recruitment Details Pre-assignment Details Arm/Group* ‐Title ‐Description Period Title(s)* ‐“Overall Study” (default) if single period Milestones ‐Milestone Title ‐STARTED Data* ‐COMPLETED Data* ‐Reason Not Completed  Type (e.g., Death)  Data *Required by ClinicalTrials.gov For definitions of these elements, see the Basic Results Data Elements Definitions: http://prsinfo.clinicaltrials.gov/results_definitions.html http://prsinfo.clinicaltrials.gov/results_definitions.html 18

19. 19

20. 20 Get Organized Using the Simple Form

21. Participant Flow Structure Tabular:Table showing progress of participants through each stage Arms/Groups:pre-populated from Protocol Section (but can be modified) Period(s):represent different stages of study (single or multiple) Milestones: specific events/time points when # of subjects reported. STARTED and COMPLETED are required data. 21 Example: If your study only has one period than you would provide the title of “Overall Study” to the period and only one Participant Flow table would be needed for your record. Example: Basic Participant Flow table for parallel design study For examples of other study designs, see ClinicalTrials.gov’s Helpful Hints: Basic Results at http://prsinfo.clinicaltrials.gov/ResultsExamples.pdfhttp://prsinfo.clinicaltrials.gov/ResultsExamples.pdf

22. Participant Flow Structure (Multiple Periods) 22 EXAMPLE: If your study has 3 different periods, you would create 3 tables and provide a descriptive title for each. STARTED & COMPLETED are required milestones. You can add others milestones to show specific events when the # of participants was reported. You will track the # of participants that do not complete the period and provide a reason why. Example adapted from ClinicalTrials.gov‘s Basic Results Helpful Hints, p.13 at http://prsinfo.clinicaltrials.gov/ResultsExamples.pdf http://prsinfo.clinicaltrials.gov/ResultsExamples.pdf

23. Participant Flow – What is wrong with this? 23 Period: First Intervention Arm A: Drug Bupropion, then placebo Arm B: Placebo, then Drug Bupropion STARTED 9295 COMPLETED 8895 NOT COMPLETED 40 Period: Second Intervention Arm A: Placebo, then Drug Bupropion Arm B: Drug Bupropion, then placebo STARTED 8893 COMPLETED 8893 NOT COMPLETED 00 For the 1 st period, there is no explanation of why some subjects did not complete the period. In the 2 nd period, the # of subjects that started does not match the # that finished the 1 st period. This may be a data error. Or if there was a washout period between the two periods, you may want to add that as a separate period and explain what happened to the subjects that dropped out.

24. 24 Participant Flow – Quick Tips Number of participants “started” should match “Enrollment, Actual” in protocol section – Pre-Assignment Details can be used to explain any discrepancies in Enrollment, Actual and total number of participants Started. Specific Periods to reflect study design and to account for # of participants starting and completing each Period Use of Milestones to convey key events – For example, number who received intervention Reasons for non-completion See http://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdfhttp://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdf

25. Module 2: Baseline Characteristics

26. Baseline Characteristics “A table of demographic and baseline data for the entire trial population and for each arm or comparison group.” Note that only baseline measures for Age and Gender are required; all other baseline measures are optional. http://prsinfo.clinicaltrials.gov/results_definitions.html http://prsinfo.clinicaltrials.gov/results_definitions.html explaining FDAAA Sec. 282(j)(3)(c)(i)

27. Description of Baseline Characteristics Module Table of demographic and baseline data for the entire trial population and for each arm or comparison group Accommodates different data types: – Continuous: measure of central tendency (e.g., mean) and measure of dispersion (e.g., standard deviation) – Categorical: for each category – (1) a count or (2) measure of central tendency and measure of dispersion

28. 28

29. 29

30. Get organized using the Templates

31. If it’s in Table 1, it probably belongs in Baseline Characteristics These will become Default measures These will become User Specified Baseline Measures

32. ClinicalTrials.gov Format

33. 33

34. ClinicalTrials.gov Format

35. User-Specified Baseline Characteristics Do include all meaningful elements that make your study understandable/ useful to others, both demographic and clinical measures, such as baseline values of outcome measures or prior and concurrent treatment characteristics. Make sure units and scales are labeled, and understandable – (e.g. what the range of a scale is and what it means. If it’s a well-known scale, within the discipline, refer to it by name and give a citation, if necessary.) Use Table 1, if there is a published article or draft. If there is no article from which to work, consult with PI.

36. Baseline doesn’t always mean only at the beginning of the study More than one period or time point can be considered “baseline” in studies which follow more complicated protocols: Crossover Dose escalation Factorial Multiple Period That is, not every one of these designs would require posting multiple baselines, but there may be times when the responsible party feels that to have a clear understanding of the findings of the study, it helps to show how baseline characteristics differed between one portion of the study and another.

37. Description Sorafenib (Nexavar, BAY43-9006) Inpatient dose escalation of sorafenib from 400 mg orally twice daily (bid) for the first cycle, 600 mg bid for the second cycle and 800 mg bid until disease progression, unacceptable toxicity or withdrawal of consent. Each cycle is 28 days. Dose reductions due to toxicities were allowed. Sorafenib (Nexavar, BAY43-9006) STARTED83 COMPLETED52 Not Completed31 Reporting Groups Cycle 1 (28 Days) Reporting Groups Description Sorafenib (Nexavar, BAY43-9006 Inpatient dose escalation of sorafenib from 400 mg orally twice daily (bid) for the first cycle, 600 mg bid for the second cycle and 800 mg bid until disease progression, unacceptable toxicity or withdrawal of consent. Each cycle is 28 days. Dose reductions due to toxicities were allowed. Sorafenib (Nexavar, BAY43-9006) STARTED52 COMPLETED44 Not Completed8 Cycle 2 (28 Days) Sorafenib (Nexavar, BAY43-9006) STARTED44 COMPLETED33 Not Completed11 Cycle 3 (28 Days) Baseline Measures Sorafenib (Nexavar, BAY43-9006 Number of Participants83 Age Continuous [units: years] Median (Full Range) Cycle 1 (n=83)61 (33 to 80) Cycle 2 (n=52)63 (33 to 72) Cycle 3 (n=44)55 (33 to 72) Example of Multiple Baselines shown From supplemental slide provided at Train-the-Trainer workshops; courtesy of ClinicalTrials.gov.

38. Important Quality Checks Review record to ensure: Categories do not overlap and all categories are presented Range and direction of scores in baseline characteristics description are explained Terms are understandable to the general public. Use footnotes if necessary to describe a score.

39. MODULE 3: OUTCOME MEASURES & STATISTICAL ANALYSES 39

40. FDAAA*: Outcome Measures “…a table of values for each of the primary and secondary outcome measures for each arm of the clinical trial…including the results of scientifically appropriate tests of the statistical significance of such outcome measures.” [Sec. 282(j)(3)(C)(ii)] *Food and Drug Administration Amendments Act of 2007 40

41. Outcome Measures Purpose The Outcome Measures module displays the results and associated statistical analyses for each prespecified primary and secondary outcome measure. Other outcome measures may also be included. See ClinicalTrials.gov Outcome Measures and Statistical Analyses Module Presentation, Deborah A. Zarin, M.D. at http://prsinfo.clinicaltrials.gov/webinars/module7/index.html

42. Outcome Measures – Data Elements To set up table – Arm/Group title* and Description – Number of Participants Analyzed* To Describe specific Outcome Measures – Outcome Measure Title* – Outcome Measure Description – Unit of Measure* – Outcome Measure Time Frame* – Measure Type (e.g., mean, median)* – Measure of Dispersion/Precision (e.g., Standard Deviation)* Data* *Required by ClinicalTrials.gov http://prsinfo.clinicaltrails.gov/results_definitions.html 42

43. Outcome Measures Data Elements Outcome Measure Type* – Options  Primary  Secondary  Other Pre-specified  Post-hoc Outcome Measure Reporting Status* Outcome Measure Safety Issue? (Y/N) Use the Outcome Measure Data Preparation Checklist found at: http://prsinfo.clinicaltrials.gov/data-prep-checklist-om-sa.pdf *Required by ClinicalTrials.gov http://prsinfo.clinicaltrails.gov/results_definitions.html 43 copied from protocol section

44. Get Organized Using the Simple Form 44 Use one simple form for each Outcome Measure you are entering into the database.

45. © 2009 Infectious Diseases Society of America Pappas PG, Chetchotisakd P, Larsen RA et al. Clin Infect Dis. 2009 Outcome Measures – Journal Article Format 45 Journal articles have similar information to ClinicalTrials.gov, just presented in a different format!

46. Outcome Measures – ClinicalTrials.gov Format, Public View NCT00145249 46

47. ClinicalTrials.gov Format (cont.), Public View NCT00145249 47

48. Outcome measure information: Please be specific as possible. Title: include the name of the specific measure. Avoid using verbs, that is, do not put “To determine…” Time Frame: must have a time point at which the outcome is assessed for the specific metric used (hours, days, weeks, years) Hint: specify which study day it is measured - do not use “until the end of study or death”. Description: describes what will be measured, not why it is measured. If the outcome measure is a questionnaire or scale, provide the range and what low or high scores mean. Safety Issue: Is this outcome measure assessing a safety issue? Outcome Measure 48

49. Example of an Outcome Measure What is wrong with this? Dysphagia Questionnaire At the end of the study Number of Patients with Complete Response to Dysphagia Baseline to 6 weeks 49

50. Data Elements – Statistical Analyses* 50 Basic Results Data Elements Definitions: http://prsinfo.clinicaltrials.gov/results_definitions.htmlhttp://prsinfo.clinicaltrials.gov/results_definitions.html *The law requires scientifically appropriate tests of statistical significance.

51. Go to Outcome Overview to Add Statistical Analysis 51

52. Add Outcome Statistical Analysis 52

53. Add Outcome Statistical Analysis 53 Choose the statistical test from the drop-down box.

54. Completed Outcome Measure and Statistical Analysis 54

55. Outcome Measures and Statistical Analyses – Quick Tips Results, including study or outcome conclusions, are not provided in narrative form. The Title and Description are comprehensible and descriptive (e.g., Arm A, B, C is not informative). For measures obtained using a scale: – Name of scale is provided in “Measure Title.” – Range and direction of scores (0 = worst; 10 = best) are indicated in “Measure Description.” – “Unit of Measure” is “units on a scale” if no other unit. See http://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdfhttp://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdf 55

56. Module 4: Adverse Events 56

57. Adverse Events Reporting Adverse Event (AE) “Unfavorable changes in health, including abnormal laboratory findings, that occur in trial participants during the clinical trial or within a specified period following the trial. Two types of adverse event data are to be reported: "Serious" and "Other (Not Including Serious)" adverse events.” http://prsinfo.clinicaltrials.gov/results_definitions.html#AdverseEve ntsDefinition 57

58. Adverse Events Reporting Serious Adverse Events (SAEs) – A table of all anticipated and unanticipated serious adverse events grouped by organ system, with number and frequency of such events in each arm of the trial Other Adverse Events (AEs) – A table of anticipated and unanticipated adverse events that exceed 5% within any arm of the trial, grouped by organ system, with number and frequency of such events in each arm of the trial. 58

59. Description of the Adverse Events Reporting Module The Adverse Events Module is a summary of results for SAEs and AEs that are collected during the study Data should be tracked and reported in accordance with the procedures for data collection as defined in the protocol Adverse Events are reported as summary data at the end of the study SAEs and AEs are presented in separate tabular format Not “real time” adverse event reporting while the study is ongoing Use the Adverse Events Reporting Additional Description to provide information on analysis population, methods of event data collection and other details http://prsinfo.clinicaltrials.gov/results_definitions.html 59

60. Adverse Event Reporting Adverse Events are entered by 2 modes – Manual entry directly into the PRS browser based interface – Download and Upload link Use the predefined Adverse Events Simple form for preparation – Simple form (template) is helpful in organizing results entry – Data is segmented and compiled by Frequency of Event Total Number of events Number of events (deaths, myocardial infarction, fever, etc.) Number events per participant 60

61. 61

62. Getting Organized Using the Serious Adverse Events Template 62

63. Getting Organized Using the Other Adverse Events Template 63

64. Adverse Events Frequency Threshold for Reporting Other (Not Including Serious) Adverse Event – The frequency of Other (Not Including Serious) Adverse Events that, when exceeded within any arm or comparison group, are reported in the results database for all arms or comparison groups. The number must be less than or equal to the allowed maximum (5%), and must not include any symbols (e.g., >=, %). Expressed as a percentage. For example, a threshold of 5 percent indicates that all Other (Not Including Serious) Adverse Events with a frequency greater than 5 percent within at least one arm or comparison group are reported. Total Number Affected by any Other (Not Including Serious) Adverse Event above the Frequency Threshold – Overall number of participants affected by one or more Other (Not Including Serious) Adverse Events above the specified Frequency Threshold (e.g., 5%). 64

65. Serious Adverse Events (Public site view) 65

66. Other Adverse Events (Public site view) 66

67. Important Quality Checks Total Number of Participants at Risk = Total Number of participants Started – Refer to the Participant Flow – Adverse Events Reporting Additional Description can be used to explain the Analysis Population (any discrepancies in total number of participants Started Participant Flow) The number of Events is not required when entering Adverse Events, however a Note will occur for each results entry field that is left blank. – You are not required to address Notes in order to submit your record Be consistent when entering event names for AEs. Use either all lower case or upper case letters. Corrections for text fields in the AE module must be corrected using “Modify” under event. When using the Download or Upload Adverse Events link, the upload will be rejected if the columns in your file do not match the order and content in the PRS. 67

68. ADDRESSING QA COMMENTS 68

69. ClinicalTrials.gov QA Results Data Flow Data Provider Inputs Results. RP “approves” & “releases” data to ClinicalTrials.gov. ClinicalTrials.gov conducts QA review of data within 30 days. APPROVES: ClinicalTrials.gov posts data on public website. RESETS to “in-progress”: ClinicalTrials.gov provides QA comments to Data Provider. 69

70. Accessing QA Comments Comments are accessed from the Main Menu – Problems: Link 70 Records with QA Comments may also be accessed using the “QA Review Comments” link on the Main Page

71. ClinicalTrials.gov PRS Common QA Comments General Expand acronyms and abbreviations Review for spelling errors Proper formatting Content Brief title is in lay language and includes condition and interventions evaluated in this study Review for internal consistency Check overall recruiting status and compare it to the study start date, the primary completion date, and the study completion date listed 71

72. ClinicalTrials.gov PRS Common QA Comments Outcome Measures Enter what is being measured, not why it is being measured ‐General terms as “safety,” “tolerability,” or “feasibility” are not sufficient Time frame, must be specific ‐“At Follow up” or “end of study” is not specific ‐Most outcome measures have one time point ‐Each unique combination outcome measure and time frame should be entered separately Time frame for change of outcome measure ‐Time frames indicate time period over which change occurred– generally two points should be entered ‐Time-to-event should include a time over which the event will be assessed When entering data into ClinicalTrials.gov enter arms and interventions ‐Specify each study arm first ‐ Specify each intervention ‐Assign each intervention to one or more study arms 72

73. REMINDER: New User Access Provide the following information to your PRS Administrator, Irine Breytburg (irine.breytburg@childrens.harvard.edu):irine.breytburg@childrens.harvard.edu o BCH Username o BCH Employee Number o Full name (e.g., John J Smith, MD) o Email address PRS Administrator sends profile request to ClinicalTrials.gov ClinicalTrials.gov emails Investigator/staff notifying of account & provides temporary password (within 2 days) Log into the ClinicalTrials.gov Protocol Registration System: https://register.clinicaltrials.gov/ https://register.clinicaltrials.gov/ 73

74. This slide set was made possible by a collaboration of CTSA organizations (Mayo Clinic, Partners, University of Michigan Medical School, Duke University) and the National Library of Medicine. The Clinical and Translational Science Awards Program (CTSA) is part of the Roadmap Initiative, Re-Engineering the Clinical Research Enterprise and is funded by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health (NIH). 74

75. BACK-UP SLIDES 75

76. FDAAA – Registration Requirements for Drugs/Biologics Is it a Clinical Investigation? – Defined as “any experiment in which a drug is administered or dispensed to one or more human subjects Is the clinical investigation controlled? – Is it designed to permit a comparison of a test intervention with a control to provide a quantitative assessment of the drug effect? – Concurrent & Non-concurrent controls Is the clinical investigation other than a phase 1 clinical trial? – Per FDAAA, Phase 1 includes initial introduction of an investigational drug into humans, metabolism, and pharmacologic actions of a drug, mechanism of action, and early evidence of effectiveness 76

77. FDAAA – Registration Requirements for Devices Is it a prospective study of health outcomes? – FDAAA defines ‘health outcome’ where primary purpose is to evaluate a defined clinical outcome directly related to human health Does the study ‘compare an intervention with a device against a control in human subjects’? – ‘Intervention defined broadly to include various techniques using the device such as (among other things): device regimens and procedures, use of prophylactic, diagnostic, or therapeutic agents’ Is the clinical study other than: – a small clinical trial to determine the feasibility of a device – a clinical trial to test prototype devices (primary outcome measure related to feasibility, not to health outcomes) 77

78. ICMJE – Registration Requirements Required for Prospective studies that: Assign subjects to an intervention with or without concurrent comparison or control groups Study the cause/effect relationship between medical intervention and a health outcome. ICMJE scope is much broader than the scope of FDAAA: Interventions include procedures, behavioral treatments, dietary interventions Health outcomes include any biomedical or health-related measure obtained in participants, including pharmacokinetic measures and adverse events 78 Source: http://www.icmje.org/publishing_10register.htmlhttp://www.icmje.org/publishing_10register.html