1. PFF Teal = 0+160+175 MAIN COLORS PFF Green = 120+162+47 Light Green = 193+216+47 Red = 242+102+73 HIGHLIGHT COLORS Light Grey = 220+220+210 Dark Grey = 100+98+76 Black = 0+0+0 CLINICAL SCIENCE: PATIENT-CENTERED ENDPOINTS – SYMPTOMS AND FUNCTIONAL STATUS SESSION LEADERS: RAFAEL L. PEREZ, MD SONYE K. DANOFF, MD, PhD

2. PFF Teal = 0+160+175 MAIN COLORS PFF Green = 120+162+47 Light Green = 193+216+47 Red = 242+102+73 HIGHLIGHT COLORS Light Grey = 220+220+210 Dark Grey = 100+98+76 Black = 0+0+0 HOW TO BUILD A PATIENT-REPORTED OUTCOME (PRO) HILARY WILSON, PHD CLINICAL SCIENCE: PATIENT-CENTERED ENDPOINTS - SYMPTOMS AND FUNCTIONAL STATUS NOVEMBER 13, 2015

3. © 2015 Evidera. All Rights Reserved. How to Build a Patient Reported Outcome (PRO) Measure PFF Summit, Washington DC November 13, 2015

4.  Employee of Evidera, a scientific consulting firm that provides methodological and strategic support to life sciences companies. As a salaried employee of Evidera, I do not receive consulting fees directly for my work. 4

5. History of PROs in Healthcare Patient Reported Outcomes (PROs) 5 “Any report coming directly from the patient without interpretation by a third party about how they feel or function in relation to a health condition and a given intervention.” FDA 2009 1

6. What is the Role of PROs in Today’s Healthcare Environment? 6 Clinical Trial Endpoints Real-world Evidence Clinical Care Patient Self- Management

7. Learning Objectives  Describe the process of developing a PRO instrument.  Discuss considerations for PRO instrument development in regulatory, clinical, and patient settings. 7

8. Process of PRO Development 8

9. Engage patients early and often…  Patients provide valuable perspective to study team throughout the instrument development process  Patient partners –Partners are chosen to serve as consultants based on their expert knowledge of what it is like to live with their disease/condition –Partners paid for their expertise as consultants vs. participants are compensated for their time as research subjects 9

10. Resources for identifying patient partners… 10

11. Determine the goal of measurement  Name and define the concept  Consider context of use  Review literature  Anything already meet the measurement need? –If so, how well? 11

12. If new instrument is warranted…  Review literature to inform concept generation 12 3 2

13. If new instrument is warranted…  Obtain expert input: patient and clinical (concept elicitation)  Draft item pool: consider cultural validation  Pretest items: cognitive interviews; pilot testing  Revise/re-test 13

14. Content validity: consensus through qualitative research 14 4

15. Evaluate practical considerations  Mode of administration  Respondent burden  Investigator burden  Language translation and cultural adaptation 15

16. Reliability and Validity  Reliability –Internal consistency: Cronbach’s α >0.70 –Reproducibility in stable subjects: Intraclass Correlation Coefficient >0.70  Construct Validity –Convergent/divergent validity: Moderate to high correlations with similar constructs, low correlations with distally related constructs –Known Groups: Scale score is significantly different among groups known to have varying clinical status  Ability to Detect Change : Change in scale score is significantly different when there is a known change in the concept of interest. 16

17. Interpretation Thresholds  Amount of change in score that is considered clinically meaningful.  No single approach to identify a responder definition – requires multiple analyses, using different anchors to converge on threshold of change.  Interpretation thresholds of deterioration and improvement in PRO measure are not always equal. 17

18. Example Instrument Development Process 18 Literature Review (analysis of web content and literature) Expert Panel (patients and clinicians) Focus Group Discussion Guide Patient Focus Groups Caregiver Focus Groups Creation of Item Pool/Draft Instrument Cognitive Debriefing Refinement INSTRUMENT Quantitative Validation

19. Considerations for PRO development in regulatory context 19

20. What regulatory bodies are looking for… Established conceptual framework Established rationale for new or revised measure Demonstrated appropriate instrument development steps Patient voice Content validity Demonstrated psychometric performance for your population Reliability Validity Ability to detect change Interpretation thresholds 20

21. EMA vs. FDA  HRQL vs. PRO –EMA Guidance (2005) 5 is focused specifically on HRQL, a specific PRO concept, and generally considers PRO measures that evaluate core symptoms of a disease are well accepted trial endpoints and they are evaluated like any other clinical endpoint. –FDA Guidance (2009) 1 is focused on all PROs (e.g., any report made by the patient unfiltered by clinician). Generally, HRQL is considered to be too unspecific/broad to support a label claim.  EMA more likely than FDA to grant multiple claims (HRQL, symptoms, treatment satisfaction).  EMA tends to accept a PRO/HRQL measure, provided it is relevant to the patient population and has adequate evidence validity.  FDA tends to adhere strictly to PRO guidance (2009), and has specific formatting requirements for dossier. 21

22. Considerations for PRO development in clinical and patient self-management context 22

23. What clinician and patient tools require…  Often multicomponent solution (i.e. objective markers, PROs, wearables) –PROs may be only one aspect  Minimal burden/high impact –3 to 5 items –Interpretation of data – what does it mean for me/my patient? o Graphical representation of longitudinal change o Thresholds to inform step-wise care 23

24. What clinician and patient tools require (cont)…  Predictive Validity –How well does the set of variables/measures predict future health events? –What thresholds are associated with future events?  User experience testing –How does the measure fit into a clinical work flow or the patient’s daily life? 24

25. In conclusion…  No single PRO instrument is optimal for all applications and across all disease indications  Patient input is critical throughout the development process for PROs for all applications  There is value in validating PROs used for non-regulatory purposes as well – emphasis on predictive validity of clinician and patient self-management tools  Validation is an ongoing and never ending process 25

26. References 1 Food and Drug Administration. Guidance for Industry on Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Federal Register. 2009;74(235):65132-65133. 2 Food and Drug Administration. The Voice of the Patient: Idiopathic Pulmonary Fibrosis. March 2015; http://www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrug UserFee/ UCM440829.pdf accessed 21Oct2015. 3 https://www.patientslikeme.com/conditions/1954-pulmonary-fibrosis accessed 21Oct2015. 4 ISPOR Task Force on Content Validity of Existing Instruments, 2009. Value in Health – Figure 2w. 5 European Medicine Agency. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products. London, 2005 26