1. FDA Approach to Review of Outcome Measures for Drug Approval and Labeling: Content Validity Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) April 17-18, 2014 Elektra Papadopoulos, MD, MPH Study Endpoints and Labeling Development (SEALD) Office of New Drugs (OND) Center for Drug Evaluation and Research (CDER)

2. The views expressed in this presentation are those of the speaker, and do not necessarily represent an official FDA position. Disclaimer 2

3. Outline Elements of the review of content validity Common instrument design shortcomings 3

4. ELEMENTS OF THE REVIEW OF CONTENT VALIDITY FOR CLINICAL OUTCOME ASSESSMENTS 4

5. 5

6. Content validity Content validity is the extent to which the content of an instrument represents important aspects of a given concept for an intended use and for a defined target population Establishing content for a new instrument may involve both qualitative and quantitative research methods. Qualitative data are essential for establishing content validity of a COA. Input from the target population of respondents is essential 6

7. Content validity review Content validity review necessitates review of each element of an instrument Instrument: A means to capture data plus all the information and documentation that supports its use e.g., –User manual –Instructions –Mode of administration (e.g., self administration or interviewer administered) –Data collection method (e.g., paper-based, computer assisted) –Scoring 7

8. Sources of evidence to support content validity Literature review Expert input –Clinical –Instrument development Patient input –Concept elicitation interviews –Cognitive interviews 8

9. Sample criteria for evaluating items 9 Patrick DL et al; Value in Health; 2011

10. COMMON DESIGN SHORTCOMINGS IN ITEM DEVELOPMENT 10

11. Common instrument design shortcomings Global ratings of a complex concept Ratings of change Assessment of an unobservable concept by an observer Measuring indirect impacts without also measuring direct Inappropriate recall period Combining more than one concept within a single item (e.g., double-barreled items) 11

12. Global ratings of a complex concept Example: –Overall, based on all your years of medical practice, how severe is this patient’s overall health condition? It is a problem because: –Respondents consider unknown/unspecified aspects of the patient’s condition and assign a value/rating in an unstandardized manner To avoid the problem: –Select/develop instruments that include separate items or scores for each important sub-concept (e.g., sign or symptom) relevant to the concept of interest 12

13. Ratings of change Example: –Compared to your pain before you had cancer, please rate your pain today It is a problem because: –Respondents are required to recall their previous state and make a comparison to some period in the past –Ratings of change do not describe the patient’s current state To avoid the problem –Use items that ask about the patient’s current state in a non-comparative way 13

14. Assessment of an unobservable concept by an observer Example: –How severe is your child’s pain It is a problem because: –Assessments of patients by other individuals should be limited to observable concepts; ratings of a patient’s feelings/symptoms/mental states cannot validly be done by a person other than the patient To avoid the problem: –Use items that ask about observable behaviors (e.g. How frequently did your child cry?) 14

15. Measuring indirect impacts without also measuring direct Example: –I am afraid I won’t have enough time to reach the bathroom. I worry my incontinence will get worse. It is a problem because: –Emotions such as “bother”, “distress”, “afraid” do not measure the effect of treatment on core disease symptoms To avoid the problem: –Use a more direct measure (e.g., log of the number of incontinence episodes) 15

16. Inappropriate recall period Example: –For a condition where itch fluctuates daily: Over the past four weeks, on average, how severe was your itching? It is a problem because: –The recall period requires not only accurate recall, but also complex mental tasks (e.g., averaging of experiences that may fluctuate over a lengthy period of time) To avoid the problem: –Use items with an appropriate recall period for the population, concept and clinical trial context 16

17. Combining more than one concept within a single item Example: –Today, how many times did you cough up blood OR sputum? It is a problem because: –Does not allow discrimination of which concept the respondent is rating To avoid the problem: –Use one concept per item Today, how many times did you cough up blood? Today, how many times did you cough up sputum? 17

18. 18 To Illustrate (Lack of) Content Validity Study population = primarily bed-ridden, minimal physical activity (e.g., walking to the bathroom) very difficult Concept of interest to evaluate treatment benefit = physical functioning Clinical Outcome Assessment = patient-reported questionnaire to assess physical functioning Question in the assessment = “Do you have trouble running to the bus?”

19. 19 To Illustrate (Lack of) Content Validity Study population reports that mobility is a significant problem While interviewing patients, it is identified that their biggest concern/challenge is the ability to stand up from a chair and start walking Performance measure used in clinical trials: 6 minute walk test No assessment of the relevant component of mobility that is meaningful to this population of patients in their daily lives

20. SF-36 Physical Function Domain 20

21. 21 Carpal Tunnel Release Hip Joint Replacement

22. Patient report of “ability to perform daily activities” 22 FDA Guidance: Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims

23. Challenges with the use of actigraphy What aspect of physical activity is measured by actigraphy? What types of data (qualitative and quantitative) would be needed to link actigraphy results to a meaningful aspect of daily functioning in the patient population of interest? For example, patients with lower back pain are limited in some aspects of functioning that are not captured by acceleration data such as the ability to stand in one place or lift/carry objects 23

24. Summary Content validity is a critical measurement property that pertains to all COAs Inadequate content validity may lead to: –inability to detect a treatment effect and –inability to interpret/describe a treatment effect Careful consideration of instrument design is critical to ensure the COA can be interpreted and accurately described in product labeling 24