PROMIS: The Right Place at the Right Time? David Cella, Ph.D. Department of Medical Social Sciences Northwestern University Chair, PROMIS Steering Committee Principal Investigator, Statistical Coordinating Center

What is PROMIS? A nine-year commitment of NIH to improve and standardize measurement of patient reported outcomes (PROs) 50 million dollar investment (excl. ARRA) An effort to harness new psychometric methods to improve the quality and interpretability of PROs An attack on the PRO “Tower of Babel”

PROMIS Roadmap Goal: Create highly valid and reliable item banks and associated computerized adaptive testing Encourage wide adoption to improve assessment of self-reported symptoms other health-related quality of life domains – … across a wide range of chronic diseases

UNC –Chapel Hill ● ● Duke University ● Stanford University Stony Brook University ● Stony Brook University ● ● University of Pittsburgh ● University of Washington Northwestern ♥ NIH ● NIH PROMIS Network:

“Item Bank” A large collection of items measuring one thing Items in the same bank are linked on a common metric Basis for –Computer Adaptive Testing (CAT) Items are selected to maximize precision and retain clinical relevance –Customized short forms tailored to the population and research purpose

Building a Bank Testing and Analysis Depression Defined and Calibrated Bank CAT and Multiple Short Forms Items Deposited After Review item

“PROMIS-1” Highlights Consensus-driven PRO framework Qualitative study of > 1,000 people Quantitative study of > 20,000 people Eleven Item Banks available to collaborators Website and functioning Assessment Center Ongoing clinical validation in chronic disease populations Clear path/agenda for “PROMIS-2” and expanded collaborations

CaPS bank in development PROMIS v1.0 bank PROMIS area tested but no bank developed for v1.0 Area addressed (in part) by bank within lineage Area not addressed yet * = Additional cancer-specific PROMIS bank PROMIS Domain Framework

Item Banking Item Bank Local Independence Measurement Range Target Population Static Forms Multi-Stage CAT Item banks can support a variety of scale assembly strategies

Advantages of Short-Forms Developed from Item Banks Flexibility in length and content –Select items matched to clinical features and severity in the target population –Select items known to provide the most information Any form selected or created produces scores on a common metric

PROMIS Profile Short Forms 11 * reduced set (the full bank has 112 items) was used for real data simulation purposes Anxiety 29 Anxiety 29 Depression 28 Depression 28 Fatigue 95 Fatigue 95 Pain Impact 41 Pain Impact 41 Sleep Disturbance 27 Sleep Disturbance 27 Physical Function 86* Physical Function 86* Social Role 14 Social Role Mental Physical Social

12 PROMIS Profile Subscales: Test Information Functions

PROMIS Profile Subscales: Correlations with Full Banks

14 Estimated Power to detect a small effect (d=0.2) when the study population is centered on the general population (T-Score = 50)

15 Estimated Power to detect a small effect (d=0.2) when the study population is centered on 1 SD worse than the general population

Computerized Adaptive Testing (CAT) Select questions based on a person’s response to previously administered questions. Iteratively estimate a person’s standing on a domain (e.g., depressive symptoms) Administer most informative items Desired level of precision can be obtained using the minimal possible number of questions.

Beginning of CAT T-Score = 50SE = 10 Best Item- I felt depressed

I felt depressed 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 52SE = 4 Next Best Item- I felt like a failure

I felt like a failure 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 53SE = 3 Next Best Item- I felt worthless

I felt worthless 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 55SE = 2 Next Best Item- I felt helpless

I felt that nothing could cheer me up 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 55SE = 2

Fatigue Experience and Impact 95-item bank Legacy Instruments SF-36 Vitality Scale (4 items) FACIT-Fatigue Scale (13 items)

4-item SF36/Vitality 4-item CAT 13-item FACIT-Fatigue 13-item CAT 98-item Bank No Fatigue Severe Fatigue SE=0.32 (r=0.90) SE=0.22 (r=0.95) Standard Error Comparison of Measurement Precision Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

PROMIS Fatigue Short Form Garcia SF et al, J Clin Onc, © 2007 Reprinted with permission of the PROMIS Health Organization and the PROMIS Cooperative Group

7-item Short-form 7-item CAT 98-item Bank No Fatigue Severe Fatigue SE=0.32 (r=0.90) SE=0.22 (r=0.95) Standard Error Comparison of Measurement Precision Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

26 PROMIS Profile Subscales vs CATs: Correlations with Full Banks

PROMIS Assessment Center and Opportunities for Interaction A publicly available, adaptable and sustainable Internet-based system that: 1.Hosts 11 item banks for precise, valid, and efficient health status assessment via short forms or CAT 2.Collects and analyze patients’ responses 3.Provides instant health status reports to users to: Enable and enhance research Improve clinical decision-making Facilitate policy-making by health plan and systems and public programs 4. Supports health services / comparative effectiveness research applications

29

Assessment Center Aims Assessment Center supports… Research and clinical applications CAT Custom or off-the-shelf short forms Non-PROMIS measures welcome Study set up, maintenance and storage Scoring, reporting and interpretation

Domains Items in Bank Items in First Short Form Emotional Distress – Anger298 Emotional Distress – Anxiety297 Emotional Distress – Depression288 Fatigue957 Pain – Behavior397 Pain – Impact416 Physical Function12410 Satisfaction with Discretionary Social Activities127 Satisfaction with Social Roles147 Sleep Disturbance278 Wake Disturbance168 Global Health PROMIS Ver 1.0 Instruments

Thank you