1. (Northwestern University, CAPriCORN CDRN)
David Cella, PhD
(Northwestern University, CAPriCORN CDRN)
Patient centeredness and patient reported outcomes: PROMIS and its place in PCORnet

2. PROMIS Cooperative Group 2004-2014 Highlights
More than 50 research protocols aligned with evolving PROMIS standards
More than 50 grants
Roadmap/Common Fund
Individual Institutes
PCORI, DoD, VA, US Army, CDC, multiple foundations, industry
More than 50,000 people have provided data
Adults and Children
English, Spanish, Chinese and other languages
Qualitative and quantitative

3. PROMIS®
Measures
Science
Software
The resource that we developed is the Patient-Reported Outcome Measurement Information System, or [CLICK] “PROMIS.”
[CLICK] PROMIS is a collection of measures of PROs:
22 measures for adults, such as physical functioning and depression,
9 measures for pediatric populations, and more.
These measures were developed using the state-of-the-science.
[CLICK] Science: PROMIS is an effort to advance the state-of-the-science in measurement of patient-reported health status.
For example, there’s a collection of statistical models known as item response theory, that have been developed for educational testing. PROMIS has been instrumental in discovering the best way to adopt these statistical models for patient-reported health data.
[CLICK] Readily available software to disseminate and administer PROMIS measures.
We’ll go into more detail about PROMIS measures, PROMIS science, and PROMIS software soon.
Now it did take some time to develop this. [TRANSITION]

4. PROMIS Measures Adult Health Measures Pediatric Health Measures
More than 1,000 individual items (questions)
51 distinct item banks or scales
20 more coming soon
20 languages
Pediatric Health Measures
More than 150 items (questions)
18 distinct banks or scales
22 more on the way
8 languages

5. PROMIS is Domain-specific; not Disease-specific
How do we actually measure each domain?
For almost every domain, we use the most advanced measurement science to create the most flexible type of tool--the item bank.

6. Cuts across different diseases
A domain is the specific feeling, function, or perception you want to measure.
[CLICK] PROMIS Domains cut across different diseases.
Talk about studying the domain of FATIGUE in people with osteoarthritis, migraine sufferers, and people undergoing chemotherapy.
Cuts across different diseases

7. PROMIS Domain Framework
Symptoms
Physical Health
Function
Affect
Self-Reported Health
Mental Health
Behavior
Cognition
Global Health
Relationships
Social
Health
Function

8. An item bank is a large collection of items measuring a single domain.
[CLICK] Definition
[CLICK] Any and all . . .
Let’s look at an item bank for the domain of Physical Functioning. [NEXT]
Any and all items can be used to provide a score for that domain.

10. T Score Referenced to the US General Population The PROMIS Metric
Mean = 50
SD = 10
Referenced to the US General Population

11. PROMIS Basic Tools Global Health Index Global-10
Derived from Item Banks
Computerized Adaptive Testing (CAT)
Dynamic testing averaging 4-5 items per domain
Fixed Length Forms
By individual domain (4-10 items)
7-domain health profiles (-29, -43, -57)

12. PROMIS® Profile Short Forms (v1) (29-43-57 items)8 6 4
Anxiety 29
Mental
Depression 28
Fatigue 95
Pain Interference 41
Physical
The 7 sub-domains were selected to represent three higher level PROMIS domains, Mental Health, Physical Health, and Social Health. The short forms of different lengths, 4, 6, and 8 items, were designed to provide incrementally more precision and coverage for clinical populations.
Sleep Disturbance 27
Physical Function
124
Social
Satisfaction with Roles 14
(+ pain intensity) 12

13. Cycle of Development and Validation
Qualitative
Research
and Item
Writing
Testing
General Population
Clinical Samples
Item Bank
Item bank is not fixed. Don’t make and put on shelf.
We can continually work to evaluate and improve it.
Perhaps adding items to help measure the extremes of the domain.
Analysis
Interpretation
Refining

14. Efficiency

15. Computerized Adaptive Tests
Question #1
high physical
function 3 1 2
Question #2 1 2
Question #3 2 1
Purpose of this slide is to walk the audience through the CAT process using an example …and thereby transmit the message of narrower measurement range and higher precision with each question …
I typically ese e.g. an example for depression, or any other topic I want to focus on … like first question “do you feel depressed”, answer “some of the time” … than we would like to know how depressed and the CAT might choose a question asking about self worth, and if the patient indicates that this is diminished also, a third question asking about thoughts of suicide might be appropriate … in this case the answer would be no, never …
And maybe compare it with another scenario like the participant would have answered the first question, no I am not at all depressed, asking about suicide would not make sense, instead the CAT might have chosen a question asking about happiness … bla bla
The conclusion is, we can make precise test over the entire range …
Questionnaire with a high precision - AND a wide range
- 1
- 2
low physical
function
- 3

16. Validity

17. Responsiveness in Depressive Disorder: PROMIS Depression

18. Comparability

19. PROMIS Measures Tested in Six Conditions
Relevant Item Banks
COPD
Physical Function
Fatigue
Pain
Social Role Satisfaction
Emotional Distress (Depression, Anxiety, Anger)
Heart Failure
Depression
Low Back Pain
Pain (Interference and Behavior)
Sleep Disturbance
Arthritis
Cancer
Emotional Distress (Depression, Anxiety)

20. Same metric, same meaning
Fatigue Item Bank
Chemotherapy trial
Osteoarthritis trial
Heart Failure trial
Items 1-10
CAT
Items 6-12
The same PROMIS item bank can be used across conditions for varying degrees of severity across conditions.
Across all types of diseases, PROMIS Fatigue scores have same interpretation. Facilitates comparisons, meta-analyses, health policy, etc.
Epilepsy trial
Diabetes trial
Items 2, 4, 9, 13
Items 1-2
Same metric, same meaning

21. PROMIS Fatigue Across Five Clinical Conditions
Cancer
w/ benefit
(2 mos)
Cancer
Chemo
(B)
N = 310
N = 229
Back Pain
(3 mos)
Back Pain
(1 mo)
Back Pain
(B)
N = 114
Depression
(3 mos)
Depression
(1 mo)
Depression
(B)
N = 64
HF Post-transplant
HF Pre-transplant
Exacerbation to Stable
N = 125
COPD Stable (B)
COPD Exacerbation (B) 50 35 40 45 55 60 65
Average for General Population

22. PROsetta Stone® A Rosetta Stone for Linking Patient-Reported Outcome Measures
USPHS Grant No. RC4 CA

23. Raw Score to T-Score Conversion Table (IRT Fixed Parameter Calibration Linking) for PHQ-9 to PROMIS
PHQ-9 Score
PROMIS T-score SE
37.4
6.4 1
42.7
5.3 2
45.9
4.8 3
48.3
4.7 4
50.5
4.3 5
52.5
4.0 6
54.2
3.8 7
55.8
3.7 8
57.2
3.6 9
58.6
3.5 10
59.9
3.4 11
61.1
3.3 12
62.3 13
63.5
3.2 14
64.7 15
65.8 16
66.9 17
68.0
3.1 18
69.2 19
70.3 20
71.5 21
72.7 22
74.0 23
75.3 24
76.7

24. Choi et al, Psychological Assessment, 26(2): 513-527, 2014
Figure 3. Comparison of clinical cut scores on the PROMIS Depression metric. CES-D: 16 or higher for positive clinical depression. PHQ-9: 5-9 (mild), (moderate), (moderately severe), 20 or higher (Severe). BDI-II: 0-13 (minimal), (mild), (moderate), 29 or higher (severe).
Choi et al, Psychological Assessment, 26(2): , 2014

25. Raw Score to T-Score Conversion Table (IRT Fixed Parameter Linking) SF-36/PF-10 to PROMIS Physical Function
SF-36/PF Score
PROMIS T-score SE 10
24.5
4.0 11
28.3
2.8 12
30.3
2.5 13
32.0
2.2 14
33.4
2.1 15
34.8
2.0 16
36.0 17
37.2 18
38.4
1.9 19
39.5 20
40.7 21
41.8 22
42.9 23
44.1 24
45.3 25
46.7 26
48.2
2.3 27
49.9 28
52.0
2.9 29
55.0
3.5 30
61.7
5.7

26. Three Diseases; Three Instruments; and a
Tower of Babel
Sample
Instrument
Baseline
Follow up
Difference
(sig)
Relative value?
RheumatoidArthritis
HAQ-DI
1.20
0.30
- 0.90
(p<.05) ?
Lung Cancer
SF-36
PF-10 20 25
+ 5.0
Diabetes
PROMIS PF CAT 45 50
Just a note, FYI, HAQ-DI is usually scored in .25 increments (0 to 3.0). We estimated in smaller increments in our chart, which is more accurate.

27. PROsetta Stone Allows Comparability for CER Physical Function Change Across Three Conditions40 25 30 35 45 50 55
Average for General Population
Baseline
Follow-up
Lung Cancer
Rheumatoid Arthritis
Diabetes

28. GI PROMIS® in Clinical Practice
Integrating GI PROMIS into practice:
Delivering to patients via free app called “My GI Health”
Currently available in L.A. VA, UCLA and U. Michigan
Initiating pilot RCT under PROMIS supplemental funding
Working with Athena Health to connect to EHR via API
In short, we are delivering GI PROMIS questionnaires and scores via a free app called "My GI Health." We are currently using a web-based portal as part of our supplemental funding from our PROMIS grant. The web-based portal is a non-tethered app that will, in time, be "baked" into the EHR via API programming. For the moment, the app can be accessed by a link within the EHR, shown in the example here with the VA CPRS system. We are about to start a pilot RCT of GI PROMIS vs. usual care in our GI clinics at the West Los Angeles VA Medical Center, UCLA, and U. Michigan. The study will evaluate patient satisfaction with care, their perceptions about their provider's interpersonal skills, and shared decision-making.
We deliver the PROMIS scores using a modified "heat map," shown in this slide deck, which we crated after iterative dashboard development with rounds of provider interviews. Details probably not important here. We are also working with the EHR vendor, athenahealth, to write APIs between My GI Health and their EHR. We hope to test that in a large network of over 49,000 sites via the athenahealth network, and are seeking funding for that study. Finally, we are finalizing the consumer-facing version of My GI Health, shown in these slides, which will allow anyone to "take the test," so to speak, to measure their GI symptoms and compare against population benchmarks. We will also allow for longitudinal tracking on the app, also shown in the slides. If you looks closely at the tracker slides, you'll see our current visualization for longitudinal PROMIS score tracking. When there is a "+" in a dot (see slide), it means the patient has improved beyond the MCID for a scale. Same in reverse if there is a "-" in a dot. Users can select the scale of interest, and it will come forward in full color, with other scales falling behind in muted pastel colors.
Quick slide overview:
Slide 1: high level summary of current activities Slide 2: Example screenshots a patient completed on "My GI Health:
Slide 3: Example report on "heat map"
Slide 4: Shows link to "My GI Health" in VA CPRS system Slide 5: Example reports available for provider, including heat map linked to a fully automated HPI triggered by the system based off PROMIS scores (this provides detailed clinical context to the PROMSI heat map) Slide 6: Screen shots of consumer app to be launched in late Fall or early Winter Slide 7: Wireframes of longitudinal tracking on consumer app

29. GI PROMIS Symptom Tracker

30. Adults who Report Good or Better Physical Health, 2010
HP2020
Target: 79.8
Increase desired
Age (years)
NOTES: Data (except data by age group) are age adjusted to the 2000 standard population.
SOURCE: National Health Interview Survey (NHIS), CDC/NCHS.
Obj. HRQOL/WB-1.1 30

31. Adults who Report Good or Better Mental Health, 2010
HP2020
Target: 80.1
Increase desired
Age (years)
NOTES: Data (except data by age group) are age adjusted to the 2000 standard population.
SOURCE: National Health Interview Survey (NHIS), CDC/NCHS.
Obj. HRQOL/WB-1.2 31

32. > 300 PROMIS Publications

33. Concluding remarks
PROMIS provides reliable, flexible and responsive treatment options for several patient-centered health domains
Assessment options powered by Epic, REDCap, Assessment Center and others have made PROMIS attractive for health outcomes, health services and population health applications.
IRT modeling and linking enable flexibility in assessment options while retaining common reporting metrics

34. Final recommendations
Common Measures Working Group
Final recommendations

35. Common Measures Working Group
Lead: Dave Cella, PhD (CAPriCORN CDRN)
Members: Reesa Laws, Thilo Deckersbach, Michael Kappelman, James Willig, and Richard Mularski
Task: Select 5-10 common measures for collection across all PCORnet sites
Activities:
Identifying measures of greatest significance to the most networks.
Selecting common measure (single items) for data collection across networks based on priority domains
Sorting core items and highly recommended items

36. Common Measures WG Recommendations
Focus on domains rather than disease or conditions
Core domains include pain, fatigue, depression, sleep, physical function and social function, and general (global) perceptions of health and life quality
Additional domains include: anxiety, medication adherence, satisfaction with care
PROMIS single item measures as common measures
Can estimate scores on the same metric as more precise assessments of the same domains, through both static short forms and CAT.
large sample sizes allows for very good estimates of group average score based on only one question.

37. Common Measures WG Recommendations
Instrument linking through co-calibration of items across PROMIS other instruments via PROsetta Stone will enable PCORnet to emphasize domain metrics over measures.

38. Recommended Common Measure Items: Core for Adults
Domain
Time Point
Question
Answer Choices
Recommendation
Physical Function
Current
Are you able to run errands and shop?
5=Without any difficulty
4=With a little difficulty
3=With some difficulty
2=With much difficulty
Core
Depression
In the past 7 days
I felt depressed
1=Never
2=Rarely
3=Sometimes
4=Often
5=Always
Fatigue
I feel fatigued
1=Not at all
2=A little bit
3=Somewhat
4=Quite a bit
5=Very much
Sleep Disturbance
I had a problem with my sleep
5=Not at all
4=A little bit
2=Quite a bit
1=Very much
Pain Interference
How much did pain interfere with your day to day activities?

39. Recommended Common Measure Items: Highly Recommended for Adults
Domain
Time Point
Question
Answer Choices
Recommendation
Anxiety
In the past 7 days
I felt uneasy
1=Never
2=Rarely
3=Sometimes
4=Often
5=Always
Highly Recommended
Social Roles & Activities
None (current ability)
I have trouble doing all of my regular leisure activities with others
5=Never
4=Rarely
2=Usually
1=Always

40. Recommended Common Measure Items: Adults and Pediatrics
Domain
Time Point
Question
Answer Choices
Recommendation
General Health
None
(current perception)
In general, would you say your health is
5=Excellent
4=Very good
3=Good
2=Fair
1=Poor
Core
Quality of Life
In general, would you say your quality of life is
Global Physical Health
In general, how would you rate your physical health?
Global Mental Health
In general, how would you rate your mental health including your mood and your ability to think?
Highly Recommended

41. Recommended Common Measure Items: Core for Pediatric
Domain
Time Point
Question
Answer Choices
Recommendation
Pain Interference
In the past 7 days
I had trouble sleeping when I had pain
0=Never
1=Almost Never
2=Sometimes
3=Often
4=Almost Always
Core
Fatigue
I got tired easily
Depression
How often do you feel really sad
1=Rarely
4=Always

42. Recommended Common Measure Items: Highly Recommended for Pediatric
Domain
Time Point
Question
Answer Choices
Recommendation
Social Health
How often do you have fun with friends
0=Never
1=Rarely
2=Sometimes
3=Often
4=Always
Highly Recommended
Family Relationships
How often do your parents listen to your ideas?

43. Questions/Comments

44. Supplemental slides: PROMIS Measures in redcap
Example screen shots of what the PROMIS measures look like in REDCAP
Supplemental slides: PROMIS Measures in redcap

46. 46