1. International Perthes Study Group
October 6-7, 2017
Patient Reported Outcomes Measurement Information System (PROMIS) for Patients with Perthes Disease
Hiroko Matsumoto, PhDc
Joshua E. Hyman, MD
Columbia University Medical Center

2. What is PROMIS?

3. Patient-Reported Outcomes Measurement Information System
What is PROMIS?
Patient-Reported Outcomes Measurement Information System
PROMIS
NIH-funded psychometric evaluation that has grown significantly over last several years: (
$90-million since 2004.
PROMIS measures the following metrics: (NIHPromis.org)
Patient-reported outcomes:
Subjective, e.g. how a patient perceives about their symptoms and function (

4. What is PROMIS? PROMIS Mission Statement
A comprehensive instrument to assess pain and pain behaviors
“PROMIS creates an opportunity for clinicians and patients to develop a common language around self-reported health status and in the process allow better assessment of the unique information from patients thereby enabling improved quality of care and quality of clinical research.”
(NIHPromis.org)

5. Patient-Reported Outcomes Measurement Information System
What is PROMIS?
Patient-Reported Outcomes Measurement Information System
Universal domains
Domain = Specific areas of importance regarding health and well-being
Cuts across different diseases
E.g., Sleep disturbance, depressive symptoms, ability to participate in usual role, global health
Generic measures versus disease specific measures
Source: HealthMeasures

6. PROMIS Pediatric Self and Proxy: 21 Domains*
*Profile domains: measuring the most important PROMIS concepts
Source: HealthMeasures

7. Patient-Reported Outcomes Measurement Information System
What is PROMIS?
Patient-Reported Outcomes Measurement Information System
Item banks
Collection of items all measuring the same thing
Can be administered in different ways (CAT, short forms) always producing a score on the same metric
Item Bank
Short Form
CAT
Computer Adaptive Test
Source: HealthMeasures

8. Static versus Dynamic Short From Static versus Dynamic
Static: Questions to follow do not change based on patient responses.
Questions presented in same sequence, regardless of prior answers.

9. PROMIS Example of Short Form: Pain Interference

10. Computer Adaptive Test
Static versus Dynamic
Static versus Dynamic
Dynamic option: Questions are presented in changing sequence, with following sequence based on prior answers.
Purpose:
Minimizes # questions;
Greater measurement precision;
Presented as Computer Adaptive Test (CAT).
CAT
Computer Adaptive Test

11. PROMIS Example of CAT Form: Pain Interference
It was hard to stay standing when I had pain

12. Longitudinal tracking of HRQoL is a strength of PROMIS
PROMIS T-score allows adult and pediatric scores to be compared directly  Longitudinal studies
Mean: 50 40 60
T-score of:
e.g. Raw score of 80
(Peds pain interference)
e.g. Raw score of 65
(Adult pain interference)
68%
of population
T-score of 60 = 15.8% of population at your age and gender perceive to have higher HRQoL than you

13. PROMIS Psychometrically sound Brief
Applicable in a variety of settings
Applicable across groups
Cover the full range of the domain
Individual domain stands alone
Available for use across ages
Score on one metric
Source: HealthMeasures

14. Data Collection by REDCap

15. PROMIS Validity Study in Perthes Disease
Funded by POSNA

16. Methods
Study Design and setting:
Prospective Validation Study
13 institutions from International Perthes Study Group
Inclusion Criteria:
Diagnosis of Perthes
Ages 8-17
Exclusion Criteria:
Surgical treatment within 6 months prior to enrollment

17. What is Validity?
The degree to which any measurement approach or instrument succeeds in describing or quantifying what it is designed to measure

18. Assessing Validities Types Methods Translational Validity
Face Validity
Content Validity
Criterion Validity
Predictive Validity
Concurrent Validity
Postdictive Validity
Construct Validity
Internal Construct Validity
Convergent Validity
(Convergent Evidence)
Discriminant Validity
(Discriminant Evidence)
Factorial Evidence (observed variables vs. latent construct)
External Construct Validity
Nomological Validity
Nomological validity: the degree to which a construct behaves as it should within a system of related constructs. Ex. As age increases, physical function decreases

19. Construct Validity
Is the measure consistent with the theoretical concept being measured?
All tests of validity ultimately designed to support/refute the instrument’s construct validity

20. Convergence and Discriminant Validities
Convergent Evidence
Demonstrates that your measure correlates highly with measures of the same construct
Groups known to differ along construct have different scores on measure
Discriminant Evidence
No to low correlation with measures of different constructs

21. PROMIS Pediatric Self and Proxy: 21 Domains*
*Profile domains: measuring the most important PROMIS concepts
Source: HealthMeasures

22. Construct Validity: Known Group Method
Waldenstrom Disease stages
Early: (IA – IIA)
Late: (IIB – IIIB)
Healed: (IV)

23. Demographics Characteristics N = 186 Age 10.23 ± 3.13 Gender (Male)
151 (82.1%)
Waldenström Stages
Early (IB; IIA)
24/186 (12.9%)
Late (IIB; IIIA; IIIB)
63/186 (33.9%)
Healed (IV)
99/186 (53.2%)
Ethnicity
White
101/186 (54.3%)
Black
3/186 (1.6%)
Asian
Hispanic
4/186 (2.2%)
Other/Unknown
75/186 (40.3%)

24. Within Physical Domain: Increased mobility was associated with lower pain and fatigue
Convergent Validity

25. Within Mental Health Domain: Increased Depression was associated with more Anxiety, and Anger
Convergent Validity

26. Across Domains: Increased Mobility was associated with less Anxiety, Depression and Anger
Convergent Validity

27. Across Domains: Increased Pain Interference was associated with more Anxiety, Depression and Anger
Convergent Validity

28. Across Domains: Increased Fatigue was associated with more Anxiety, Depression and Anger
Convergent Validity

29. Discriminant Validity
Across Domains: Peer Relationship have no to low associations with Physical Health
Discriminant Validity

30. Discriminant Validity
Across Domains: Peer Relationship have no to low associations with Mental Health
Discriminant Validity

31. All domains had the worst scores in the Early Stage group
Categories/Means
Early Stage
N=24
Late Stage
N=63
Healed
N=99
Fatigue
42.9
39.9
41.4
Mobility
40.2
45.5
48.6
Pain Interference
49.1
45.7
46.0
Anxiety
49.6
43.7
43.9
Depression
49.8
45.4
Anger
51.3
43.2
44.3
Peer Relations
48.0
51.7
51.2 )
NB: Minimally important difference (MID): Smallest change of a PRO that is perceived to be important by the patient  2-3 point (Yost et al )

32. Conclusions
PROMIS has construct validity in measuring QoL of patients with Perthes disease
Demonstrate that PROMIS domains correlates highly with measures of the same construct
Low correlation with domains measuring different construct
Groups known to differ along construct have different scores on measure
However…………

33. Discussions
When compared with U.S. average, some domain scores in patients with Perthes disease were counterintuitive:
Lower scores: Fatigue, pain interference, anxiety, depression, anger
Intuitive domain scores:
Lower score in Mobility
Lower score in peer relations in Early Stage group

34. Discussions and Next Steps
PROMIS not working?
Disease Specific Instrument
Skewed group?
More patients to recruit
Counterintuitive but reality?
Responsive testing (measure changes)
Further Known Group validity testing within Healed Group – Stratify by Stulberg classification

35. Thank You!
Hiroko Matsumoto