1 HPM 214 Course Review March 9, 2015 (9:00-11:50 am) HPM Broxton Avenue Los Angeles, CA 90024

Where we are now in HPM214 1.Introduction 2.Profile Measures (SF-36 due) 3.Preference-Based Measures 4.Designing Measures 5.Evaluating Measures 6.PROMIS/IRT/Internet Panels 7.Reviews of Manuscripts 8.Course Review (Cognitive interviews due) 9.Final Exam (3/16/15) 2

HPM 214 Assignments Class participation (25%) Two class assignments (25%) –Complete the SF-36 v2 survey at 36.org/demos/SF-36v2.htmlhttp:// 36.org/demos/SF-36v2.html –Conduct and summarize 5 cognitive interviews with a self-administered HRQOL survey. Extra credit (2-page critique of published HRQOL article). 3

U.S. Health Care Issues Access to care –~ 50 million people without health insurance Costs of care –Expenditures ~ $ 2.7 Trillion Effectiveness (quality) of care 4

How Do We Know If Care Is Effective? Effective care maximizes probability of desired health outcomes –Health outcome measures indicate whether care is effective Cost ↓ Effectiveness ↑ 5

What Are Health Outcomes ? Traditional clinical endpoints –Death, disease occurrence, other adverse events –Clinical measures/biological indicators Blood pressure Blood hemoglobin level Symptoms (e.g. fever) Health-Related Quality of Life

7 HRQOL is Multi-Dimensional HRQOL Physical Mental Social

8 Health-Related Quality of Life (HRQOL) How the person FEELs (well-being) Emotional well-being Pain Energy What the person can DO (functioning) Self-care Role Social

9 HRQOL is Not Quality of environment Type of housing Level of income Social Support

Patient-Reported Outcomes (PROs) “Any report coming from patients about a health condition and its treatment” (U.S. FDA, 2006) Including Health-related quality of life (HRQOL) Satisfaction with treatment Patient reports about care Needs assessment Adherence to treatment

Patient-Reported Outcomes (PROs) “Any report coming from patients about a health condition and its treatment” (U.S. FDA, 2006) Including Health-related quality of life (HRQOL) Satisfaction with treatment Patient reports about care Needs assessment Adherence to treatment

Patient-Reported Measures (PRMs) Background characteristics –Age, education, income Health care experiences –Reports about care (e.g., communication) Behavior –Adherence to physician recommendations Outcomes –Satisfaction with care –HRQOL

Uses of HRQOL Measures

HS 214, Winter 01·11·10

Behavioral Risk Factor Surveillance System (BRFSS) Telephone interview (random digit dialing) of nationwide survey of U.S. adults % reporting poor or fair health about 16%

Greater % of fair or poor health reported by older adults (33% for 75+ vs. 9% for 18-24)

Greater % of fair or poor health reported by females (17%) than males (15%)

Uses of HRQOL Measures

Observational Study Observation of groups (non-random assignment) Outcomes Survival Clinical HRQOL Casemix adjustment needed + Conditions/comorbidity + Severity + Demographics

Uses of HRQOL Measures

Randomized Trial Design Outcomes –Survival –Clinical –HRQOL Control for case- mix may not be required Study Population Randomize Intervention Group Control Group

Uses of HRQOL Measures

HRQOL Assessment by Providers May Facilitate patient-physician communication Improve clinician understanding of patients’ problems (particularly those of a psychosocial nature) Detmar SB, Aaronson NK. Quality of life assessment in daily clinical oncology practice: a feasibility study. Eur J Cancer. 1998;34(8): Detmar SB, Muller MJ, Schornagel JH, Wever LD, Aaronson NK. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. J Am Med Assoc. 2002;288(23): Hess R, Tindle H, Conroy MB, et al. A randomized controlled pilot trial of the Functional Assessment Screening Tablet to engage patients at the point of care. JGIM. 2014; 29(12): Velikova G, Brown JM, Smith AB, Selby PJ. Computer-based quality of life questionnaires may contribute to doctor-patient interactions in oncology. Br J Cancer. 2002;86(1):51-9. Velikova G, Booth L, Smith AB, et al. Measuring quality of life in routine oncologypractice improves communication and patient well-being: a randomized controlled trial. JClin Oncol. 2004;22(4):

24 In general, how would you rate your health? Excellent Very Good Good Fair Poor

25 Does your health now limit you in walking more than a mile? (If so, how much?) Yes, limited a lot Yes, limited a little No, not limited at all

26 How much of the time during the past 4 weeks have you been happy? None of the time A little of the time Some of the time Most of the time All of the time

27 - Profile: Targeted vs. Generic - Preference Types of HRQOL Measures

28 Targeted HRQOL Measures Designed to be relevant to particular group. Sensitive to small, but clinically-important changes. More familiar and actionable for clinicians. Enhance respondent cooperation.

29 Kidney-Disease Targeted Items During the last 30 days, to what extent were you bothered by cramps during dialysis? Not at all bothered Somewhat bothered Moderately bothered Very much bothered Extremely bothered

30 SF-36 Generic Profile Measure Physical functioning (10 items) Role limitations/physical (4 items) Role limitations/emotional (3 items) Social functioning (2 items) Emotional well-being (5 items) Energy/fatigue (4 items) Pain (2 items) General health perceptions (5 items)

31 Scoring HRQOL Profile Scales Average or sum all items in the same scale. Transform average or sum to 0 (worse) to 100 (best) possible range z-score (mean = 0, SD = 1) T-score (mean = 50, SD = 10)

32 HRQOL in HIV Compared to other Chronic Illnesses and General Population Hays et al. (2000), American Journal of Medicine T-score metric

33 HRQOL in HIV Compared to other Chronic Illnesses and General Population Hays et al. (2000), American Journal of Medicine T-score metric

34 HRQOL in HIV Compared to other Chronic Illnesses and General Population Hays et al. (2000), American Journal of Medicine T-score metric

35 Hypertension Diabetes Current Depression Stewart, A.L., Hays, R.D., Wells, K.B., Rogers, W.H., Spritzer, K.L., & Greenfield, S. (1994). Long-term functioning and well-being outcomes associated with physical activity and exercise in patients with chronic conditions in the Medical Outcomes Study. Journal of Clinical Epidemiology, 47, Physical Functioning in Relation to Time Spent Exercising 2-years Before LowHigh Total Time Spent Exercising range

36 SF-36 PCS and MCS PCS_z = (PF_Z * 0.42) + (RP_Z * 0.35) + (BP_Z * 0.32) + (GH_Z * 0.25) + (EF_Z * 0.03) + (SF_Z * -.01) + (RE_Z * -.19) + (EW_Z * -.22) MCS_z = (PF_Z * -.23) + (RP_Z * -.12) + (BP_Z * -.10) + (GH_Z * -.02) + (EF_Z * 0.24) + (SF_Z * 0.27) + (RE_Z * 0.43) + (EW_Z * 0.49) PCS = (PCS_z*10) + 50 MCS = (MCS_z*10) + 50

SF-12 Items by Scale –General health (1) –Physical functioning (3b, 3d) –Role-Physical (4b, 4c) –Role-Emotional (5b, 5c) –Bodily pain (8) –Emotional well-being (9d, 9f) –Energy/fatigue (9e) –Social functioning (10) 37

38 Debate About Summary Scores Taft, C., Karlsson, J., & Sullivan, M. (2001). Do SF-36 component score accurately summarize subscale scores? Quality of Life Research, 10, Ware, J. E., & Kosinski, M. (2001). Interpreting SF-36 summary health measures: A response. Quality of Life Research, 10, Taft, C., Karlsson, J., & Sullivan, M. (2001). Reply to Drs Ware and Kosinski. Quality of Life Research, 10,

39 Farivar et al. alternative weights PCS_z = (PF_z *.20) + (RP_z *.31) + (BP_z *.23) + (GH_z *.20) + (EF_z *.13) + (SF_z *.11) + (RE_z *.03) + (EW_z * -.03) MCS_z = (PF_z * -.02) + (RP_z *.03) + (BP_z *.04) + (GH_z *.10) + (EF_z *.29) + (SF_z *.14) + (RE_z *.20) + (EW_z *.35) Farivar, S. S., Cunningham, W. E., & Hays, R. D. (2007). Correlated physical and mental health summary scores for the SF-36 and SF-12 health survey, V. 1. Health and Quality of Life Outcomes, 5: 54. [PMCID: PMC ]

40 Is Complementary and Alternative Medicine (CAM) Better than Standard Care (SC)? CAM SC CAM SC Physical Health CAM > SC Mental Health SC > CAM

41 Does Taking Medicine for HIV Lead to Worse HRQOL? dead 1 Nodead dead 2 Nodead 3 No50 4 No75 5 No100 6 Yes0 7 Yes25 8 Yes50 9 Yes75 10 Yes100 Subject Antiretrovirals HRQOL (0-100) No Antiretroviral375 Yes Antiretoviral550 Group n HRQOL

42 Cost-Effectiveness of Health Care Cost ↓ Effectiveness (“Utility”) ↑

Preference Elicitation Standard gamble (SG) Time trade-off (TTO) Rating scale (RS) –  SG > TTO > RS  SG = TTO a  SG = RS b (Where a and b are less than 1) Also discrete choice experiments 44

45

46 SF-6D Brazier et al. (1998, 2002) — 6-dimensional classification (collapsed role scales, dropped general health) — Uses 12 SF-36 items (PF: 3a, b, j; R: 4c, 5b; SF: 10; BP: 7, 8; MH: 9b, f; EN: 9e) --- About 18,000 possible states -— 249 states rated by sample of 836 from UK general population

SF-6D Example Mean = 0.73 (SD = 0.14) Adjusted R-squared of 39% for 43 dfs Only 2 of 23 conditions had non- significant associations (melanoma, endometrial cancer)

48 HRQOL in SEER-Medicare Health Outcomes Study (n = 126,366) Controlling for age, gender, race/ethnicity, education, income, and marital status.

Summary of SF-6D Example Unique associations of multiple chronic conditions on health-related quality of life are generally similar and additive, not interactive The largest unique associations of chronic conditions with health-related quality of life among Medicare managed care beneficiaries was observed for four chronic medical conditions –Stroke, COPD/asthma, sciatica, arthritis of the hip Advanced stage of cancer is associated with noteworthy decrement in health-related quality of life for four “big” cancers (breast, prostate, colorectal, lung)

50 End goal is measure that is “Psychometrically Sound” Same people get same scores Different people get different scores and differ in the way you expect Measure works the same way for different groups (age, gender, race/ethnicity) Measure is practical

51 First law of survey development: Only do it when necessary

52 Second law: Know thy respondent

53 Third law: Practice before you play “Cut and try, see how it looks and sounds, see how people react to it, and then cut again, and try again” Converse & Presser (1986, p. 78) Identify problems with –Comprehension of items (stem/response options) –Retrieval of information –Skip patterns –Response burden

54 Fourth law: Keep it simple and short

55 Fifth law: Believe the survey respondent, but only so much

Four Levels of Measurement Nominal (categorical) Ordinal (rank) Interval (numerical) Ratio (numerical)

Levels of Measurement and Their Properties Property LevelMagnitude Equal Interval Absolute 0 NominalNoNoNo OrdinalYesNoNo IntervalYesYesNo RatioYesYesYes

Measurement Range for HRQOL Measures NominalOrdinalIntervalRatio

Variability Responses fall in each response category Distribution approximates bell-shaped “normal” curve (68.2%, 95.4%, and 99.6%)

Reliability Reliability is the degree to which the same score is obtained for thing being measured (person, plant or whatever) when that thing hasn’t changed. –Ratio of signal to noise

Kappa Coefficient of Agreement (Corrects for Chance) kappa = (observed - chance) (1 - chance) “Quality Index”

Reliability Minimum Standards 0.70 or above (for group comparisons) 0.90 or higher (for individual assessment)  SEM = SD (1- reliability) 1/2  95% CI = true score +/ x SEM  if z-score = 0, then CI: -.62 to +.62 when reliability = 0.90  Width of CI is 1.24 z-score units

63 Range of reliability estimates for blood pressure for multi-item self-report scales Hahn, E. A., Cella, D., et al. (2007). Precision of health-related quality-of-life data compared with other clinical measures. Mayo Clin Proceedings, 82 (10),

Multitrait Scaling Analysis Internal consistency reliability –Item convergence Item discrimination

65 Item-scale correlation matrix

Validity Does instrument measure what it is supposed to measure? Content Validity –Includes face validity Construct Validity –Many synonyms

67 % Dead (n=676) (n=754) (n=1181) (n=609) SF-36 Physical Health Component Score (PCS)—T score Ware et al. (1994). SF-36 Physical and Mental Health Summary Scales: A User’s Manual. Self-Reports of Physical Health Predict Five-Year Mortality

Evaluating Construct Validity ScaleAgeObesityESRDNursing Home Resident Physical Functioning Medium (-). Small (-) Large (-) Depressive Symptoms ? Small (+) ? Medium (+) Cohen effect size rules of thumb (d = 0.2, 0.5, and 0.8): Small correlation = Medium correlation = Large correlation = r = d / [(d 2 + 4).5 ] = 0.8 / [( ).5 ] = 0.8 / [( ).5 ] = 0.8 / [( 4.64).5 ] = 0.8 / = (Beware r’s of 0.10, 0.30 and 0.50 are often cited as small, medium, and large.)

Responsiveness to Change HRQOL measures should be responsive to interventions that changes HRQOL Need external indicators of change (Anchors)

Minimally Important Difference (MID) External anchors –Self-report –Provider report –Clinical measure –Intervention Anchor correlated with change on target measure at or higher Anchor indicates “minimal” change

Listed below are a few statements about your relationships with others. How much is each statement TRUE or FALSE for you? 1. I am always courteous even to people who are disagreeable. 2. There have been occasions when I took advantage of someone. 3. I sometimes try to get even rather than forgive and forget. 4. I sometimes feel resentful when I don’t get my way. 5. No matter who I’m talking to, I’m always a good listener. Definitely true; Mostly true; Don’t know; Mostly false; Definitely false

72 Physical Functioning and Emotional Well-Being at Baseline for 54 Patients at UCLA-Center for East West Medicine EWB Physical MS = multiple sclerosis; ESRD = end-stage renal disease; GERD = gastroesophageal reflux disease. 72

73 Significant Improvement in all but 1 of SF-36 Scales (Change is in T-score metric) Changet-testprob. PF RP BP GH EN SF RE EWB PCS MCS

Effect Size (Follow-up – Baseline)/ SD baseline Cohen’s Rule of Thumb: ES = 0.20 Small ES = 0.50 Medium ES = 0.80 Large 74

75 Effect Sizes for Changes in SF-36 Scores Effect Size PFI = Physical Functioning; Role-P = Role-Physical; Pain = Bodily Pain; Gen H=General Health; Energy = Energy/Fatigue; Social = Social Functioning; Role-E = Role-Emotional; EWB = Emotional Well-being; PCS = Physical Component Summary; MCS =Mental Component Summary

76 Defining a Responder: Reliable Change Index (RCI) Note: SD bl = standard deviation at baseline r xx = reliability

% Improve Significantly % Improving% DecliningDifference PF-1013% 2%+ 11% RP-431% 2%+ 29% BP-222% 7%+ 15% GH-5 7% 0%+ 7% EN-4 9% 2%+ 7% SF-217% 4%+ 13% RE-315% 0% EWB-519% 4%+ 15% PCS24% 7%+ 17% MCS22%11%+ 11%

Item Responses and Trait Levels Item 1 Item 2 Item 3 Person 1Person 2Person 3 Trait Continuum 78

Reliability Target for Use of Measures with Individuals  Reliability ranges from 0-1  0.90 or above is goal  SE = SD (1- reliability) 1/2  Reliability = 1 – (SE/10) 2  Reliability = 0.90 when SE = 3.2  95 % CI = true score +/ x SE 79

Convenience Internet Panels PROs –Relatively inexpensive and faster –Able to get to low incidence subgroups CONs –Data integrity False answers Answering too fast Same answer repeatedly Duplicate surveys from same person –Respondents may differ from intended target on measured (more educated) and on unmeasured characteristics

Probability Panels Selection probabilities known. –Need sampling frame (denominator) Get internet access for those without it. 81

Example Questions 1)What is the difference between a profile and preference-based measure? 2)Name a profile measure. 3)Name a preference-based measure. 4)What is a quality-adjusted life year? 5)What does “ACE” stand for? 82

Example Questions (2) 1)What is the difference between a PRM and a PRO? 2)What are the 3 underlying dimensions of HRQOL? 3)Is social support an indicator of HRQOL? 4)What is known about using HRQOL in clinical practice? 5)How much of the time during the last 4 weeks have you been happy?” is an item that measures what? 83

Questions