1. 1 Class 1 Concept Development and Concept Definitions September 20, 2007 Anita L. Stewart, Ph.D. Institute for Health & Aging University of California, San Francisco

2. 2 Overview of Entire Class (EPI 225) u How to identify, critique and select good measures for your research u Homework – weekly u Two discussion sessions (optional) –2:15-3:00 and 4:30-5:00 u Focus on one concept of interest to you –Health dimension or a determinant of health –Measurable (need to find 2-3 measures to review)

3. 3 Inappropriate Measures can Result in: u Conceptual inadequacy u Poor data quality (e.g. missing data) u Poor variability u Poor reliability and validity u Inability to detect associations –Correlations, mean differences

4. 4 Overview of Class 1 u Concept development for new measures u Complexity of most concepts u Importance of defining concepts prior to selecting existing measures u Goal –Provide ideas for developing your own concept before considering measures

5. 5 Typical Sequence of Developing New Self-Report Measures Develop concept Create item pool Pretest/revise Field survey Psychometric analyses Final measures

6. 6 Typical Sequence of Developing New Self-Report Measures Develop concept Create item pool Pretest/revise Field survey Psychometric analyses Final measures

7. 7 Concept/Construct u A variable that is relatively abstract as opposed to concrete –e.g. health status, stress, decisional conflict, acculturation u Cannot be assessed directly u An abstraction based on observations of certain behaviors or characteristics

8. 8 Concept, Construct, Variable, Latent Trait, Latent Variable u Terms defined differently by different investigators/authors u Meaning is very similar –considerable overlap among them –often used interchangeably

9. 9 Latent Trait u Latent - present but not visible, unobservable u Latent trait - unobservable set of characteristics that can be empirically inferred and estimated through answers to a set of questions

10. 10 Measures of Concepts u Concepts and constructs are defined and operationalized in terms of observed indicators u Measures are “proxies” for the latent variables we cannot directly observe

11. 11 Depicting Latent Variables and Measures (Manifest Variables) Variable A Measure A Variable B Measure B

12. 12 Complexity of Concepts u By definition, due to abstract nature of concepts, most (if not all) are complex u Complexity means –Hard to define –Usually multidimensional

13. 13 Dimension, Domain, Sub-domain, Component u Dimension: a distinct component of a multidimensional construct that can be theoretically or empirically specified (e.g., physical and mental health) u Terms somewhat interchangeable

14. 14 “Health” as a Concept u What first comes to mind when you think of the word “health”? Health

15. 15 Some Common Terminology of “Health” u Health indicators u Health status u Health outcomes u Clinical status u Functional status u Functioning and well-being u Quality of life u Health-related quality of life

16. 16 Definition Varies by Perspective Health Bench scientist Individual Population science Clinician

17. 17 Perspectives on Health Bench ScientistClinicianIndividual Molecular, genetic, cellular X?? Anatomic, physiologic signs X Symptoms XX Diagnosed conditions XX Severity of conditions XX Functioning in daily life X Well-being X

18. 18 What is Perfect Health? An ideal toward which people are oriented – not a state they expect to attain. “A healthy person is someone who has been inadequately studied” (p. 31) AC Twaddle, The concept of health status, Soc Sci Med 1974;8

19. 19 Health as a continuum? Perfect health Less than perfect health Illness

20. 20 What are Normal Health States? Perfect health (abnormal) Less than perfect health (normal) Illness (abnormal)

21. 21 Contradictory? Or Just Different Perspectives? u Signs of illness (MD) u Perceive oneself to be in “excellent” health (individual) u No signs of illness (MD) u Perception of being in poor health u Cellular abnormalities (bench scientist) u No signs of illness (MD)

22. 22 Concept of Quality of Life u What first comes to mind when you think of the word “quality of life”? Quality of Life

23. 23 Quality of Life Terminology u Satisfaction with life “as a whole” u Well-being u Global well-being u Subjective well-being u Sense of well-being u Global happiness

24. 24 Dimensions or Domains of Quality of Life Quality of …. u Marriage u Family life u Health u Neighborhood u Friendships u Job u City u Housing u Standard of living u National government u Local government …. and many more

25. 25 Health-Related Quality of Life u Domains of life quality relevant to health, disease, and medical care u Well-being –Physical (malaise, symptoms) –Mental (psychological well-being) u Functioning –Limited in activities because of health problems or treatments

26. 26 MOS Framework: Functioning and Well- Being From the Patient’s Perspective u Physical functioning u Pain u Energy/fatigue u Sleep problems u Cognitive functioning u Psychological distress/well-being u Social activity limitations due to health u Role limitations due to physical health u Role limitations due to emotional problems u Health perceptions Stewart AL, reading for week 1.

27. 27 MOS Framework: Functioning and Well- Being From the Patient’s Perspective u Physical functioning u Pain u Energy/fatigue u Sleep problems u Cognitive functioning u Psychological distress/well-being u Social activity limitations due to health u Role limitations due to physical health u Role limitations due to emotional problems u Health perceptions Stewart AL, reading for week 1. Health-related quality of life?

28. 28 MOS Framework: Domains are Indicators of Physical and Mental Health Physical Mental Indicators Health Health Physical functioning X Pain X Energy/fatigue X X Sleep problems X X Cognitive functioning X Psychological distress/well-being X Social activity limitations due to health X X Role limitations due to physical health X Role limitations due to emotional problems X Current health perceptions X X

29. 29 MOS Framework: Physical and Mental Health are “Latent Variables” Physical health Mental health

30. 30 Physical Health MOS Physical Health: Dimensionality Physical functioning Health perceptions Pain Energy & fatigue Role limitations due to physical health

31. 31 Methods For Developing Concepts u Qualitative (formative research) u Mixed methods (iterative process of qualitative and quantitative methods to develop and test measures)

32. 32 What are Qualitative Methods? u Data consist of words (text), not numbers u Questions are open-ended u Focus on inductive analytic approaches u Many types: ethnography, participant- observation, direct observation, focus groups, in-depth interviews

33. 33 Focus Groups u Open-ended guided group discussion with probing of responses u Listen to perspective of likely “subjects” –How they discuss “concept” in their own words u Participants stimulate comments of others

34. 34 Focus Group Methods u Led by experienced moderator –usually 1 - 2 hours u Purposeful sampling of 6-10 homogenous participants per group u Audio-record and transcribe discussion

35. 35 Example: Developing Concept of Expectations Regarding Aging (ERA) u Construct thought to reflect how older adults think about aging –Do they expect to decline on various dimensions? –Would their expectations affect decisions about health and health care?

36. 36 Concept Development ERA : Qualitative Methods u Conducted 5 focus groups of older adults u 10 vignettes describing older adults experiencing common age-associated changes. Probed “does this describe changes you would expect to occur with aging?” u Analyzed transcripts using qualitative methods u Results: 26 “domains” Sarkisian CA et al. Med Care. 2003;39:1025-36.

37. 37 Examples of “Expectations About Aging” Domains u Physical function u Cognitive function u Social function u Pain u Sexual function u Personal autonomy u Loneliness u Appearance u Energy u Urinary incontinence u Falling u Happiness u Spirituality u Being treated differently

38. 38 Mixed Methods u Iterative process integrating qualitative and quantitative methods

39. 39 Combining Qualitative and Quantitative Methods in Concept Development u Prior to quantitative: to develop concepts, framework, hypotheses, and content for structured survey items or interventions u After quantitative: –to help identify reasons for survey items not performing well quantitatively –to explore possible explanations for unexpected results

40. 40 Mixed Methods: Iteratively Refine Concept Based on Quantitative Results Develop concept Create item pool Pretest/revise Field survey Psychometric analyses Final measures

41. 41 Concept Development of Interpersonal Processes of Care: Mixed Methods u Interpersonal processes between physicians and patients, emphasis on vulnerable patients u Basis for concept: –Review literature, satisfaction with care measures –Clinical experience of investigators u Initial conceptual framework included three broad domains: communication, decision making, and interpersonal style, each with subdomains Stewart et al., 1999 in readings

42. 42 Concept Development of Interpersonal Processes of Care: Mixed Methods u Second phase: –Items written to reflect all domains –Administered to diverse patient sample –Empirical scaling analyses identified 13 scales Stewart et al., 1999 in readings

43. 43 “Empirically Confirmed” Conceptual Framework of IPC I. COMMUNICATION III. INTERPERSONAL STYLE General clarity Respectfulness Elicitation/responsiveness Courteousness Explanations of Perceived discrimination -processes Emotional support -self care -medications Empowerment II. DECISION MAKING Responsiveness to patient preferences Consideration of ability to comply

44. 44 Pathway to First IPC “Empirically Confirmed” Conceptual Framework Develop concept Create item pool Pretest/revise Field survey Psychometric analyses Final measures ?? Stewart et al., Milbank Quart, 1999

45. 45 Second Round of IPC Studies u Dissatisfied with resulting framework –Not able to measure several concepts important to patients u Obtained funding for another round of measurement studies

46. 46 Mixed Methods: Revisit Original Concept Develop concept Create item pool Pretest/revise Field survey Psychometric analyses Final measures

47. 47 Developing Revised IPC Concept Draft IPC II conceptual framework IPC Version I framework in Milbank Quarterly 19 new focus groups - African American, Latino, and White adults Updated literature review of quality of care in diverse groups

48. 48 IPC: Second Conceptual Framework u Wrote additional items u Pretested, reduced item set u Administered to large sample of diverse patients u Analyzed data for best measures http://medicine.ucsf.edu/cadc/cores/measurement/ipc/IPCframework_4.13.06.pdf

49. 49 Pathway to Current IPC Conceptual Framework Develop concept Create item pool Pretest/revise Field survey Psychometric analyses Final measures Stewart et al., Health Serv Res, 2007 Stewart et al., Milbank Quart, 1999

50. 50 Concept Development: Cultural Sensitivity of Physicians u For minority patients and those with limited English proficiency –Sensitivity of providers to their cultural perspective hypothesized to be an important aspect of interpersonal processes of care u Hypothesized scale did not work in either of our IPC studies

51. 51 New Qualitative Analyses u 19 focus groups of ethnically diverse adults –Groups homogeneous on ethnicity, language (English/Spanish), gender, and age u Two open-ended questions: –What does the word culture mean to you? –What do your doctors understand about your culture that might affect your visits? u Transcripts analyzed using qualitative methods Nápoles-Springer AM et al. Health Expectations. 2005;8:4-17.

52. 52 Results: Multiple Dimensions of Cultural Factors Affecting the Medical Encounter Examples … u Values, beliefs, and attitudes u Communication including sensitivity to language access u Complementary and alternative medicine u Privacy and modesty u Ethnic concordance of physician and patient u Empowerment and submissiveness

53. 53 Overview of Class 1 u Concept development for new measures u Complexity of most concepts u Importance of defining concepts prior to selecting existing measures u Goal –Provide ideas for developing your own concept before considering measures

54. 54 Basic Steps in Selecting Appropriate Measures for Your Studies 1. Specify role of concept in research question 2. Define concept for your study 3. Specify context 4. Identify and review potential measures for a) conceptual and psychometric adequacy b) practicality and acceptability 5. Select best candidates 6. Pretest selected measures in your groups 7. Choose best ones based on pretest results OR 8. Adapt if necessary to address problems This week

55. 55 Role of Concept in Your Research u Evaluate intervention (outcome) u Predict an outcome u Describe population u Predict future event (e.g. health, use of emergency care) u Covariate

56. 56 Define Concept for Your Research u A methodological task before identifying appropriate measures u Need to clarify concept definition prior to selecting any measures –Define the concept YOU are interested in measuring

57. 57 Importance of Defining Concept u Distinction between “label” or name of a measure and what it actually contains u May find “good” measure of your concept –But mismatched »Measures something different than you need »Measures only part of what you need (missing elements)

58. 58 Define Concept u Define it first from your point of view –How you would define it based on your experience and understanding of your research question u For dependent variables –Describe how intervention or independent variables might affect it - types of changes you expect u For independent (predictor) variables –Describe how it might be predictive

59. 59 Example of Complexity of Concept u Shared decision making

60. 60 Shared Decision Making: Some Brief “Definitions” u Patient and physician share in the process of deciding on treatment u Patient takes active role in deciding u Physician encourages patient to take active role u Physician is responsive to patient preferences regarding participating in decision

61. 61 Varying Definitions Suggest Complex Concept Process of shared decision making MD propensity to encourage shared DM (participatory DM style) Patient desire for shared DM (active patient orientation)

62. 62 Varying Definitions Suggest Complex Concept Process of shared decision making MD propensity to encourage shared DM (participatory DM style) Patient desire for shared DM (active patient orientation)

63. 63 Definitions of “Participatory Decision- Making Style” u “Propensity of physicians to involve patients in treatment decisions by providing treatment options, a sense of control over treatment decisions, and a sense of responsibility for care.” –Kaplan SH et al., Med Care, 1995;33:1176 u “Propensity (of physicians) to offer patients choices among treatment options and to give them a sense of control and responsibility for care.” –Kaplan SH et al., Ann Int Med, 1996;124:497

64. 64 Varying Definitions Suggest Complex Concept Process of shared decision making MD propensity to encourage shared DM (participatory DM style) Patient desire for shared DM (active patient orientation)

65. 65 Process of Shared Decision Making in ICU: Several Dimensions u Information exchange: –MD provides medical information relevant to decision –Surrogate provides relevant information re patient’s values/preferences u Deliberation –MD and surrogate discuss which treatment option is most appropriate given patient’s values and condition u Treatment decision –MD and surrogate make clear what they think is correct treatment choice Douglas White, MD, Dept of Med, Div of Pulmonary and Critical Care

66. 66 Homework: Define Concept u Summarize briefly your research question u Choose one concept and clarify its role (e.g. dependent, independent variable) u Define concept from your point of view –Is it multidimensional? What are the dimensions? u Summarize 1-2 “state-of-the-art” definitions –How similar is your own definition to these?