1. 1 Adapting Research Methods for Diverse Ethnic Groups Eliseo J. Perez-Stable, M.D. Anita Stewart, Ph.D. Anna Nápoles-Springer, Ph.D. University of California, San Francisco Clinical Research with Diverse Communities EPI 222, Spring 2002

2. 2 Overview u Integrating qualitative and quantitative methods u Using focus groups to develop questionnaires and community interventions u Using cognitive interviewing to develop questionnaires

3. 3 What are Qualitative Methods? u Data consist of words, not numbers u Richly descriptive, open-ended u Focus on inductive analytic approaches u Many types: ethnography, participant- observation, direct observation, interviews, focus groups

4. 4 When are Qualitative Methods Useful? u Open-ended interviews typically used in quantitative research in new areas of study u Especially critical in cross-cultural studies due to lack of information u Useful when need in-depth knowledge about issues especially with less studied cultural groups

5. 5 When are Qualitative Methods Useful? u To understand the meaning of participants’ events, situations, and actions u To understand contextual influences on participants’ actions u To identify unanticipated phenomena and influences (e.g., exploratory studies to design questionnaires and identify variables for study) u To understand the processes underlying observed relationships between variables

6. 6 Combining Qualitative and Quantitative Methods u Qualitative Methods to Develop Quantitative Tools u Qualitative Methods to Explain Quantitative Results Qualitative Results Quantitative Results Qualitative

7. 7 Combining Qualitative and Quantitative Methods cont. u Quantitative Methods to Expand on Qualitative Study u Quantitative and Qualitative Methods Equal QualitativeResults Quantitative Qualitative Results Quantitative (Tasjakkori A. Sage Publications, Inc. 1998)

8. 8 Sampling In Qualitative Research u Quantitative research-2 types of sampling: –Probability –Convenience u Qualitative research-3 rd type of sampling: –Purposeful »Deliberately select settings, persons or events to best answer research questions

9. 9 Goals of Purposeful Sampling u Achieve representative, typical settings, individuals, or activities u Adequately capture heterogeneity, sample for broadest range of variation rather than typical individuals u Examine extreme cases that test theory u Set up contrasts to examine differences between settings or individuals (Maxwell JA. Sage Publications, Inc., 1996)

10. 10 Qualitative Methods to Develop Quantitative Tools u Focus Group- –Open-ended guided group discussion with probing of responses u Cognitive Interviewing –Individual interviews using open-ended probes to assess how items are interpreted and adequacy of response choices

11. 11 Open-ended Interviews to Develop Structured Questionnaire Example: Constructing attitudinal scales u Use open-ended interviews (individual or group) to obtain statements on topic of interest u Organize statements into “item pool” u Pretest closed-ended items u Modify items based on pretest and administer in structured interview

12. 12 Overview u Integrating qualitative and quantitative methods u Using focus groups to develop questionnaires and community interventions u Cognitive interviewing to develop questionnaires

13. 13 Focus Groups u Group discussion led by experienced moderator usually 1.5 - 2 hours u Purposeful sampling of 6-10 homogenous participants per group u Use open-ended questions with follow-up probes for clarification, usually covering 10-12 topics u Participants stimulate comments of others u Audio-record and transcribe discussion

14. 14 Focus Groups-Logistics u Moderator skills: listening, communication, negotiation, cultural similarity to participants u Costs of group: $600 - $1000 per group (incentives, audio-taping, transcription, translation, food) u Convenient and hospitable community setting u In-person recruitment with telephone and mail follow-up

15. 15 Latino Smoking Attitudes - From Focus Groups to Community Intervention u Family concerns - 2nd hand smoke u Heightened health issues u Importance of appearance and interpersonal relations - Simpatía u Habitual use less important u Addiction less of a concern

16. 16 The Subjective Culture of Smoking: Focus Groups u Examine similarities and differences in attitudes, beliefs and behaviors (Triandis) u Sampling stratified by comparison groups u Probability vs. purposeful sampling u Antecedents and consequences u Use information to design intervention

17. 17 The Subjective Culture of Smoking - Structured Questionnaire u 17 antecedents to smoking (habitual activities, social activities, emotional states) u 15 reasons to quit (family, appearance, health) u 3 reasons to keep smoking (weight, nervousness, concentration)

18. 18 Reasons for Trying to Quit Smoking Latinos and Anglos, 1990 Latinos Anglos Difference 95% CI diff Effect on others62.8 51.7+11.1 (+18.9, +3.3) Smell51.6 33.7+17.9 (+25.6, +10.2) Cost50.6 48.6+2.0 (+9.9, –5.9) Wrinkles49.0 33.9+15.1 (+22.8, +7.4) Criticized by family 43.9 24.0+19.9 (+27.3, +12.5) Family pressure40.7 23.4+17.3 (+24.6, +9.9) Burn clothes40.1 25.1+15.0 (+22.4, +7.6)

19. 19 Reasons for Continuing to Smoke Latinos and Anglos, 1990 Latinos Anglos Difference 95% CI diff Not to gain weight19.9 26.1 –6.2 (+0.5, –12.8) Feel less nervous49.4 36.8 +12.6 (+20.4, +4.8) Helps concentration28.8 29.7 –0.9 (+6.3, –8.1)

20. 20 Multivariate Model for Latinos Reasons to Quit or Continue OR95% CI Criticized by family1.93(1.26, 2.98) Burn clothes1.57(1.02, 2.42) Children's' health1.67(1.08, 2.57) Bad breath2.07(1.40, 3.06) Family pressure1.69(1.10, 2.60) Good example1.83(1.21, 2.76) Not to gain weight0.38(0.24, 0.59)

21. 21 Community Intervention: Programa Latino Para Dejar de Fumar u Guia Para dejar de Fumar--self-help guide u Electronic media in Spanish u Emphasis on family and collective orientation of culture u Immediate effects of smoking u Health effects in different context u Physicians role

22. 22 Overview u Integrating qualitative and quantitative methods u Using focus groups to develop questionnaires and community interventions u Cognitive interviewing to develop questionnaires

23. 23 Distinction Between International and U.S. Studies - Why Pretest? u International studies assume conceptual non- equivalence to begin with –Different nations, languages u Usually dealing with translated measures u During translation, items can be added or modified to improve conceptual and semantic equivalence –Product is an “adapted” instrument

24. 24 Typical International Approach Assess Conceptual Equivalence (Qualitative) Assess Psychometric Equivalence (Quantitative) Begin here (assumes conceptual differences across countries) If new domains or definitions are found, can revise and add items Translated “adapted” version is the goal Assures conceptual adequacy prior to testing psychometric adequacy

25. 25 Typical U.S. Approach in Studies of English Speaking Diverse Groups u Select existing well-tested measures (developed in mainstream) and assume they will work (universality) u Assumes perspectives of diverse group are similar to mainstream –“Cultural hegemony” (Guyatt) –“Middle-class ethnocentrism (Rogler)

26. 26 Typical U.S. Subgroup Approach When No Translation is Done Most studies begin here (assumes universality of constructs) If problems If equiv. Assess Conceptual Equivalence (Qualitative) Assess Psychometric Equivalence (Quantitative) No Guidelines! If refine items based on qualitative studies, no longer have comparable instrument Proceed with analysis. May miss important domains and definitions

27. 27 Special Script for Recruiting People for Cognitive Interviewing u Often do not understand their role u Explain how their help fits into the larger study, goal of main study, process of creating questions u Explain their role clearly: –“help us learn how to ask better questions” –“help us make questions clearer for others”

28. 28 Cognitive Interviewing: Purposes of Probing Questions u To learn.... if respondents understand the words and phrases the way you intended (meaning)?.. about the process of answering the questions.. about usefulness of response choices -Whether response choices are adequate -How they use the response choices.. whether item might be hard or unacceptable u Results can be used to revise items

29. 29 Cognitive Interviewing: Example of Probing Questions Meaning of words/phrases and questions u What does the word _______ mean to you? u What does the phrase ________ mean to you? u I asked you ______. What did you think of when I said ______?

30. 30 Sample Result: Probing the Meaning of a Phrase I’m going to ask you questions about how the office staff treated you personally …. What does the phrase “office staff” mean to you? “the receptionist and the nurses” “nurses and appointment people” “the person who takes your blood pressure and the clerk in the front office” (We intended receptionist and appointment staff)

31. 31 Sample Result: Probing the Meaning of a Phrase I asked you how often doctors ask you about your health beliefs. What does the term “health beliefs” mean to you? “.. I don’t want medicine” “.. How I feel, if I was exercising…” “.. Like religion? --not believing in going to doctors?” We changed the question to “personal beliefs about your health”

32. 32 Sample Result: Probing the Meaning of a Phrase (Pasick et al. 2000) During the last 12 months, how many times have you visited a doctor or other health professional just for a checkup (physical examination) - even when you were feeling well. u Chinese women: why would you go to a doctor if you were not sick? u Latina women: could not rephrase the question u African American women: hypertension so prevalent, could not distinguish “monitoring” from general checkup

33. 33 Probe on Difficulty in General u Can ask respondents whether they think others would have difficulty answering a question or would answer the question honestly

34. 34 Example of Probe on Difficulty: CES-D Item “During the past week, how often have you felt that you could not shake off the blues, even with help from family and friends” u Probe: Do you feel this is a question that people would or would not have difficulty understanding? –Latinos more likely to report people would have difficulty than other groups TP Johnson, Health Survey Research Methods, 1996

35. 35 Example: Probing the Process of Answering u I asked you _____ and you answered____. –Why did you pick this answer? –What were you thinking of when you picked this answer? –Can you tell me what you were thinking when you answered this way? »Can you give me some examples? u What came to mind when I asked you _____?

36. 36 Sample Result: Probing the Process of Answering When I asked you how often doctors gave you a chance to say what you thought was important, you answered “rarely” - what were you thinking of when you picked your answer? u “Sometimes I would be slow thinking, by the time I ask something, they are gone.... my doctor knows how to exit that door.”

37. 37 Example: Use of Response Scale u Do diverse groups use the response scale in similar ways? u Sample Result: on questions about cultural competence of providers….. interviewers reported that Asian respondents who were completely satisfied did not like to use the highest score on the rating scale CPEHN Report, 2001

38. 38 Sample Result: Use of Response Scale u In an exercise class of Samoans, instructor asked them to rate the difficulty of the exercise he just did on a 1-10 scale u They did not understand what a 1-10 scale was

39. 39 Sample Result: Learning about Acceptability Comments during interview to question “How often did doctors ignore your feelings?” “What do you mean ignore my feelings? Does that mean I’m bawling in front of him?” “What type of feelings?..when they ask “how are you feeling? Other kinds of feelings?” “This type of question would only be asked in extreme cases. This question is too intimate.”

40. 40 Other Cues to Problems in Face-to- Face Pretests When administering the survey… u Be aware of behavioral cues related to specific items or to questions in general –Long pauses in answering –Discomfort –Yawning –Looking at their watch

41. 41 Interviewer’s Role u Be flexible during interview –probe on items that appear to be problematic u If a long pause in answering.. –“I noticed you pausing - what were you thinking about answering that question?”

42. 42 Approaches to Adapting Standard Measures u Add parenthetical phrases where words are hard for target group to understand u Substitute more culturally appropriate examples –e.g., limitations in moderate activities such as playing tennis - substitute soccer u If items need substantial adaptation, –Administer new and “standard” items –Can analyze measurement properties of standard measure and “adapted” measure

43. 43 Tradeoffs of Using Adapted Measures u If “adapted” measure works better… –You improved internal validity - able to answer your question with this measure –You lost external validity - can’t compare your scores to other studies u If “adapted” measure does not work … –Can still use original measure »It may not work either

44. 44 What to do if Measures Are Not Adequate or Equivalent in a Specific Study u Need guidelines for how to handle data when substantial non-comparability is found in a study –Drop bad or biased items from scores »Compare results with and without biased items –Analyze study by stratifying diverse groups u The current challenge for measurement

45. 45 Conclusions:Integrating Quantitative and Qualitative Methods u In diverse populations, qualitative work is necessary in addition to more traditional quantitative studies u Prior to quantitative: to develop concepts, items appropriate to culture u After quantitative: –to help identify reasons for items not performing well quantitatively –to explore possible explanations for unexpected results