1. Cross Cultural Research
Dr. K. A. Korb
University of Jos
15 May 2009

2. OUTLINE Issues in Cross-Cultural Research
Validity
Participant
Instrument
Types of cross-cultural Research
Dr K. A. Korb
University of Jos

3. Validity
Validity: The degree of accuracy with which a conclusion is drawn
Researchers have a burden to demonstrate that their research finding have validity
The validity of a study’s finding is directly related to the validity of the research methods
Interpretive validity: Valid communication between the research and target group
Dr K. A. Korb
University of Jos

4. Validity
Ecological validity: Extent that findings generalize beyond the research procedure to the natural context
Theoretical (construct) validity: Establishing valid operational measures for the concept being studied
This related to:
Independent variables
Dependent variables
Dr K. A. Korb
University of Jos

5. Research Example Theory: Research Study: Theory: IV: Regular Reading
DV:
Reading
Achievement
Theory:
Read book
every day with
an adult for
10 min.
Scores on
reading section
of classroom
exams
Research Study:
IV:
Regular
Reading
DV:
Reading
Achievement
Theory:
Dr K. A. Korb
University of Jos

6. Validity
Equivalence: For a cross-cultural comparison to be meaning, two condition must be met:
Equivalence in the conceptual meaning of the theoretical constructs between cultures
Equivalence in the relevance of the empirical method between cultures
If any aspect of a research study is not equivalent across cultures then the comparison not valid
Dr K. A. Korb
University of Jos

7. Validity Types of Equivalence
Theoretical:Equivalent in meaning of theoretical framework tested and specific hypotheses tested
Linguistic: Words used in the instruments are similar across languages
Measurements: Instruments are equally valid and reliable across cultures
Sampling: Samples are representatives of their culture and equivalent on noncultural demographic characteristics
Procedure: Procedures used to collect data across cultures are similar
Dr K. A. Korb
University of Jos

8. Participants Cultures to be studied Subgroups within the cultures
Differ on independent variables of interest
Check to confirm that the cultures do differ on the independent variable
Subgroups within the cultures
Should be as similar as possible on non-cultural relevant demographic characteristics
Individuals within the subgoups
Dr K. A. Korb
University of Jos

9. Instruments Perspective 1: Use similar instruments for each culture
Assumption: The instrument is not biased against a particular culture
Conclusion: Differences between groups reflect construct differences in each culture
Advantage: Comparison is straight forward
Disadvantage: The instrument might not be culturally meaningful for all cultural groups
Dr K. A. Korb
University of Jos

10. Instruments Perspective 2: Use different instruments for each culture
Assumption: The same instrument cannot be equivalent across culture because psychological constructs are context-bound
Conclusion: Difficult interpretation on comparisons across cultures
Advantages: Measures behaviors that are more appropriate for each culture
Disadvantage
Uncertain whether the instruments are measuring conceptually similar constructs
Comparison between cultures is difficult
Dr K. A. Korb
University of Jos

11. Instruments
Linguistics Equivalence: Can be established through back-translation of an instrument
Step 1, Forward Translation: Translate from English into Language
Step 2, Back Translation: An independent person translates the translation back into English.
The instrument is considered equivalent in both languages if the back-translation is equivalent to the original
Dr K. A. Korb
University of Jos

12. Instruments
Cultural Bias: The instruments does not measure equivalent concepts in both cultures
Theoretical/construct bias: The definition of the concept differs between cultures
Item bias: Poor item translation
Procedural bias: Differences in testing procedures across cultures
Testing conditions
Family with testing procedures
Response styles
Dr K. A. Korb
University of Jos

13. Instruments
Response Style: Systematic tendency to respond in a certain way to items on a questionnaire
Acquiescence bias: Agree with statements as presented
Central tendency bias: Avoid extreme categories
Social Desirability: Tendency to portray oneself in favorable light
Reference Group Effect
Participant self-report by comparing themselves to others
Individuals with different cultural will have different standards of comparison
Dr K. A. Korb
University of Jos

14. Reference Group Effect
Standard item: I have respect for the authority figures with whom I interact.
Cultural comparison item: Compared to most Japanese I know, I think I have respect for the authority figures with whom I interact.
Dr K. A. Korb
University of Jos

15. Reference Group Effect
Dr K. A. Korb
University of Jos

16. Control of External Variables
To reduce alternative explanations of differences between cultural groups
Cultural populations should be selected a priori (before the experiment) based on ethnographic descriptions
Measure the dependent variable by two or more measures
Eliminate effect of irrelevant variables though statistical analysis (ANCOVA)
Dr K. A. Korb
University of Jos

17. Research Designs
Causal- Comparative: Compare groups of people on an independent variable to determine the effects of an independent variable that cannot be manipulated
Correlational: Compare one group of people on multiple variables to determine the relationship between variables.
Experimental: Compare treatment and control groups on a dependent variables. The experimenter must implement a treatment.
Dr K. A. Korb
University of Jos

18. Cross-Cultural Research
Cross- Cultural Comparisons: Studies comparing different cultures on the same psychological variable
Determine how individuals within cultures differ on psychological variables of interest
Cultural Studies: Studies based on a theoretical framework that predicts and explains differences between cultures
Correlation studies: Studies that determine whether the relationships between psychological variables are similar across cultures
Linkage studies: Studies that establish links between contents of culture and psychological variables of interest
Unpacking studies
Experiments
Dr K. A. Korb
University of Jos

19. Unpaking Studies
Instead of culture as independent variable, a Context Variable is hypothesized to influence the dependent variable
Context variable: Specific variable that explains cultural differences that then influences the dependent variable
Both the context variable and the dependent variable are measured
Degree to which the context variable influence the dependent variable is statically tested
Dr K. A. Korb
University of Jos

20. Cross-Cultural Experiments
Priming Experiment: Experimentally manipulate mindsets related to culture to determine whether behavior differs as function of a primed mindsets
Example (Trafimow,Triandis,& Goto, 1991)
Treatment 1: Private Mindset
Please think of what makes you different from your family and friends.
Treatment 2: Collective Mindset
Please think of what you have in common with your family and friends. What do they expect from you to do?
Participants were then asked to write a paragraph that described themselves
Dr K. A. Korb
University of Jos

21. Frequency of Individually-Oriented and Group-Oriented Responses for American and Chinese (Trafimow et al., 1991)
Dr K. A. Korb
University of Jos

22. Conclusion
Only compare populations that have theoretical reasons for differing on a dependent variable
Replace culture with specific independent variables that might influence the dependent variable
What aspects of the cultures cause differences in the dependent variable?
Dr K. A. Korb
University of Jos

23. Revision
What validity consideration need to be taken into account in cross-cultural research studies?
What are some key issues to consider when developing instruments in cross-cultural research studies?
Describe research designs that are useful in cross- cultural research studies
Dr K. A. Korb
University of Jos