Chapter Ten Research Bias Winston Jackson and Norine Verberg Methods: Doing Social Research, 4e

2© 2007 Pearson Education Canada The Nature of Bias Research bias: a preference or predisposition to favour a particular outcome thus indicating a systematic distortion of research conclusions may be inadvertent or intentional occurs in all disciplines and can influence all phases of a projects

3© 2007 Pearson Education Canada Bias in Research... In everyday life, we tend to seek corroboration of our preconceptions Helps us make sense of a complicated world Affirms our pet theories Science potentially blinds researchers because of the expectations of certain findings, or the belief in certain theories Like culture, science produces blinders e.g., view of women in development of psychology

4© 2007 Pearson Education Canada View of Women in Psychology Early research about women began with an assumption of women’s intellectual inferiority Instead of challenging that assumption, researchers sought ways to understand it Early research focused on: Gender differences in brain structure Role maternal instinct plays in maintaining women in passive, subservient role Evidence that culture influences science

5© 2007 Pearson Education Canada Sources of Bias Having ‘valid’ research results requires systematically testing alternative explanations Done to avoid researcher affect Researcher affect refers to the bias that results from a researcher having fallen in love with some particular explanation or view of the world May inadvertently use procedures that lead to conclusions supporting one’s preferred explanation or world view

6© 2007 Pearson Education Canada Sources of bias in the research process Bias can enter the research process at any stage of the research cycle Selection of the problem Sampling design Funding Data collection Data analysis Reporting the findings Use of the findings

7© 2007 Pearson Education Canada 1. Selection of Problem Some topics are judged as more important by funding agencies, one’s peers, community Results in selection of variables conventionally considered important There has been a bias toward quantitative approaches

8© 2007 Pearson Education Canada 2. Sampling Some samples will have a known slant Study of attitudes toward capital punishment done in a rural communities with a prison Problem of sexism Androcentricity presenting the world from an exclusively male perspective as if this perspective were universal

9© 2007 Pearson Education Canada 2. Sampling (cont’d) Overgeneralization When a study claims to study all people but samples only or largely males Study of social class – exclusively male, but generalized to the pop’n Related to overspecificity: when single-sex terms are used to describe situations applicable to both sexes “the nurse … she” ; “the doctor … he”

10© 2007 Pearson Education Canada 2. Sampling (cont’d) Gender insensitivity Ignoring gender as an important variable Familism – a form of gender insensitivity treating the family unit as the unit of analysis rather than the individuals in the family or assuming something affects all family members in a similar way

11© 2007 Pearson Education Canada 2. Sampling (cont’d) Double standard Using different measures for women and men Sexual dichotomism - treating the sexes as discrete social and biological cohorts rather than as two cohorts with shared characteristics

12© 2007 Pearson Education Canada 3. Funding Granting agencies claim to fund projects on the basis of merit, but are subject to bias Traditional areas have better funding Special funding available in “hot” areas Science and health research better funded than social science and humanities research Research done in a social context Peers and funding agencies value some topics and research methods more than others

13© 2007 Pearson Education Canada 4. Data Collection Experimenter effect: a tendency to produce findings that are consistent with the experimenter’s expectations Rosenthal’s “smart rats” study showed that research assistants’ knowledge of the hypothesis biased the results Clever Hans – horse who could count horse took its cues from the audience Called expectancy – results will be biased in the direction of the expected results

14© 2007 Pearson Education Canada 4. Data Collection (cont’d) What does this mean? The social researcher should avoid specifying the hypothesis of the study to either the respondent or the research assistant Research participants who know or guess the expectations of the researcher are likely to respond in the way that conforms to those perceived expectations – helpful subjects These distortions are called demand characteristics

15© 2007 Pearson Education Canada 5. Data Analysis Coding research results is subject to two types of error: random and systematic error Random error refers to inconsistencies that enter into the coding process but which display no systematic pattern Not threatening to the results of the study Systematic errors refer to errors that distort the data in one particular direction e.g, how to deal with missing data Influences (biases) the results in one direction

16© 2007 Pearson Education Canada 5. Data Analysis (cont’d) Data massaging – researchers may manipulate the coding of the variables to influence results Hunting for results – new computer technology makes it easy to test various hypotheses with different operational definitions for the variables Nothing inherently wrong with “exploring” the data however, social scientists need to assess whether coding influenced results and report data massaging

17© 2007 Pearson Education Canada 6. Reporting of findings Journals prefer to publish positive results (results that support the hypothesis) Some non-scientific considerations influence whether results are published Are findings culturally acceptable? Insensitivity to minorities, sexism, etc. Are findings acceptable to one’s peers? Corporate interests? E.g., medical researchers must ensure they are not affiliated with a pharmaceutical company

18© 2007 Pearson Education Canada 7. The use of findings The findings of science are powerful tools Scientific evidence taken seriously by courts, politicians, the media, the public Unfortunately, the literature is easily misrepresented One may present results that support their position, ignore contrary results “Scientist-for-hire” – may present results desired by the person paying for their services

19© 2007 Pearson Education Canada A Perspective: Understanding Bias Research is social behaviour Expectations of others to be met, norms of behaviour to be followed, and findings to be anticipated The social component of science is frequently at odds with the fundamental canons of science Science as practice is neither value-free nor wholly objective Impressive achievements of science offset by gaps between the ideal and practice

20© 2007 Pearson Education Canada Myths and Realities FPO Table 10.1 Myths and realities of the social science, page 296

21© 2007 Pearson Education Canada Rules for Minimizing Bias 1. Educate key players about problems of bias 2. Avoid sexism 3. Identify roles played by the research process 4. Eliminate bias in description phase of project 5. In exploratory research, let disconfirmation be your guide (test alternative hypotheses) 6. Policy recommendations should be identified as value-based 7. Be skeptical of research findings

22© 2007 Pearson Education Canada Rules (cont’d) 8. Read literature cautiously 9. Distinguish advocacy from pure research 10. Orient research to disconfirmation 11. Use theory to generate testable hypotheses 12. Be sensitive to your own outcome preferences 13. Do not disclose hypotheses to subjects or assistants 14. Cover the attitudinal continuum

23© 2007 Pearson Education Canada Rules (cont’d) 15. Be accepting of all responses 16. Specify data analysis procedures in advance 17. Check for random and systematic errors 18. Report extent of data massaging

24© 2007 Pearson Education Canada Social science afflictions: humous look at the problem of bias in social research Theoretical rigor mortis Methodological paralysis True believer fever Good cause syndrome Guerilla raider’s syndrome Scientism ailment Anti-science fever Replicationitis No affliction affliction

25© 2007 Pearson Education Canada Afflictions: Theoretical Rigor Mortis Characterized by a quest to do research which will support a favoured theory, those suffering from this affliction, when asked about it, protest their health, and continue to do research which is rigidly interpreted as supportive of a particular theory. The interest appears to be not in testing the theory's limits but to finding additional support for it. Cure: Do a project which puts the favoured theory at risk. Publish the results. Use your real name.

26© 2007 Pearson Education Canada Afflictions: Methodological Paralysis An inability to move to different research designs, measurement procedures, or analytical techniques. People who suffer from this affliction spend their time in locating problems that can be done using familiar design and measurement procedures. Cure: Conduct a study using a design, measurements, and statistical procedures never used before. Enjoy it. Understand it..

27© 2007 Pearson Education Canada Afflictions: True Believer Fever Whether an hypothesis or pet finding, these are the people always in search of evidence to support their particular view of the world; strongly challenge competing evidence, ruling it as irrelevant. Cure: Write a paper enumerating the flaws in your perspective. Mean it.

28© 2007 Pearson Education Canada Afflictions: Good Cause Syndrome These are the researchers with a cause: social science is used where its findings can be made to support the preferred view. Evidence to the contrary is dismissed as methodologically or theoretically flawed. These are the researchers who inevitably show up to help the underdog – or the overdog. They may also suffer from terminal liberalism, terminal political correctness, or terminal conservatism – all for a good cause, all in the name of virtue, all in the name of social science. Cure: Write an essay defending cannibalism. Try to believe it.

29© 2007 Pearson Education Canada Afflictions: Guerrilla Raiders Syndrome These patients typically do not do original research. Instead they fashion a career out of attacking a particular theoretical or methodological approach. This syndrome is often jointly found in those who suffer from True Believer Syndrome. Cure: Do a primary research project. High cure rate.

30© 2007 Pearson Education Canada Afflictions: Scientism Ailment The belief that if we follow the models of the physical sciences, that we will eventually understand social behaviour and it will become as predictable as are relations in engineered physical systems. Cure: For each day during the next month, predict the weather 10 days in advance. Check performance.

31© 2007 Pearson Education Canada Afflictions: Anti-Science Fever Characterized by the belief that social behaviour cannot be understood using orientations borrowed from the physical sciences. Anger is directed at the science practitioners who are favoured in our culture. Cure: Marriage to someone with the Scientism Ailment is recommended. Check frequently: there is a danger of a double murder.

32© 2007 Pearson Education Canada Afflictions: Replicationitis This terminal boredom condition is characterized by the researcher who keeps replicating the same study—with minor variation in samples, or in the variables involved. This is the person who makes a career out of studying the same set of variables. Incessantly. Cure: Cut off funds. Get a hobby.

33© 2007 Pearson Education Canada Afflictions: No Affliction Affliction These are the social scientists who think that their research is unbiased—untainted by brushes with their socio-cultural milieu. Cure: not treatable since the patient is in strong denial. Pray.