Download presentation
Presentation is loading. Please wait.
1
Psychology 242, Dr. McKirnan.
3/10/14 Survey research ✓ Topic areas & formats Testing Hypotheses General issues in Survey research Sources of bias (or fraud…) Examples of surveys
2
Survey research; General uses of surveys
Survey methods have a wide range of applications, from single-item consumer satisfaction (“How useful did you find this web site”) to full-fledged, theory-driven behavioral research. For convenience we will consider 5 categories: Descriptive research Testing hypotheses; Testing the generalizability of experimental results; Predicting an event or outcome; Pragmatic / applied questions.
3
Uses of surveys; descriptive research
Epidemiology is the study of how behaviors, disease states, or similar issues are distributed across the population. Epidemiology uses many methods, such as standard crime or disease reporting. Even Google search data (Google Trends) can be to track, e.g., heroin overdose. Many epidemiological studies use direct survey methods, such as phone or face-to-face survey interviews. Knowledge of, e.g., how to access health care… Feelings or moods, such as the rate and distribution of depression… Behavioral patterns, such as alcohol or drug use or gun ownership…
4
Uses of surveys; descriptive research
Epidemiology is the study of how behaviors, disease states, or similar issues are distributed across the population. Epidemiology uses many methods, such as standard crime or disease reporting. Even Google search data (Google Trends) can be to track, e.g., heroin overdose. Many epidemiological studies use direct survey methods, such as phone or face-to-face survey interviews. Knowledge of, e.g., how to access health care… Feelings or moods, such as the rate and distribution of depression… Behavioral patterns, such as alcohol or drug use or gun ownership…
5
? Counting Google searches to assess social interest
Do Chicago Cubs’ quest for the World Series inspire national interest? Google Search data during the MLB Division playoffs, 2016: Interest in the LA Dodgers is limited to California & Nevada; Interest in Cleveland is limited to Ohio; Interest in the Toronto Blue Jays is limited to Canada; The Cubs truly arouse national attention. ?
6
The origins of epidemiology
Dr. John Snow’s Cholera map & the closing of the Broad St. pump. In London of 1854 a cholera outbreak raged through several poor neighborhoods of London. Sewage and other effluvia that ran through gutters created a dreadful smell (a “miasma”) that was blamed for the outbreak. The concept of infectious disease transmission through water supplies was not well understood. Click for Wikipedia article. Dr. John Snow, one of the physicians charged with stopping the epidemic, noted a particularly fetid cesspool in front of 40 Broad St., proximal to a water pump used by the neighborhood. He decided to empirically map the cholera cases in the area. He proposed that water from the pump, not the miasma, was the cause of the outbreak. He was was generally disbelieved, but convinced the town governors by his evidence. Only much later would tracking of disease outbreak be labeled ‘epidemiology’. Image:
7
The origins of epidemiology
Dr. John Snow’s Cholera map & the closing of the Broad St. pump. Snow’s map showed the bulk of cases to be near the pump at 40 Broad St…. …and to radiate out from there. As he noted in his 1855 book: Click for Ted talk by Steven Johnson. "I had an interview with the Board of Guardians of St. James's parish, on the evening of Thursday, 7th September, and represented the above circumstances to them. In consequence of what I said, the handle of the pump was removed on the following day.” S. Johnson, The Ghost Map (2007), Riverhead Books. By carefully describing the distribution of cases and the circumstances around the pump, Snow was able to empirically demonstrate a likely cause. His hypothesis was supported by the epidemic quickly subsiding once the pump handle was removed.
8
Uses of surveys; descriptive research
Epidemiology Political / social description is what we often think of as surveys. Opinion polls about society, the government, or current events, e.g. Gallup Polls, or systemic studies by Pew Memorial Trust. The Consumer Confidence Index is a highly standardized poll that is used for basic economic decision making. The Census, of course, is our national information source.
9
Uses of surveys; descriptive research
Epidemiology Political / social description. Testing hypothesis Assessing blocking variables We often assess blocking variables to test how a given attitude or behavior varies across important social groups. Correlational studies A key form of analysis is examining the association among different variables e.g., gender, age group, ethnicity, geographic location… e.g., what are the correlates of dieting…
10
Uses of surveys; descriptive research
Epidemiology Political / social description. Testing hypothesis Assessing blocking variables Correlational studies Examine generalizability of experimental results E.g., The Consumer Reports survey on therapy we discussed in quasi-experiments Predict event or outcome; E.g., election polling Pragmatic / applied, E.g., marketing, or consumer surveys.
11
What do surveys measure?
Knowledge Information re: current events, political or consumer choices Awareness of Public health resources, health practices, etc. Attitudes and Beliefs Preferences or evaluations: e.g., attitudes toward gays, ethnic groups, etc., consumer preferences. Beliefs about political or social events: “which party provides the strongest security for the U.S….?” Feelings or moods: quality of life, depression / anxiety, marital satisfaction, etc. Behavior Behavioral intentions; Intent to vote, financial plans, etc. Self-reports of previous or on-going behavior; topics range from voting to alcohol and drug use.
12
Who do we want to generalize to ?
Surveys; populations Who do we want to generalize to ? Our sampling frame is based on our hypothesis or empirical question. Sampling: breadth internal validity tradeoff Key dimensions: Demographic ethnic / age / gender groups, “all Americans”… Behavioral “likely voters”, alcohol users, home buyers... Self-identification Republicans / Democrats, “students”…
13
Question Formats: Closed-ended items
Specific rating scale or highly structured prompts Most reliable for concrete behaviors An attitude can be assessed in several ways: Direct (face valid) assessment Psychology 242 is a wonderful course… Does not strongly agree at all agree Behavioral (content valid) indictors How many times this semester have you skipped 242 class? ______ How many hours per day do you spend reading 242 material? ______ 2.5 Researchers typically use the M of several related items to create a more reliable measure of a variable.
14
“Closed-ended” items, cont.
Example: CES-D Depression inventory: We may use the M score of these 9 items as our depression index… Moods & Feelings Below is a list of different feelings. Circle the number that shows how many days you felt each of these over the PAST WEEK. Rarely or A Little A moderate Most or all of none of of the Time amount of the time the time the time (less than 1 day) (1 or 2 days) (3 - 4 days) (5 - 7 days) I was bothered by things that usually do not bother me. I felt I could not shake off the blues even with help from my friends or family. I had trouble keeping my mind on what I was doing. I felt depressed I felt that everything I did was an effort My sleep was restless I was happy I enjoyed life I felt sad Sum of item ratings / 9 These items are “reversed” in the final score
15
“Closed-ended” items, cont.
Or we may count the number of symptoms We may have a cut point: e.g., moderate depression is defined as 4+ symptoms Moods & Feelings Below is a list of different feelings. Circle the number that shows how many days you felt each of these over the PAST WEEK. Rarely or A Little A moderate Most or all of none of of the Time amount of the time the time the time (less than 1 day) (1 or 2 days) (3 - 4 days) (5 - 7 days) I was bothered by things that usually do not bother me. I felt I could not shake off the blues even with help from my friends or family. I had trouble keeping my mind on what I was doing. I felt depressed I felt that everything I did was an effort My sleep was restless I was happy I enjoyed life I felt sad # of symptoms: items rated 2 or 3
16
Closed-ended items, summary
Chief virtue: clear operationalization Specific & concrete; we know exactly what the participant is responding to Easy to quantify & use statistically Can be tested for reliability Chief liability: potential insensitivity Brief & simply worded; potentially superficial “Top down”; issues are imposed on the participant Discrimination studies: no option for “has no attitude” Attitudes / moods: not sensitive to participants’ personal perspectives (?).
17
Survey formats; “Open-ended” items
General textual / qualitative response; More sensitive to the respondent How have you enjoyed your methods class so far? Please list the three things that first come to mind when you think of Psychology 242. More difficult to interpret Can be analyzed as qualitative data (see discussion in Descriptive data.) Can be quantified; frequency counts of citations or statements “linkages” analysis of co-occurring statements Often presented as textual portrayal plus minor quantitative analysis.
18
Example of mixed survey formats
Closed-ended attitude scale Open-ended description Simple behavioral index. Example of mixed question format from survey of women’s sexual practices. Personal Safer Sex Guidelines How strict are your personal guidelines or rules for safer sex (e.g., condom use, “safe relationships,” etc.)? Not at all Somewhat Very Extremely Strict Strict Strict Strict What are your rules for safer sex? Have you ever refused to have sex with someone to stay safe? Never once or a few many twice times times
19
Survey topics & item types
Surveys assess: Knowledge Attitudes or preferences Ongoing or intended behavior Closed-ended formats Highly structured, easy to analyze Potentially insensitive Open-ended formats More sensitive to the participant Potentially ambiguous or difficult to analyze Surveys typically… Use multiple items Employ several formats. S U M M A R Y Psychology 242 is a wonderful course… …list the three things that first come to mind…
20
Psychology 242, Dr. McKirnan.
Survey research Topic areas & formats ✓ Testing Hypotheses General issues in Survey research Sources of bias (or fraud…) Examples of surveys
21
Alcohol: Knowledge 1 What is the chemical in alcohol? a) ethanol b) zymase c) dehydrogenase d) nicotine Let’s do our own practice survey!
22
Which of these drinks contains the least alcohol?
Alcohol: Knowledge 2 Which of these drinks contains the least alcohol? A = 12 ounces of beer B = 5 ounces of wine C = 1 1/2 ounces of vodka D = 8 ounces of malt liquor E = All are equal
23
How long does it take for alcohol to affect the brain?
Alcohol: Knowledge 3 How long does it take for alcohol to affect the brain? a) 10 seconds b) 60 seconds c) 10 minutes d) 90 minutes
24
Drinking helps me feel whatever way I want to feel.
Alcohol attitudes, 1 Drinking helps me feel whatever way I want to feel. A = agree strongly B = agree C = somewhat agree D = slightly agree E = do not agree
25
Alcohol makes me feel happy.
Alcohol attitudes, 2 Alcohol makes me feel happy. A = agree strongly B = agree C = somewhat agree D = slightly agree E = do not agree
26
Drinking adds a certain warmth to social occasions.
Alcohol attitudes, 3 Drinking adds a certain warmth to social occasions. A = agree strongly B = agree C = somewhat agree D = slightly agree E = do not agree
27
How often do you have a drink containing alcohol?
Alcohol behavior, 1 How often do you have a drink containing alcohol? A = never B = monthly or less C = 2 to 4 times a month D = 2 to 3 times a week E = 4 or more times a week
28
Alcohol behavior, 2 How many drinks containing alcohol do you have on a typical day when you are drinking? A = 1 B = 2 C = 3 D = 4 or 5 E = 6 or more
29
Alcohol behavior, 3 How often during the last year have you found that you were not able to stop drinking once you had started? A = never B = less than monthly C = monthly D = weekly E = daily or almost every day
30
Alcohol behavior, 4 How often during the last year have you failed to do what was normally expected from you because of drinking? A = never B = less than monthly C = monthly D = weekly E = daily or almost every day
31
Knowledge / attitude / behavior composites
Here is our survey. Our data set: Several items assess each Hypothetical Construct. Typically we also collect demographics (age, gender, ethnicity…) and other Ψ variables. We choose items based on our Theory about alcohol use and problems. We can simply Describe our data in each category. We might Block the data by important categories of participants Or we can Test Hypotheses about correlations among variables Knowledge What is the chemical in alcohol Which of these drinks contains the least alcohol? How long does it take for alcohol to affect the brain? Attitudes and beliefs Drinking helps me feel whatever way I want to feel. Alcohol makes me feel happy. Drinking adds a certain warmth to social occasions. Behavior How often do you have a drink containing alcohol? How many drinks containing alcohol do you have on a typical day when you are drinking? How often during the last year have you found that you were not able to stop drinking once you had started? How often during the last year have you failed to do what was normally expected from you because of drinking? How many students use alcohol at all? Do men have a more positive attitude toward alcohol than women?
32
Knowledge / attitude / behavior composites
Research questions: How well do the items within each category correlate? Are they internally Reliable? Can we construct a Composite Variable from each item set? Blocking Variables: Gender? Ethnicity? Age?... What are the Correlations among composite variables? How much are knowledge or attitudes & beliefs associated with alcohol use? Are alcohol use and problems correlated? Are these effects Mediated by other variables? Do depression, isolation, or stress make people vulnerable to alcohol use or problems? Knowledge What is the chemical in alcohol Which of these drinks contains the least alcohol? How long does it take for alcohol to affect the brain? Attitudes and beliefs Drinking helps me feel whatever way I want to feel. Alcohol makes me feel happy. Drinking adds a certain warmth to social occasions. Behavior How often do you have a drink containing alcohol? How many drinks containing alcohol do you have on a typical day when you are drinking? How often during the last year have you found that you were not able to stop drinking once you had started? How often during the last year have you failed to do what was normally expected from you because of drinking?
33
Class data, n = 97 Example Multiple items assessed the Hypothetical Constructs of Alcohol- related information, attitudes, use, and problems. Information: # of correct answers for items 1 3. Attitudes: items 4, 5 & 6. Use: drinking frequency * # drinks / occasion (items 7 & 8). Problems: Items 9 & 10. Information: Mode = 2 items correct.
34
Class survey data, Attitudes
Example To test weather the 3 attitude items comprised a single Hypothetical Construct, we examine their intercorrelations All the correlations and high and statistically significant, so we can combine them into a single composite variable.
35
Class survey data, Attitudes
The frequency distribution for the composite attitude variable shows an approximate normal distribution. Example The Mean and Median are equal, near the center of the distribution The mode reflects a larger block of participants with moderately strong attitudes.
36
Class survey data, Alcohol problems
Example We perform the same operations for items 9 & 10, the alcohol problems measures. The correlation between these items is very high. We can validly combine them into one variable.
37
Class survey data, Alcohol problems
Example Most people report no problems, as shown in the Mean and Median The rest of the distribution shows a strong positive skew. This skew pulls the Mean higher than the Mode & Median.
38
Class survey data, Correlation results
The amount of information people have about alcohol is unrelated to their alcohol use or problems: The correlations are low; Neither is statistically significant. Positive attitudes toward alcohol are strongly associated with alcohol use. … and moderately (statistically significantly) with alcohol problems. Thus, attitudes are more important to alcohol abuse than is knowledge. Example
39
Correlations and mediators
With more data we could use mediating analyses to develop to test theories about alcohol use / problems Say we find that people who are lonely or depressed (negative affect) have more alcohol use & problems. Why might this be? People with negative affect may come to expect (hope?) that alcohol will make them feel better. The attitude may be what actually leads to alcohol use. Here attitudes / beliefs would mediate the effect of negative affect on alcohol use & problems. Depression loneliness stress Positive attitudes toward alcohol Alcohol use & Problems
40
Correlations and mediators
Using mediating analyses to develop to test theories about alcohol use / problems Say we find that people who are lonely or depressed (negative affect) have more alcohol use & problems. Why might this be? People with negative affect may come to expect (hope?) that alcohol will make them feel better. The attitude may be what actually leads to alcohol use. Here attitudes / beliefs would mediate the effect of negative affect on alcohol use & problems. Using a mediating analysis to ask “why” is the core step in Theory Development. Depression loneliness stress Positive attitudes toward alcohol Alcohol use & Problems
41
Correlations and mediators
Survey data are important for developing or testing new theories of behavior Can have high external validity Can model different relations among variables. Problems? The data are correlational and cross-sectional. The causal arrows may go in different directions. Depression loneliness stress Positive expectation of alcohol Alcohol use & Problems
42
Correlations and mediators
Survey data are important for developing or testing new theories of behavior Can have high external validity Can model different relations among variables. Problems? The data are correlational and cross-sectional. The causal arrows may go in different directions. Genetics: Positive alcohol effects & negative affectivity There may be a third variable that we did not measure. Depression loneliness stress Positive expectation of alcohol Alcohol use & Problems
43
Cures for correlation problems?
Longitudinal studies Examine changes over time to better determine cause and effect Experimental studies Simulate and manipulate the predictors, e.g.; Induce temporary stress / negative affect Assess attitudes Assess alcohol use
44
We Use Longitudinal data
To examine change To approximate “cause and effect” Research has shown that depression, loneliness & stress are correlated. So, we combine them into a single composite variable. This is our Predictor (Measurement version of IV) Depression Negative Affect Loneliness Stress The same for alcohol use and problems. This is our outcome (Measurement version of DV) Alcohol use Alcohol abuse Alcohol problems
45
Longitudinal data: Path analysis
We test the hypothesis that Negative Affect correlate with alcohol abuse & problems… Time 1 (baseline) Time 2 (follow-up) …at baseline …and at follow-up Negative Affect Negative Affect Data at each time frame are cross-sectional: We cannot determine if emotions “cause” the alcohol outcome, or visa versa. Alcohol abuse Alcohol abuse
46
Longitudinal data: Path analysis
Looking at patterns across time lets us make more sensitive statements. If the measures are reliable Negative Affect at Time 1 will correlate with NA at Time 2. Time 2 (follow-up) Time 1 (baseline) Negative Affect Negative Affect And the same for the alcohol variable Alcohol abuse Alcohol abuse
47
Longitudinal data: Path analysis
To test the hypothesis we look at the “cross” correlations… How strongly does Affect at baseline correlate with Alcohol at follow-up? Time 1 (baseline) Time 2 (follow-up) Negative Affect Negative Affect Alcohol abuse Alcohol abuse A strong correlation would tell us that Affect does lead to more Alcohol abuse over time.
48
Longitudinal data: Path analysis
To test the hypothesis we look at the “cross” correlations… Similarly, how strongly does Alcohol at T1 correlate with changes in Affect at T2? Time 1 (baseline) Negative Affect Alcohol abuse Time 2 (follow-up) Negative Affect Alcohol abuse This correlation tells us how much alcohol use leads to negative affect over time. .
49
Longitudinal data: Path analysis
Testing the hypothesis: does Negative Affect ‘cause’ Alcohol use & problems Time 1 (baseline) If the correlation between Affect at Time 1 and Alcohol at follow-up is very strong… Time 2 (follow-up) Negative Affect Negative Affect Alcohol abuse Alcohol abuse
50
Longitudinal data: Path analysis
Testing the hypothesis: does Negative Affect ‘cause’ Alcohol use & problems Time 1 (baseline) Negative Affect Alcohol abuse …and the correlation between Alcohol at T1 and Affect at T2 is relatively weak… Time 2 (follow-up) Negative Affect Alcohol abuse
51
Longitudinal data: Path analysis
Testing the hypothesis …we can conclude that Affect may be a more important “cause” of alcohol use / problems Time 1 (baseline) Time 2 (follow-up) Negative Affect Negative Affect than the other way around. Alcohol abuse Alcohol abuse
52
Longitudinal data: Path analysis
Testing the hypothesis Of course we may find that Alcohol “causes” Affect, not the other way around… Time 1 (baseline) Negative Affect Alcohol abuse Time 2 (follow-up) Negative Affect Alcohol abuse Either way, we have a better sense of cause and effect than with cross-sectional data.
53
Summary: Testing Hypotheses
Surveys typically use multiple items to measure each hypothetical construct Correlations among items tell us if they are reliable in measuring the same construct. We use Mediating Analyses to Test hypotheses about correlations between constructs Build or test theory Cross-sectional analyses are difficult to interpret Causal direction? 3rd variable problem Longitudinal analyses help us determine causal direction Summary
54
Psychology 242, Dr. McKirnan.
Topic areas & formats Testing Hypotheses General issues in Survey research Sources of bias (or fraud…) Examples of surveys
55
Forms of survey administration
Self-report questionnaire “Paper and pencil” or internet-based; Primarily closed-ended, structured questions Limited open-ended items Assume at least moderate reading level Cheap & easy to administer Internet: Representativeness very dubious Face-to-face interview “Door step”, formal research center, or telephone Allows in-depth qualitative questions Many studies combine questionnaire & interview formats (Telephone version becoming obsolete) All data collection increasingly computer-based
56
General issues in surveys
Cost / population access Different methods are more / less likely to reach certain populations, e.g.: Disfranchised / poor populations often not reached by internet or telephone Cell phones & avoidance of telemarketers less availability for telephone surveys Stigmatized populations less available for face-to-face interviews, more available via internet. Participant sophistication Participants may not be able to accurately report certain topics Attitudes toward stem cell research from readings. What factors are most important to your choice of political candidate.... Describe the amounts and types of proteins you eat during a typical week... “Rationality bias”; many questions (incorrectly?) assume a rational reason for behavior: Why do you have unsafe sex... What is your chief reason for using alcohol each night…
57
Social Desirability Responding
Clear face-valid items addressing embarrassing topics yield less valid responses How often are you dishonest with your friends? Have you ever cheated on an exam....? High social desirability wording elicits inaccurate responses… Do you support protecting our Nation’s forests for future generations? (Does “yes” mean you an “environmentalist”?). Do you feel there are ways your husband could be closer...? (Does “yes” make you are unhappy in your marriage”?). Populations differ in social desirability responding; may be a confound in studying group differences Women report more suicidal thoughts, but may be more willing to disclose, creating a possible confound… Desirability can be minimized by: Anonymous surveys Assurances of confidentiality Computer administration (no personal interaction) Careful wording / pilot testing of items
58
Surveys: Social desirability responding
Do people lie on surveys? Men routinely report more sex partners than do women. If the sample is unbiased by gender, partners should balance for men & women. Social desirability hypothesis: Women underestimate partners Men overstate partners Much of the difference due to: A high proportion of women who report 1 partner A few men who report many partners. Possible sample bias (confound?) in who responds to such surveys? Click for the article
59
General issues in surveys: Time Frames
Rare(er) events require a long time frame to assess When was your last doctor’s visit… These questions asses the last time you left a romantic relationship… Longer term recall can be surprisingly unreliable Recall of last doctor visit highly unreliable when checked against medical records Shorter time frame yields more reliable responding Memory is better for more recent effects “Exit interviews” from medical visits far more reliable than even 2-week retrospective measures. Current, concrete behaviors are more accurately reported than behavioral trends. In general, how often do you miss a dose of your medication Let’s go over each of the past 7 days and tell me if you took or missed your medication dose… Less reliable than…
60
General issues in surveys: Question Order
Questions trigger participant’s memory or attention, and can bias questions that follow, e.g.: Do you think Social Security & Medicare payments have kept up with inflation.. Do you favor or oppose Democratic efforts to expand Medicare payments... Bias can be limited by counterbalancing questions. Using different question orders in different versions of the survey. then
61
General Issues Summary
Survey administration Internet increasingly important as self-report method Face-to-face interviews more common in clinical research Time frames & question order can influence responses Population access & sophistication Some groups are difficult to reach Creates threat to External validity Assumption that participants understand survey materials often questionable. Social desirability responding Inhibited responding threatens Internal Validity May represent a confound if groups differ in desirability set. Summary
62
Psychology 242, Dr. McKirnan.
Topic areas & formats Testing Hypotheses General issues in Survey research Sources of bias (or fraud…) Examples of surveys
63
Bias / Fraud in survey research
Social research is increasingly important to political & cultural debates. Effects of gay marriage Political “approval” ratings Scientific consensus on global warming Research on working mothers … Pressure for confirmatory results encourages bias or outright fraud (see this week’s article on Opinion Polls). In the study structure Items used Sample In the interpretation of results “Cherry picking” Simple distortion
64
Example of fraudulent survey use
Opposition to gay marriage Judges have struck down religious doctrine as a basis for disallowing gay marriage Example April DeBoer, left, and Jayne Rowse on Friday after closing arguments in their challenge to Michigan’s marriage restriction. Credit Mandi Wright/Detroit Free Press, via Associated Press Opponents have sought evidence of civil harm to justify discrimination Harm to children has emerged as key issue in the debate APA; multiple studies show no harm. Mark Regenerus’ Survey: Children in households with gay/lesbian parent fare worse. Funded & cited widely by gay marriage opponents Study sample and interpretation of results wildly biased Has been disavowed by Renenerus’ Academic Department and American Sociological Assn. Continues to be central “evidence” in court cases
65
Bias in survey research: Leading or biased items
Source of survey fraud; question wording can elicit a response desired by the researcher; How much do you support the administrations’ actions to protect you and your children from terrorists… Wording can “normalize” a response, e.g., When do you feel that it is O.K. to cheat on an exam? ..when I really do not know the material .. when others are doing it .. when I think the exam is unfair Vague wording can be interpreted in a biased fashion Is there anything your husband could do to be more intimate with you? “Push” polls: a survey can be used to actually create an attitude.
66
Push Polls: An item “traps” the participant into endorsing a specific view. When the “data” are released the biased wording is ignored. Publicity about the “findings” (e.g., by politically biased news organizations) are used to further create or change attitudes. Many political & social organizations use this strategy to… Ostensibly measure attitudes objectively Use the “results” to influence popular opinion. Example of a highly biased survey: The Republican National Comittee health care survey. This “survey” attempted to create fear of health care reform. It got limited distribution, but is a great example of a Push Poll
67
GOP health survey (2009 – 2010) Some of these items are simple lies – or manipulations based on lies – designed to induce anti-health care attitudes… Other are powerful (and dishonest) emotional manipulations ✓ ✓ ✓ ✓
68
Forms of survey bias: Provide leading or emotionally manipulative information to induce an attitude rather than simply measure it, to provide politically useful “data”.. Questions that, if you accept their assumptions, can only be reasonably answered one way…
69
Biased surveys: Democratic example
An example from the Democrats, that is also used for fund-raising.
70
Democratic biased survey (2007)
Manipulative presentation of questionable information Simple emotional manipulation Distorted description that may be changed in presentation of findings “Who could disagree” item.
71
Summary: Manipulating attitudes by surveys
1. Ask manipulative or highly leading questions 2. Find high levels of agreement (and potentially change participants’ attitudes). 3. Publicize – and often distort or overstate – the “findings” via highly biased news sources 4. News reports themselves lead to attitude change among people who are uncertain or uninformed.
72
Psychology 242, Dr. McKirnan.
Topic areas & formats Testing Hypotheses General issues in Survey research Sources of bias (or fraud…) Examples of surveys By key elements of survey design
73
Examples of surveys & data, 1
Consumer reports survey of mental health care question Satisfaction with therapy. Differences between types of therapy. population Self-Identified group: U.S. mental health care users sample Self-selected convenience sample: Readers who got therapy & returned the survey, n=4000 data Attitudes & behavior Self-report questionnaire, cross-sectional findings Descriptive & hypothesis tests High satisfaction for most treatments
74
Examples of surveys & data, 2
“Monitoring the future” youth studies question Social behavior Academics Alcohol & drug use Health. population Demographic group: All U.S. youth, 15 -> 21 years old. sample Random sample: Sample of High School health classes, n=3000 -> 5000. data Knowledge, attitudes and behavior Face--to--face interviews & questionnaires, longitudinal (bi-yearly) findings Mostly descriptive Assess yearly trends/shifts in drugs, grades, emotional well being
75
Examples of surveys & data, 3
Gallup, Time/CNN, other polls question Political opinions, Lifestyle information Social attitudes, e.g., managed care populations Demographic: Eligible voters, Target age groups Self-identified: “Democrats”… Behavioral: Voters, ACA users… General: - U.S. adult population samples National, random Digit dial telephone, n=150 to >500 data Knowledge, Attitudes, Behavior Brief interview, cross-sectional findings Descriptive Ratings of politicians, Consumer preferences Approach to Affordable Care Act
76
Examples of surveys & data, 4
Exit polls question Election outcome, possibly stratified by state / region population U.S. electorate National and/or local electoral district sample Probability Stratified random sample of electoral districts. data Self-reported behavior Self-report interview, cross-sectional findings Descriptive/ predictive Increasingly inaccurate predictions See reading on shifts in use of polling data in U.S. politics
77
Examples of surveys & data, 5
”Social Issues Survey” of Chicago gay / lesbian community question - Stress & coping - Alcohol & drug use - Responses to HIV / AIDS population Self-identified Self-identified gay, lesbian, & bisexual adults in Chicago. sample Targeted multi-frame community newspapers, organizations, & mailing lists, n=3500 data Attitudes & behavior Self-report questionnaire, cross-sectional findings Descriptive & hypotheses - High experience of discrimination - Less stress & alcohol-drug use than expected
78
Examples of surveys & data, 6
National Institute on Drug Abuse Household survey of Alcohol and Drug use question Alcohol-drug use and problems, treatment use, health effects. population National All U.S. adults sample Random Multi-stage: Census tract Household, 3. Any adult member n>4000 data Knowledge, attitudes & behavior Face - to - face Interview, successive cross-sectional (each 5 years) findings Typically descriptive Age & regional differences in substance use, trends over time in use & problems Data often used for hypothesis-oriented 2nd analyses (i.e., as archival data).
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.