1. Outline 1. Definition 2. When and why to use surveys
3. How to create a survey
Surveys

2. Surveys - Definition
A survey is a procedure for systematically collecting information about attitudes, preferences, knowledge, or behavior by asking people questions.
We distinguish between surveys and survey research, the purpose of which is to increase scientific knowledge (for example, by testing theories).
Surveys

3. When to Use Surveys
When you want information about a population, but you cannot measure the whole population.
It is usually the case that you cannot measure whole population
Doing so is unnecessary
Population = all the cases of interest.
Surveys

4. Survey research - Definition
Survey research has 3 major characteristics:
Asking people questions using a formal procedure
Given those characteristics, survey research is different from other field methods.
E.g., results of case studies may not be generalizable
Surveys

5. Survey research - Definition
Survey research has 3 major characteristics:
Using a quantitative method that requires standardized information as input
Surveys

6. Survey research - Definition
Survey research has 3 major characteristics:
Generalizing results from sample to population it was drawn from
Information is obtained from a sample and must be generalizable to the population the sample was drawn from
Surveys

7. When to Use Survey Research
Two distinct forms of survey research:
Exploratory
Used in new research areas where little theory has developed
“descriptive”
Of course, the existence of a correlation is not proof of causation, but one can test a hypothesis about cause by looking for the correlation.
Surveys

8. When to Use Survey Research
Two distinct forms of survey research:
Exploratory
Explanatory
Looking for evidence of cause-effect relations among variables
“If C causes D, then C and D should be correlated.”
Of course, the existence of a correlation is not proof of causation, but one can test a hypothesis about cause by looking for the correlation.
Surveys

9. Advantages of survey approach
Surveys are:
Cheap
Relative to measuring whole population
Surveys

10. Advantages of survey approach
Surveys are:
Cheap
Practical
Can be done in a short time
Can be done over phone or through mail or internet
Surveys

11. Advantages of survey approach
Surveys are:
Cheap
Practical
Accurate
Results true of population within a small margin of error
Surveys

12. Margin of Error
A range of values
True population value is likely to be in this range.
If range is large, survey results are not useful
Size of range depends upon sample size and confidence level chosen for estimating population value.
Here, confidence level refers to confidence in the process – we have confidence that true population value will be in that range of values on (1-)*100% of occasions that we use this procedure.
Surveys

13. Margin of error = range of values
which contains
population value %
Value obtained from sample

14. How to create a survey Research questions Design Survey questions
Sampling procedure
Administration procedure
Analyzing the results
Surveys

15. Research question Start with a behavioral theory.
Use it to generate research questions.
Specify type of information that will (or won’t) meet your scientific objectives
Surveys

16. Research question Be as specific as possible
Think about interpretation
In reports, you won’t discuss the actual questions on your survey instrument – you’ll discuss some theoretical construct
What is it?
Surveys

17. Design Longitudinal Same samples measured at different times
Potential problem with loss of subjects
Can take a long time to get data with this design, but if the results differ, it can’t be because of cohort effects, since the same people are involved (except for attrition effects). But it’s difficult to specify whether differences reflect change in society or change in subjects.
Surveys

18. Design Longitudinal Cross-sectional
One or more samples measured at one time
If groups differ, it’s difficult to say why
E.g., compare 20 year olds to 50 year olds – do their results differ because of age? Or cohort effect?
Surveys

19. Design Longitudinal Cross-sectional Successive independent samples
Different samples at different times
Samples supposed to be drawn from same population – but that may be arguable if interval is long
This design can not measure change in individuals. It can measure change in the population if the same population is sampled each time.
Surveys

20. Survey questions If a survey exists that does the job, use it
Otherwise…
Write a first draft
Surveys

21. Survey questions If a survey exists that does the job, use it
Otherwise…
Write a first draft
Get feedback and revise draft
Surveys

22. Survey questions If a survey exists that does the job, use it
Otherwise…
Write a first draft
Get feedback and revise draft
Pretest your questionnaire
Surveys

23. Survey questions If a survey exists that does the job, use it
Otherwise…
Write a first draft
Get feedback and revise draft
Pretest your questionnaire
Edit questions
Surveys

24. Survey questions If a survey exists that does the job, use it
Otherwise…
Write a first draft
Get feedback and revise draft
Pretest your questionnaire
Edit questions
Formalize procedures
Surveys

25. Survey questions Avoid bias “Do you believe in killing unborn babies?”
“Should women be forced to bear unwanted children?”
Surveys

26. Question from an ABC poll
“Schiavo suffered brain damage and has been on life support for 15 years. Doctors say she has no consciousness and her condition is irreversible. Her husband and her parents disagree about whether she would have wanted to be kept alive. Florida courts have sided with the husband and her feeding tube was removed on Friday. What's your opinion on this case - do you support or oppose the decision to remove Schiavo's feeding tube?”
Did incorrect use of the term “life support” influence the result?
Surveys

27. A Gallup Poll question the same week
“As you may know, on Friday the feeding tube keeping Terri Schiavo alive was removed. Based on what you have heard or read about the case, do you think that the feeding tube should or should not have been removed?” 
Surveys

28. Fox News poll 18 days earlier
“Terri Schiavo has been in a so-called 'persistent vegetative state' since Terri's husband says his wife would rather die than be kept alive artificially and wants her feeding tube removed. Terri's parents believe she could still recover and want the feeding tube to remain.  If you were Terri's guardian, what would you do? Would you remove the feeding tube or would you keep the feeding tube inserted?”
Surveys

29. Issues
Is “requiring tube feeding” the same as “being on life-support”?
What does the public understand from the term “life-support”?
Does the term bias the result?
27% Republican, 32% Democrat, 41% other or don't know.   
Surveys

30. Issues Were results influenced by fact poll conducted in one evening?
Were results influenced by whether the person involved was referred to as “Schiavo” or “Terri”?
National Council on Public Polls (NCPP) says: "Surveys conducted on one evening, or even over two days, have more sampling biases -- due to non-response and non-availability -- than surveys which are in the field for three, four or five days.“ (
Surveys

31. Survey questions Be careful about social desirability
Think about question sequence
Use filter questions
Surveys

32. Survey questions Don’t write two questions as one
What would “No” mean in answer to this question:
Do you support the Administration’s decisions to increase class sizes and start classes earlier in the day?
Surveys

33. Survey questions
Do the people you’re surveying have the information required to answer your questions?
Check to see before you ask your questions
Surveys

34. Survey questions
“Are you familiar with the Liberal Party’s platform for this election?”
before
“In your view, will the Liberal Party’s platform produce positive or negative effects on the economy?”
Surveys

35. Survey questions Is the meaning of your question clear?
If you refer to “young people” do you mean 9 year olds? 25 year olds?
Surveys

36. Survey questions Is the meaning of your question clear?
“Which newspaper do you read?”
Does this mean everyday? Sometimes?
What if the respondent reads more than one?
Point is you have to think about these issues before you have a respondent in front of you or on the phone.
Surveys

37. Survey questions Is the meaning of your question clear?
Be clear about the time frame of the behavior you are asking about
E.g., CFQ asks about frequency of mental slips over the last six months
Surveys

38. Sampling procedure Major task: to select a representative sample
Representative sample – shares distribution of relevant characteristics with population
Biased sample – distribution of relevant characteristics is systematically different.
E.g., attitude to reduction in # of UWO parking spaces? Sample students who drive to UWO, not general student population.
Surveys

39. Sampling procedure Some technical terms you need to know Population
Sampling frame
Sample
Element
Surveys

40. Sampling frame
Sample
Population
Element

41. Types of sampling procedure
Non-probability sampling
Elements vary in probability of being chosen
Those probabilities are not known
Accidental samples
Purposive samples
Surveys

42. Types of sampling procedure
Non-probability sampling
Probability sampling
Researcher knows the probability of inclusion for each element in population.
Simple random sampling
Stratified random sampling
Simple random sampling occurs when every element in the population has an equal probability of being selected into the sample.
Surveys

43. Administration procedures
Phone
Cheap & fast
But who is home?
Verbal instructions and response alternatives may be hard to remember
Surveys

44. Administration procedures
Phone
Face-to-face
Expensive
But maximal control
Interviewer can see whether respondent understands question
Respondent may work harder
Surveys

45. Administration procedures
Phone
Face-to-face
Mail
Inexpensive
Significant problem with response bias
Surveys

46. Analyze the Results
Select a data analysis procedure before you collect data.
Correlations
Path analysis
Factor analysis
Surveys