1. Research Methods in Business Fall 2011 Survey Research
Dr. Stefan Wuyts
Associate Professor Marketing
Koç University

2. Agenda 1. General issues in survey research 2. Measurement scales
3. Questionnaire design
4. Sampling methods
5. Sample size determination

3. 1. General issues Classification of research data
Secondary
Data
Primary
Data
Qualitative
Data
Quantitative
Data
Descriptive
Causal
Survey
Data
Other
Data
Experimental
Data

4. Errors in Survey Methodology
Errors reduce accuracy and quality of raw data
Response Biases
Strongly disagree Strongly agree
Acquiescence (yea-saying) and disacquiescence (nay-saying)
Extreme response style (use extreme response categories)
Midpoint responding (use middle-scale category)
Noncontingent responding (respond randomly)

5. Social desirability bias
Errors in execution (e.g., problem definition)
Interviewer bias
Sample control problems
Who completes the survey?
Self-selection bias / Nonresponse bias

6. How to increase response rate?
Incentives
Monetary incentive (typically $5-10)
Charity
Lottery
Offer choices
Promise summary report or benchmark
Actions
Inform respondents on beforehand
Involve (moral or hierarchical) authority
Reminders

7. Note on response rate Imagine you face the following situation:
You send out 1000 surveys
400 eligible respondents complete survey
You find out that 300 respondents were not eligible
Of the remaining 300, 100 people refused to participate and the other 200 were not reached (returned mail)
What is your response rate?

8. Nonresponse Error
CASRO response rate formula:

9. Three Data Collection Modes
Person-administered: an interviewer reads questions, either face-to-face or over the telephone, to the respondent and records his or her answers (in-home, mall intercept, in- office, telephone center)
Computer-assisted: computer technology plays an essential role in the interview work (Computer Assisted Telephone/Personal Interviewing: CATI & CAPI)
Self-administered: the respondent completes the survey on his or her own (paper & pencil (mail or drop-off), link to online survey)

10. When selecting an approach, many factors may play a role:
Errors
Response bias
Response rate
Representativeness
Interviewer bias
Sample control
Respondent
Personal contact
Feedback
Rapport
Monitoring
Stimulation
Target group
Negativity
Required skills
Anonymity
Pace
Data
Quality control
Adaptability
Graphics
Richness
Security
Economics
Speed
Cost
Real-time
Convenience
Tabulation

11. 2. Measurement scales Measurement process: Two essential steps:
Construct development
Operationalization (The process of assigning descriptors to represent the range of possible responses to a question about a particular object or construct)
Question-response format:
The nature of the property being measured
Previous research studies
The data collection mode
The ability of the respondent
The scale level desired for analysis

12. Scale characteristics
Scaling involves creating a continuum upon which measured objects are located
1. Description
The unique labels or descriptors that are used to designate each value of the scale. All scales possess description.
Male/female; yes/no
2. Order
The relative sizes or positions of the descriptors are known. Order is denoted by descriptors such as greater than, less than, and equal to.

13. Scale characteristics
3. Distance
Ability to express absolute differences between the scale descriptors.
1 YTL difference between 25 YTL and 26 YTL
10 degrees difference between 25 C and 35 C
4. Origin
The presence of a unique or fixed beginning or true zero point.
Zero market share, zero YTL, zero purchases

14. Scaling: Primary scales of measurement
Nominal
Scale
Ordinal
Ratio
Interval

15. Why important?
The scale affects what may or may not be said about the property being measured.
Examples:
If you wish to calculate an average, you must use an interval or ratio scale.
If you have a nominal or ordinal scale, you must summarize the results with a percentage or frequency distribution.

16. Primary scales of measurement: Nominal scale

17. Primary scales of measurement: Nominal scale (cont.)
Labels for identifying and classifying objects.
No indication of amount/intensity of characteristic.
Statistics: limited, frequency counts, percentages, mode
Marketing examples: brand numbers, store types

18. Primary scales of measurement: Ordinal scale

19. Primary scales of measurement: Ordinal scale (cont.)
A ranking scale in which numbers are assigned to objects to indicate the relative extent to which the objects possess some characteristic.
Tells about more or less, not how much more or less.
Choice of numbers is irrelevant (1 2 3 or ).
Statistics: nominal scale + percentile, quartile, median
Marketing examples: market position, social class

20. Primary scales of measurement: Interval scale

21. Primary scales of measurement: Interval scale (cont.)
Numerically equal distances represent equal values in measured characteristic (distance can be compared)
The location of the zero point is not fixed.
Any positive linear transformation of the form y = a + bx will preserve the properties of the scale.
Statistics: nominal and ordinal + arithmetic mean, standard deviation, and other (but no ratios)
Marketing examples: attitudes, opinions

22. Primary scales of measurement: Ratio scale

23. Primary scales of measurement: Ratio scale (cont.)
All the properties of the nominal, ordinal, and interval scales.
Absolute zero point.
Meaningful to compute ratios of scale values.
Only proportionate transformations of the form y = bx, where b is a positive constant, are allowed.
Statistics: all can be applied to ratio data, including ratios.
Marketing examples: age, income, sales

24. What type of measurement scale?
Q1. The following is a question on a survey:
Please check the appropriate price range that indicates the amount you spend each week on gasoline for your car:
_____ 1. $ $10.00
_____ 2. $ $20.00
_____ 3. $ $30.00
_____ 4. $ $40.00
What is the level of measurement that is reflected by the data collected by this question?
Ordinal Scale

25. What type of measurement scale? (cont’d)
Q2. The number of children in a family is an example of what kind of data?
Q3. In a survey of luxury car owners, respondents were chosen from 4 states; California, New York, Illinois, and Ohio. What is the level of measurement that is reflected by the states the owners were selected from?
RatioScale
Nominal Scale

26. Comparative and noncomparative scales
Each object is scaled independently of the others. Resulting data are assumed to be interval or ratio scaled.
Scaling
Techniques
Comparative
Scales
Paired
Comparison
Constant
Sum
Rank
Order
Noncomparative
Itemized
Rating Scales
Continuous
Likert
Semantic
Differential
Direct comparison of stimulus objects; data interpreted in relative terms and have only ordinal or rank order properties

27. Comparative scaling (1) Paired comparison
Comparison of two objects; ordinal data.
Under assumption of transitivity, it is possible to convert paired comparison data to a rank order.
Ten pairs of shampoo brands: indicate which shampoo in the pair you prefer for personal use.
Jhirmack
Finesse
Vidal Sassoon
Head & Shoulders
Pert
# of times preferred 1 2 3 4

28. Comparative scaling (2) Rank order scale
Rank objects according to criterion; ordinal data.
Respondent may dislike brand 1 in absolute sense!

29. Comparative scaling (3) Constant sum scale
Allocate constant sum of units; twice as important then twice as many points; difficult!
Respondent may dislike brand 1 in absolute sense!

30. Comparative scales Advantages and drawbacks
Sensitive to small differences
Same reference points for all respondents
Easy
Less halo effects
Drawbacks
Ordinal data
Restricted to stimulus objects, not generalizable

31. Noncomparative scaling (1) Continuous rating
Placing mark at appropriate position on line that runs from one extreme to the other. Leads to interval data.

32. Noncomparative scaling (2) Itemized: Likert scale
Number or brief description for each category
Likert: agreement or disagreement
Reverse-code negative items
Respondent may dislike brand 1 in absolute sense!

33. Noncomparative scaling (3) Semantic differential scale
7-point rating scale, bipolar labels; positive and negative adjectives; -3 to +3 or 1 to 7
Respondent may dislike brand 1 in absolute sense!

34. 3. Questionnaire design A questionnaire
Is a formalized set of questions for obtaining information from respondents.
Translates information needed into set of specific questions that respondents can and will answer.
Must motivate respondent to be involved in, cooperate, and complete the interview.
Should minimize response error.

35. Pretesting the questionnaire
Focus on content, wording, order, layout, difficulty, but also respondent’s reactions to the survey (via personal interviews).
Important: take respondents from same population as the final survey for pretesting the survey instrument.

36. The Funnel Approach to Ordering Questions
Broad or General Questions
Narrow or Specific Questions
The Funnel Approach to Ordering Questions

37. Use ordinary words
Q: What’s wrong? “Do you think the distribution of soft drinks is adequate?”
A: Simplify language:  
“Do you think soft drinks are readily available when you want to buy them?”

38. Are several questions needed?
Q: What’s wrong? “Do you think Coca-Cola is a tasty and refreshing soft drink?”
A: Double-barreled question: two or more questions are combined into one. Two distinct questions :  
“Do you think Coca-Cola is a tasty soft drink?” and
“Do you think Coca-Cola is a refreshing soft drink?”

39. Can the respondent remember?
Q: What’s wrong? “How many liter of soft drinks did you consume during the last four weeks? ”
A: Does the respondent remember that? Alternative:
How many liter of soft drinks do you consume in a typical week?
1.                  ___ Less than 1
2.                  ___ 1 to 3 liter per week
3.                  ___ 4 to 6 liter per week
4.                  ___ 7 or more liter per week

40. Can the respondent remember?
Q: What’s wrong?
Please consider the last technology innovation project that you were involved in […]
“At the beginning of this technology innovation project, how well did you consider alternative technological options?”
A: Does the respondent remember that?
Hindsight bias colors the responses!

41. Define the issue Q: What’s wrong? “Which brand of shampoo do you use?”
A: Define the issue in terms of who, what, when, and where:  
“Which brand or brands of shampoo have you personally used at home during the last month?
In case of more than one brand, please list all the brands that apply”

42. The W's
Defining the Question
Who
The Respondent
It is not clear whether this question relates to the individual respondent or the respondent's total household.
What
The Brand of Shampoo
It is unclear how the respondent is to answer this question if more than one brand is used.
When
Unclear
The time frame is not specified in this question. The respondent could interpret it as meaning the shampoo used this morning, this week, or over the past year.
Where
Not Specified
At home, at the gym, on the road?

43. Use unambiguous words
Q: What’s wrong? In a typical month, how often do you shop in department stores? _____ Never _____ Occasionally _____ Sometimes _____ Often _____ Regularly
Better:
In a typical month, how often
do you shop in department stores? _____ Less than once _____ 1 or 2 times _____ 3 or 4 times _____ More than 4 times
A: The scale is unnecessarily ambiguous.

44. Avoid leading or biasing questions
Q: What’s wrong? “Do you think that patriotic Americans should buy imported automobiles when that would put American labor out of work?”
A: The question clues the respondent to what the answer should be. Better:  
“Do you think that Americans should buy imported automobiles?”

45. Is the respondent informed?
Q: What’s wrong? “What do you think about the Philips Streamium?”
A: First need a filter question to measure familiarity and past experience; or include a don’t know option.

46. Can the respondent articulate?
Q: What’s wrong? “Describe the atmosphere of a department store”
A: You will need to help the respondent, for example by showing pictures or by providing descriptions to help them articulate their responses.

47. Is the respondent willing to answer?
Q: What’s wrong? “Please list all departments from which you purchased merchandise on your most recent shopping trip to a department store”
A: too much effort; simplify the task:

48. To overcome unwillingness to answer:
Sometimes respondents are not willing to answer because topic of question is sensitive, embarrassing, related to prestige…
To overcome unwillingness to answer:
Make question appropriate given context, legitimate it: explain why you ask the question;
Move sensitive questions toward the end of the questionnaire;
If question is about embarrassing behavior, underscore that such behavior is common;
Third-person technique.

49. Avoid implicit alternatives
Q: What’s wrong? “Do you like to fly when traveling short distances?”
A: Alternative is not explicitly expressed. Better:  
“Do you like to fly when traveling short distances, or would you rather drive?”

50. Avoid implicit assumptions
Q: What’s wrong? “Are you in favor of a balanced budget?”
A: Questions should not be worded so that the answer is dependent upon implicit assumptions about what will happen as a consequence. Better:  
“Are you in favor of a balanced budget if it would result in an increase in the personal income tax?”

51. Avoid generalizations and estimates
Q: What’s wrong? “What is the annual per capita expenditure on groceries in your household?”
A: Less difficult to assess when translated into two different questions :  
“What is the monthly expenditure on groceries in your household?”
&
“How many members are there in your household?”

52. Checklist Use ordinary words Are several questions needed?
Can the respondent remember?
Define the issue
Use unambiguous words
Avoid leading or biasing questions
Is the respondent informed?
Can the respondent articulate?
Is the respondent willing to answer?
Avoid implicit alternatives
Avoid implicit assumptions
Avoid generalizations and estimates

53. 4. Sampling methods Define the Population Determine the Sampling Frame
Select Sampling Technique(s)
Determine the Sample Size
Execute the Sampling Process

54. Element: 18 year old females
Sampling Unit: Households with 18 year old females
Time Frame: Upcoming Summer
Extent: Domestic United States

55. Target Population:
Single parent households
in Chicago
Sampling Frame:
List supplied by a
commercial vendor
Sampling
Frame Error

56. Two Basic Sampling Methods
Probability samples: members of the population have a known and equal probability of being selected into the sample
Non-probability samples: probability of selecting members from the population into the sample are unknown

57. ? Nature of research Statistical considerations
Economic considerations
Probability Sampling Techniques
Simple Random
Sampling
Cluster
Stratified
Systematic ?
Nonprobability Sampling Techniques
Convenience
Sampling
Judgmental
Quota
Snowball

58. Nonprobability Sampling
With nonprobability sampling methods, selection is not based on equal chance.
Convenience sampling
Judgment sampling
Quota sampling
Snowball sampling
Still used very often. Why?
Decision makers want fast, relatively inexpensive answers… nonprobability samples are faster and less costly than probability samples.
Nature of research, statistical considerations

59. Nonprobability Sampling
2.1 Convenience samples: drawn at convenience of interviewer
Error occurs in the form of members of the population who are infrequent or nonusers of that location
2.2 Judgment samples: require judgment or “educated guess” as to who should represent the population
Subjectivity enters in here, and certain members will have a smaller chance of selection than others
2.3 Quota samples: use a specific quota of certain types of individuals to be interviewed
Often used to ensure that convenience samples will have desired proportion of different respondent classes
2.4 Snowball samples: require respondents to provide the names of additional respondents
Members of the population who are less known, disliked, or whose opinions conflict with the respondent have a low probability of being selected

60. Probability Sampling: Simple Random Sampling
Simple random sampling: the probability of being selected into the sample is “known” and equal for all members of the population
E.g., Blind Draw Method
Random Numbers Method

61. Probability Sampling: Simple Random Sampling
Advantage:
Known and equal chance of selection
Disadvantages:
Complete accounting of population needed
Cumbersome to provide unique designations to every population member
Sample might not be representative

62. Probability Sampling: Systematic Sampling
Systematic sampling: way to select a random sample from a directory or list that is much more efficient than simple random sampling
Skip interval=population list size/sample size

63. Probability Sampling: Systematic Sampling
Advantages:
Approximate known and equal chance of selection…it is a probability sample plan
Better than SRS when sampling frame is organized in relevant and systematic way
Efficiency…do not need to designate every population member (as opposed to SRS)
Disadvantages:
Small loss in sampling precision
Worse than SRS in case sampling frame is cyclical in nature.

64. Probability Sampling: Stratified Sampling
When the researcher knows the answers to the research question are likely to vary by subgroups… identify strata that are internally homogeneous and that differ from other strata on relevant variables.
Question: “To what extent do you value your college degree?”
We would expect more agreement (less variance) as classification goes up. That is, seniors should pretty much agree that there is value. Freshmen will have less agreement.

65. Probability Sampling: Stratified Sampling
We expect this question to be answered differently depending on student classification. Not only are the means different, variance is less as classification goes up. Seniors agree more than Freshmen.

66. Probability Sampling: Stratified Sampling
Stratified sampling: method in which the population is separated into different strata and a sample is taken from each stratum
Proportionate stratified sample
Disproportionate stratified sample

67. Probability Sampling: Stratified Sampling
Stratified sampling allows the researcher to allocate more sample size to strata with more variance and less sample size to strata with less variance. Thus, for the same sample size, more precision is achieved.
This is normally accomplished by disproportionate sampling. Seniors would be sampled LESS than their proportionate share of the population and freshmen would be sampled more.

68. Probability Sampling: Stratified Sampling
Advantage:
More accurate overall sample of skewed population
Disadvantage:
More complex sampling plan requiring different sample size for each stratum

69. Probability Sampling: Cluster Sampling
Cluster sampling: method in which the population is divided into groups, any of which can be considered a representative sample (so: internally heterogeneous, no differences between clusters). E.g. area sampling.

70. Probability Sampling: Cluster Sampling
In cluster sampling the population is divided into subgroups, called “clusters.”
Each cluster should represent the population.
Area sampling is a form of
cluster sampling –
the geographic area
is divided into clusters.

71. Probability Sampling: Cluster Sampling
Advantage:
Economic efficiency…faster and less expensive than SRS
Disadvantage:
Cluster specification error…the more homogeneous the clusters, the more precise the sample results

72. Imagine the following information flows among consumers:
Who is in a favorable position in this information exchange network?

73. Kelly
Jack
Mark
Ian
Sandy
Ruth
Bas
Dave
Lynn
Sarah
Pete
Dawn
Gary
Ann
Bill
Erik
John
Jane
Don
Matt

74. Sandy
Sarah
Mark
Bas
Ruth
Jack
Erik
Dave
Kelly
Lynn
Jane
Dawn
Pete
Don
Bill
Matt
John
Ann
Gary
Ian

75. 5. Sample size determination
The determination of sample size is a function of both qualitative and quantitative factors.
Qualitative factors in determining sample size:
Related to the analysis:
the nature of the research
the nature of analysis
the number of variables
Related tot the firm:
the importance of the decision
resource constraints

76. Example Estimating the Mean of a Population
What is the required sample size?
Management wants to know customers’ level of satisfaction with their service. They propose conducting a survey and asking for satisfaction on a scale from 1 to 10.
Management wants to be 99% confident in the results and they do not want the allowed error to be more than ±.5 scale points.
What is n?

77. Quantitative logic relies on three determinants of sample size:
Variability
Accuracy
Confidence

78. Variability
Larger variability: need for larger sample

79. Accuracy (or precision)
Refers to how close a random sample’s statistic is to the true population’s value it represents
Important points:
Sample size is not related to representativeness (the sampling technique, on the other hand, is related to representativeness)
Sample size is related to accuracy
Larger desired level of precision: need for larger sample

80. error
error
sample size

81. Confidence
The Confidence Interval Method of Determining Sample Size: confidence interval represents an area under the normal distribution (e.g., 95% confidence interval) XL _ XU X
0.475
For higher confidence: need for larger sample

82. The Confidence Interval Method of Determining Sample Size is based upon the Central Limit Theorem…
Central limit theorem: a theory that holds that values (such as mean attitude levels) taken from repeated (large) samples of a population are distributed according to a normal curve
More formally: as the sample size increases, the distribution of the sample mean of a randomly selected sample approaches the normal distribution

83. Example of Sampling Distribution: Average Household Incomes among 1000 Random Samples

84. Sample size determination
Definitions and symbols:
A parameter is a summary description of a fixed characteristic or measure of the target population. A parameter denotes the true value which would be obtained if a census rather than a sample was undertaken.
A statistic is a summary description of a characteristic or measure of the sample. The sample statistic is used as an estimate of the population parameter.
Random sampling error: The error when the sample selected is an imperfect representation of the population of interest.

85. Accuracy or precision level: When estimating a population parameter by using a sample statistic, the precision level is the desired size of the estimating interval. This is the maximum permissible difference between the sample statistic and the population parameter.
Confidence interval: The confidence interval is the range into which the true population parameter will fall, assuming a given level of confidence.
Confidence level: The confidence level is the probability that a confidence interval will include the population parameter.

87. Sampling distribution
Sampling distribution of the mean is a normal distribution;
The mean of the sampling distribution of the mean = population parameter μ;
Standard deviation of sampling distribution = standard error of mean
z-value:

88. Reflects variability
Reflects confidence
Reflects precision

89. What is z? { Look in row labeled 1.5 and column labeled .06 to find
Standard Normal Probabilities 5 4 3 2 1 - . Z f ( z ) S t a n d r N o m l D i s b u
1.56 { z
Look in row labeled 1.5 and column labeled .06 to find
P(0  z  1.56) = .4406

90. What z-value corresponds with 95% confidence?
Standard Normal Probabilities

91. Thus:
Specify level of precision
Specify level of confidence
Determine z (1.96 for 95%; 2.58 for 99%, see Table in book appendix)
Determine σ
Determine n
Once sample is drawn, s can be used to approximate σ, leading to new confidence interval (or different precision given particular level of confidence)

92. Example Estimating the Mean of a Population
What is the required sample size?
Management wants to know customers’ level of satisfaction with their service. They propose conducting a survey and asking for satisfaction on a scale from 1 to 10. (range = 9)
Management wants to be 99% confident in the results and they do not want the allowed error to be more than ±.5 scale points.
What is n?

93. Estimating a Mean: What is n?
σ = 9/6 or 1.5
z = 2.58 (99% confidence)
D = .5 scale points
n = 60

94. Estimating a Percentage: What is n?
What does this mean?
After the survey, management may make the following statement: (assume satisfaction mean is 7.3)
“Our most likely estimate of the level of consumer satisfaction is 7.3 on a 10-point scale. In addition, if s equals 1.5, we are 99% confident that the true level of satisfaction in our consumer population falls between 6.8 and 7.8 on a 10-point scale”

95. Note that if s ǂ 1.5 we need to recalculate the precision of the results, using the same formula:

96. Formula gives required size!
Make sure you reach the required size: Incidence rate refers to the rate of occurrence or the percentage of persons eligible to participate in the study. In general, if there are c qualifying factors with an incidence of Q1, Q2, Q3, ...QC, each expressed as a proportion, Incidence rate = Q1 x Q2 x Q3....x QC Initial sample size = Final sample size Incidence rate x Completion rate