STATISTICS FOR MANAGERS LECTURE 2: SURVEY DESIGN

COMPONENTS OF SURVEYS 1. Sampling 2. Methods of data collection 3. Question design 4. Interviewing

1. SAMPLING How well a sample represents a population depend on: 1. Sample frame 2. Specific selection procedure 3. Sample size

1.1. SAMPLE FRAME Sample of people that has a chance to be selected, given the sampling approach. Classes: Sampling is done from a more or less complete list of individuals of the population Sampling is done from a list of people that go somewhere or do something Sampling is done in two or more stages

1.1. SAMPLE FRAME Characteristic of the sample frame to be evaluated: Comprehensiveness: a sample can be representative only of the sample frame. Most sampling leave out a few people. Examples: household based sample exclude people in dormitories, prison and nursing homes.

1.1. SAMPLE FRAME Characteristic (cont.) Probability of selection.It is not necessary that a sampling scheme give every member in the sample the same chance of selection. However, we should be able to find out the probability of selection for each individual selected. Example.

1.1. SAMPLE FRAME Characteristic (cont.) Efficiency. In some cases sampling frames include units that are not among those wanted to sample. Examples: elderly using general household and eliminating with no old people; random dialing. Only issue: cost.

1.2. SELECTION OF UNITS One stage sampling: Simple random sampling Systematic sampling Stratified sampling Multistage sampling: no way to get at the population directly. Link population with groups.

1.2. SELECTION OF UNITS Simple random sampling: members of a population are selected one at a time, independent of one another and without replacement. Unless the list is short and all unit are prenumbered drawing from a simple random sample is very laborious.

1.2. SELECTION OF UNITS Systematic sampling. The researcher determines the number of entries in the list and the number of elements to be selected. For instance list of 8500 people and 100 required. 1 out of 85 should be selected. The process start with a random number.

1.2. SELECTION OF UNITS Systematic sampling (cont.). Then we take any 85th person in the list. Problem: if the list is ordered following some criteria or has a recurring pattern that will differently affect the sample depending on the random start. Examples: couples club were the male partner is always listed first: only male or female.

1.2. SELECTION OF UNITS Stratified sampling. Generally little is known about the characteristics of the individuals in the population before the sample. However, we may know some characteristics. Usually stratification involves some regional variable.

MULTISTAGE Examples: Sampling students from schools Area probability sampling Random digit dialing Respondent selection: who should be interviewed in the household?

MULTISTAGE If information is easy to report any adult who is home can answer the question. If information is specialized the researcher wants to interview the household member most knowledgeable.

MULTISTAGE Group work: Why are registered unemployed many less than survey unemployed in Spain? Data

1.3. SAMPLE SIZE Sample size and sampling errors will be covered in the next lecture.

2. DATA COLLECTION Choice of data collection mode depend on: Sampling Type of population Available staff Response rate Question form

2. DATA COLLECTION Methods Personal interview Telephone interview Self-administrated Mail Drop off and pick latter Internet

2.1 Personal interview Advantages There are some sample design that can be implemented best by personal interview (area probability samples) Most effective way of enlisting cooperation for most populations Interviewer can answer respondents questions, explain complex questions, etc.

2.1 Personal interview Advantages (cont) Multimethod data collection (including visual cues) are feasible. Confidence building is possible (including reassurances for sensitive material) Probably longer survey instruments are possible in person than in any other mode

2.1 Personal interview Disadvantages It is likely to be more costly than the alternatives. A trained staff of interviewers that is geographically near the sample is needed Total data collection period is likely to be longer than telephone procedures

2.1 Personal interview Disadvantages (cont) Some samples (high rise building, high crime areas, elites, students) maybe more accesible by some other mode

2.2 Telephone interview Advantages Lower unit cost than personal interviews Random digit dialing (RDD) Better access to certain population Shorter data collection periods Interviewer administration (better than mail or internet)

2.2 Telephone interview Advantages (cont) Interviewer staffing and management easier than personal interviews: less staff, not necessary to be near the sample, supervision and quality control potentially better Likely better response rate from a list sample than mail

2.2 Telephone interview Disadvantages Sampling limitations (omitting those without telephone). Non response higher than personal interview. Questionnaire or measure constraints. Less appropriate for personal or sensitive questions.

2.3 Self-administrated Advantages Easy presenting questions requiring visual aid. Asking questions with long and complex response categories. Asking batteries of similar questions. Respondent do not have to share answer with interviewer.

2.3 Self-administrated Disadvantages Specially careful questionnaire design is needed. Open questions usually not good idea. Good reading and writing skills by respondent are needed. The interviewer cannot exercise quality control Cannot control who answer the questions

2.4 Mail Advantages Low cost Minimal staff and facilities Provide access to widely dispersed samples and people difficult to reach by phone or in person. Respondent have time to give a thoughtful answer, to look up records, etc.

2.4 Mail Disadvantages Ineffectiveness of mail as way of enlisting cooperation Disadvantages of not having interviewer involved in data collections Need for good mailing addresses for sample.

2.5 Drop off-pick up later Advantages The interviewer can explain the study, answer questions and designate a household respondent. Response rates tend to be like those of personal interviews. More opportunity for thoughtful answer and check out the records. Does not requires trained interw. staff

2.5 Drop off-pick up later Disadvantages Cost about as much as personal interviews. Field staff is required (although less trained that for personal interviews).

2.6 Internet surveys Advantages Low unit cost of data collection Potential high speed of return All the advantages of self-administrated instruments. Al the advantages of computer assisted instruments Time for checking out records, etc.

2.6 Internet surveys Disadvantages Limited to samples of internet users Need for good addresses Challenges of enlisting cooperation Various disadvantages of not having an interviewer involved in data collection.

Exercises How is the Census conducted? Other special samples: Trial juries Measures of TV shares Measures of effectiveness of ads.

3. Question design Decide what you want to find out; this is the most important step in writing a questionnaire. Always test your questions before taking the survey. Keep it simple and clear Use specific questions instead of general ones, if possible.

3. Question design II Relate your questions to the concept of interest. Decide whether to use open or closed questions. Report the actual question asked. Avoid questions that prompt or motivate the respondent to say what you would like to hear.

3. Question design III Use forced-choice rather than agree- disagree questions. Ask only one concept in each question. Pay attention to the question order effect.

3. Question design IV Specific issues: Contingent valuation Data on the registry of accounting. Trust questions and experiments.