Survey Research By Dr Rojnath Pande

Survey Research Longitudinal Surveys Cross-Sectional Surveys

Characteristics of Survey Research In survey research, the researcher selects a sample of respondents and administers a questionnaire or conducts interviews to gather data, which are then used to describe the population Surveys are used to learn about attitudes, beliefs, opinions, behaviors, to name a few Surveys are often used because researchers can gather accurate information about a large number of people using a small sample Thus we have both surveys as a research tool and survey research as a research design.

Examples of Survey Research Topics Descriptive Studies – What is the average length of time teachers use to prepare lessons? Relationship Studies – Is there a relationship between teacher attitudes toward discipline and student satisfaction with the class? Explanatory Studies – Why are students in one school achieving better than similar students in another school?

Advantages of Survey Research Versatility Efficiency Generalizability Cost effectiveness Small samples provide for reliable extrapolation of results

Longitudinal Surveys Studies in which the same group of subjects are studied over time Trend studies use the same population across time but use different samples from that population each time Cohort studies examine a specific group (same population) over time Panel studies are cohort studies that use the same sample each time

Cross-Sectional Surveys Attempt to simulate longitudinal data in a shorter time frame Data are gathered from multiple samples of the same population simultaneously May be used to study a phenomenon at one time or to gather data from multiple groups at the same time

A Note on Surveys… Need to know, in advance, how the data will be used rather than “fishing” for whatever’s out there Data collection methods may vary but they must be standardized Instructions should be clear Both the instructions and survey should be pilot tested Because a typical response rate to a mailed survey is only 40%-60%, follow ups should be conducted Nonrespondents may introduce bias

Questions to Ask Before Doing Survey Research Do you have a clear hypothesis? Do your questions focus on that hypothesis? Will participants answers provide accurate answers to your questions?** To whom will your results apply?**

Answers May Be Inaccurate Because: People don’t know the causes of their own behavior People’s memories are inaccurate People are not good at predicting their future behavior People are not telling you what they know because of – Social desirability bias – Demand characteristics – Response sets

Generalizing Your Results Do you know what your population is? How can you get a representative sample of that population? Will nonresponse bias prevent you from generalizing your results to that population?

Advantages and Disadvantages of Survey Research:Conclusions Easy way to get a lot of information However, that information: – Will not have internal validity – May not have construct validity because of self-report problems – May not have external validity because of poor sampling or because of nonresponse bias – May not answer research question because survey questions weren’t focused on hypotheses

The Advantages and Disadvantages of Different Survey Instruments Written Instruments * Interviews*

Written Instruments Self-administered questionnaires – Cheap, easy to distribute to large sample--but nonresponse bias is big problem – Anonymous which may promote honest responses Investigator-administered questionnaires – Higher response rates – May hurt sense of anonymity and thus decrease honesty of responses Note: A highly refined version of the investigator- administered questionnaire is the psychological test**

Psychological tests May be part or all of a questionnaire Even if you don’t use questions from tests, you can try to use strategies that test-makers use**

Things Questionnaire Writers Can Learn from Test Makers Pretest questions and measure Standardize how measure is administered Be objective Be aware of response sets Seek evidence for measure’s reliability and validity

Interviews May be worth added expense if – It increases response rate – Need ability to clarify questions, follow up on responses May not be worth added expense if construct validity is harmed by – Interviewer bias – Participant trying to impress interviewer Telephone interviews may be ideal solution**

Advantages of Telephone Interviews Cheaper than face-to-face interviews Can have high relatively high response rate Interviewer bias controlled by – Lack of body cues – Monitoring/taping interviewer’s calls

Planning a Survey Deciding on a research question Choosing the format of your questions** Choosing the format of your interview--if you use an interview** Editing your questions** Sequencing your questions** Refining your survey instrument** Choosing a sampling strategy**

Choosing the Format of Your Questions Fixed alternative – Yes/No Reliable Not powerful – Likert Open-ended – May not be properly answered – May be difficult to score

Choosing the Format of Your Interview Unstructured – Interviewer bias is a serious problem – Data may not be hard to analyze Semi-structured – Follow-up questions allowed – Probably best for pilot studies Structured – Standardized, reducing interviewer bias

Editing Questions: Nine Mistakes to Avoid 1. Avoid leading questions 2. Avoid questions that invite the social desirability bias 3. Avoid double-barreled questions 4. Avoid long questions 5. Avoid negations 6. Avoid irrelevant questions 7. Avoid poorly worded response options 8. Avoid big words 9. Avoid ambiguous words & phrases

Sequencing Questions To boost response rate, put innocuous questions first, personal questions last To avoid wasting time, qualify early To increase accuracy, keep similar questions together To boost response rate, put demographic questions last

Putting the Final Touches on Your Survey Instrument Professional appearance Proof reading Pilot testing Practice coding responses--may lead to refining questionnaire so that it is easier to code responses

Choosing a Sampling Strategy Random sampling – Proportionate stratified random sampling Convenience sampling – Quota sampling Conclusions – Only random sampling will be representative – Nonresponse bias may ruin your sample

Administering the Survey Informed consent Clear instructions Debriefing Confidentiality

Analyzing Survey Data Summarizing data Inferential statistics

Summarizing Data Interval or ratio data – Mean – Correlation coefficients – Tables of means Nominal data – Frequencies, percentages – Phi coefficient – Tables of frequencies

Using Inferential Statistics Interval or ratio data – Looking at relationships between pairs of variables If have two groups, could use t-test between means If not, should use test to see whether the correlation between two variables was significant Be aware that if you do numerous statistical tests, you may be setting yourself up for a Type 1 error – To look at more than two variables at once, you can do ANOVA Multivariate analysis of variance, multiple regression, factor analysis, and other sophisticated tests

Using Inferential Statistics Nominal data – Chi-Square test – Be aware that if you do numerous statistical tests, you may be setting yourself up for Type 1 errors

Concluding Remarks Survey research is the most used research method Survey research is the most misused research method You know how to use rather than abuse survey research

The End