SURVEYS AND QUESTIONNAIRES See Robson Chapter 8

Typical Survey Features  Use of a fixed quantitative design  Collection of a small amount of data in a standard form from a large number  Selection of representative samples from known populations  Generally closed-ended  Not hard and fast rules

Survey Types  In-Person Questionnaire  Self-Administered Postal (mail) Questionnaire  Telephone Survey  Not restricted to the use of questionnaires  E.g. traffic survey – observational

Advantages of Surveys  All Survey Types:  Simple and straightforward approach  Adaptable to collect generalizeable information from almost any population  Data standardization  Interview Surveys:  Interviewer can clarify questions  Presence of interviewer encourages participation

Advantages Cont.  Self-Administered/Mail:  Way of retrieving information about history of a large group  Efficient – Large data sets, low cost, short timeframe Note: May be disadvantage if researcher is seduced into using a survey when it is not most appropriate to research question  Allows anonymity/encourages frankness  Allows large geographical distribution

Disadvantages of Surveys  All Survey Types:  Context, discourse, and meaning  Lack of relation between attitude and behavior  Social desirability response bias  Data affected by characteristics of respondents  Self-Administered/Mail:  Low response rate – still representative?  Misunderstandings not detected  No control  Respondents may not take seriously

Disadvantages Cont.  Interview Surveys:  Data affected by interviewers/interview bias  Affected by interactions of interviewer/respondent characteristics  Concerns of anonymity – respondent less open

Developing Survey Questions  Pilot work – semi-structured interviews, focus groups, other data collection  Previous studies  Theoretical frameworks

Activities in a Sample Survey 1. Initial design and planning 2. Designing the questionnaire 3. Pre-Testing 4. Final design and planning 5. Data collection 6. Analysis and reporting

1. Initial Design and Planning  Most straightforward task for survey are ‘how many,’ ‘how much,’ ‘who,’ ‘where,’ and ‘when’  Gauge public opinion  Test theories  Theoretical framework important when trying to move beyond description to explanation  Unit of analysis  Population  Sampling frame

2. Designing the Questionnaire  Provides a valid measure of the research questions  Gets the cooperation of respondents  Elicits accurate information  Be clear  Fixed-alternate responses should be accurate, exhaustive, mutually exclusive, and on a single dimension  Keep response rate in mind for mail surveys  Standardize in-person surveys

3. Pre-Testing  First stage:  Informal – Colleagues, friends, family  Ask if questions are clear, simple, and unambiguous  Second stage:  Respondents from the groups of interest  Give thoughts on the question  Third Stage  Formal Pre-test  Pilot version

4. Final Design and Planning  Edit questionnaire  Spelling  Layout  Finalize coding and analysis procedures

5. Data Collection  Follow your plan!  Keep clear notes/records  Address issues as they arise

6. Analysis  Generally straightforward with closed questions  Codes can be arbitrary (but consistent) or can be the actual number  Have a code for non-response  Open questions – Simplify many responses by classifying into smaller number of groups  Coding categories chosen from sample of all respondents  Coding effectively changes open questions to defined set of responses

Survey Sampling  Selection from the ‘population’  Sample size  Would a smaller set of longer more detailed surveys be preferable to more?  Probability/representative samples  Non-Probability samples

INTERVIEWS See Robson Chapter 9

Interview Features  Generally one-on-one and face-to-face  Can be in group settings or by phone  Can be primary/only approach or in combination with other methods  Open-ended questions are the norm

Types of Interviews  Structured  Extreme example is the survey  Semi-Structured  Unstructured  Allows more flexibility of response  Extreme is the ‘depth interview’  Respondent interviews/informant interviews

Advantages of Interviews  Flexible and adaptable  Asking people directly is a short-cut in seeking answers (versus observation)  Offers possibility of follow-up  Non-verbal cues  Rich and highly illuminating material

Disadvantages of Interviews  Requires considerable skill/experience  Lack of standardization/reliability concerns/bias  Time consuming and requires lots of preparation  Can be limiting geographically

Interview Process  Listen more than you speak  Questions should be straightforward, clear, non- threatening  Avoid leading  Enjoy it!  Take a full record  General format: Intro, Warm-up, Main Body, Cool- Off, Closure

Interview Content  Content consists of:  A set of items/questions Closed Open Scale  Probes and prompts  Proposed sequence of questions

Other Interview Types  Informant interviews  Non-directive interview: Totally in control of interviewee  Focused interview: Situational analysis Interview guide  Group Interviews  Most common are a hybrid of discussion and interview  Allows group interaction  Focus Groups

Focus Group Advantages  Highly efficient  Checks and balances  Focus on most important topics  Enjoyable  Inexpensive, flexible, quick to set up  Stimulated by thoughts of others  Contributions encouraged  Those with limited reading/writing skills not excluded  Less inhibited members serve to break the ice

Focus Group Disadvantages  Limited number of questions  Facilitating requires considerable expertise  Needs to be well managed  Conflicts may arise  Lack of confidentiality  May not be generalizeable  Live and immediate  Participants may try to impress one another  Difficult to follow-up on views of individuals

Focus Groups  Homogeneous Groups:  Facilitates communication  Promotes exchange of ideas and experiences  Gives a sense of safety  May result in ‘groupthink’  Heterogeneous Groups:  Can stimulate and enrich discussion  May inspire new ideas  Risk of power imbalances  Can lead to lack of respect for opinions  Can lead to dominant participant destroying group process

Analysis of Interview Data  Taping and Transcribing  Allows concentration on the interview  Alternative to complete transcription is to be selective  Too late to think about analysis when interviews are complete