1. Survey Design Workshop:
Uses, Payoffs, and Pitfalls

2. Workshop Agenda: Questionnaires revisited...
Uses, benefits, and limitations
Question Wording and Levels of Data
Activity: Critique of the exemplar by Li, D. (1998).
2. Interviews
Things to consider
Creation of questions for the exemplar by Li, D. (1998).

3. Questionnaires: Uses To get a ‘snapshot’ of what is happening now:
factual/demographical information
information about attitudes/preferences
information about behaviour
can obtain correlational data

4. Benefits of Questionnaires:
we gain insight into what is happening now
data collection is efficient, easy
questionnaires are inexpensive to administer
sites like Survey Monkey ease the process

5. Limitations of Questionnaires:
can not infer a causal relationship***
→the independent variable is not controlled like in an experimental method
the level of questioning may limit the type of analysis you can do (more on this later)
generalizability may be difficult with a small sample size
consider that your findings may be contextual (N=?; random ?)
getting the questions ‘right’ is crucial and requires skill

6. Things to consider in writing questions
In addition to the criteria we studied, it is important to:
ask questions sensitively: some topics worth studying are sensitive in nature
ie. morals, religion, attitudes and experiences with racism

7. Quick Review of Question Types
Closed-ended questions: limited options in the answer
Types: Nominal, Ordinal, Interval, Ratio
Open-ended questions: a wide variety of answers
Types: fill in the blanks, short answer, partial dialogue

8. Planning your questionnaire
Theory
Research questions
Questionnaire questions (content)
Questionnaire questions (format)
Data Analysis
Findings and Conclusions

9. Considerations for close-ended question format and levels of data:
Because respondents are ‘forced’ to answer from the options provided, you cannot get into the depth of their answers (hence, a ‘snapshot’)
You may find that afterwards, you lack a deeper understanding of the issue than you intended
You will have to pilot your questions beforehand
You have to apply the right level of questioning in response to your research question and intended data analysis

10. Nominal Questions
What are they?  attributes are named; they are categorical only ie. Demographics, favorite television show What kind of stats can you obtained ?  frequency of each category and mode  percents and frequencies can be used to test your hypothesis What stats can’t be obtained? A median or mean.  You can’t talk about average scores since averages are numerical, not nominal.

11. Ordinal Questions
What are they?  ranking of items, they measure direction of answer choices ie. beginning, intermediate, and high What kind of stats can you describe?  frequency of each category and mode  percents and frequencies can be used to test your hypothesis  median, the middle value from the respondents’ answers What stats can’t be obtained? mean.  averages are numerical, not nominal.

12. Interval Scale Questions
What are they?  the distance between intervals is numerical, equal ie. Likert scale My teacher is friendly: ie. Semantic differential scale My teacher is unfriendly _ _ _ _ _ friendly.

13. Interval Scale Questions
What kind of stats can you describe?
→ you can get all measures of central tendency INCL. mean
Why? Interval scales are numerical.
The benefit:
With a mean AND a large sample size, you can begin to generalize. (T test) ?

14. Activity: Critique of the exemplar
Li, D. (1998). “It is always more difficult than you plan and imagine”: Teachers’ perceived difficulties in introducing the communicative approach in South Korea. TESOL Quarterly 32(4)
Jeff, how is our APA?

15. Let’s begin with Li’s research question
p. 682

16. Interviews
The 3 major interview types can be classified as unstructured, semi structured and structured. Patton (1990)groups them into:
informal conversational interviews,
general interview guide approach &
standardized open ended interviews

17. Uses To learn about teachers and students background
To learn more about teachers and students “Reported Behaviour”
To learn more about teachers and students “opinions and attitudes”

18. Informal interviews BENEFITS LIMITATIONS
Informal/ conversational interviews
Good for Intensive Case Studies
Open ended length
Flexibility in asking interviewee different questions
Unstructured
Time consuming for gathering systematic information
Not advisable for large populations
Difficult to find patterns in data collected

19. General interview guide
BENEFITS
LIMITATIONS
General interview guide approach
More structured in terms of question design
Allows for easier data compilation
Interviewer refrains from pursuing open ended Questions
Less difficult to establish patterns in data collection

20. Standardized open- ended
BENEFITS
LIMITATIONS
Standardized open ended interview
Highly structured using exact wording
Data Collected is easy to compile and Analyse
No Flexibility in Format

21. Focus Group Interviews
BENEFITS
LIMITATIONS
Focus Groups
More People (6-8) can be interviewed at a time
Less Time consuming for Data Collection
No Flexibility.
Power Relationships/ Asymetrical Relationships
Requires Facilitation
Not very useful for compiling statistical data

22. Things to consider Question wording Power Relationships
Recording the interview: Note taking/ tape recording)
Interview Data Analysis (case analysis/ cross case analysis)
Content Analysis: topic labelling + coding
General Limitations: Not everything an interviewee says reflects reality.

23. Characteristics of successful interview designs
Good question wording such that yes /no questions are avoided
Minimize biases & avoid “leading questions
Brief and debrief:
Inform interviewees about the study and provide some assurance to students ( anonymity or confidentiality if necessary) Ethics
Be sensitive to students/ respondents’ responses

24. Activity
Write 5-10 Questions you would use in an interview

25. References
Gravetter, F.J. & Wallau, L.B. (2016). Statistics for the behavioral sciences.(10th ed.) Wadsworth, CENGAGE Learning.
Li, D. (1998). “It is always more difficult than you plan and imagine”: Teachers’ perceived difficulties in introducing the communicative approach in South Korea. TESOL Quarterly 32(4)
McKay, S. L. (2006). Researching second language classrooms. Mahwah, NJ: Lawrence Erlbaum Associates.
Nunan. D. & Bailey, K. M. (2008). Exploring second language classroom research: A comprehensive guide. Boston: Heinle ELT