1. ECC511 Research Methods in Education Dr. Jill Taggart, 2014
Survey Design
ECC511 Research Methods in Education
Dr. Jill Taggart, 2014

2. Surveys Not the easy option They take time to create accurately
Benefit: you can obtain information that cannot be obtained through other research methods.
It is a long process of creating, testing and delivery

3. Techniques Likert Scales Thurstone method Semantic differentials
Open-ended questions
Closed questions

4. Likert Scales (Likert, 1932)
Used to measure attitudes from i.e. strongly disagree to strongly agree
Procedure:
1. choose an attitude object (i.e. opinion, attitude towards pets, attitudes towards counselling)
2. Generate as many statements a possible to measure this attitude (one sentence each).
3. Decided what a high score on your survey indicates. A high score would indicate favourable attitude to pets, or counselling.

5. Likert Scales (Likert, 1932)
4. Make sure you have equal numbers of positive and negative statements. A positive statement AGREES with what a high score indicates. A negative statement DOES NOT AGREE with what a high score indicates.

6. Likert Scales (Likert, 1932)
Positive or negative statements?
Owning a pet can decrease lonely feelings.
Pets have parasites that can cause disease in humans.
Pets are expensive to keep.
Counselling is not useful for bereaved people.
The only good thing about counselling is the ride home.
I learn a great deal from my school counsellor.

7. Likert Scales (Likert, 1932)
Then randomize your positive and negative statements. Why?
Under each item/statement give the respondent a choice of responses (1-5)?
Create 2 copies of your survey: one to photocopy (which is blank) and the other to score every statement
Then get 10 people to complete your survey. Each participant receives a total score by adding up each statement’s score

8. Likert Scales (Likert, 1932)
ECC 511, Survey Design, Dr. Jill Taggart
Likert Scales (Likert, 1932)
Next step: Items analysis
Rank order all participants’ total scores picking out top 3 and bottom 3 scorers.
Then write down the responses to Q. 1 for the three people in the high scoring group, and then for the low scoring groups:
High scoring: SA, SA A
Low Scoring: SD, SD, SD
This item should be accepted as the responses from the two groups are distinctly different.
Items analysis: RANK order the total scores from all of the participants and choose the top three and the bottom three scores. You must now analyse each individually to see if it is a good DISCRIMINATOR between high and low scores.
Step 1: wite down the responses to Q. 1 for the three people in the high scoring group, and then for the low scoring groups:
High scoring: SA, SA A
Low Scoring: SD, SD, SD
This item should be accepted as the responses form the two groups are distinclty different.

9. Likert Scales (Likert, 1932)
ECC 511, Survey Design, Dr. Jill Taggart
Likert Scales (Likert, 1932)
But…if:
High scoring: A, DK, SD
Low Scoring: A, A, D,
This item should be rejected as the responses are distinctly different
The accepted items now form the basis of your survey. Create 2 master copies as you did before.

10. Likert Scales (Likert, 1932)
ECC 511, Survey Design, Dr. Jill Taggart
Likert Scales (Likert, 1932)
Pro Likert scale:
Ideal for measuring attitudes
Easy to swap positive to negative statements (and vice versa)
Easy to score and add up a final total
A reliable measure after the items analysis
Con:
Items analysis is rather subjective.
It can be difficult to create enough initial statements without repetition
It’s repetitious with the same responses for each statement

11. Thurstone method (Thurstone, 1931)
Choose an attitude object
Create as many statements (positive and negative to measure object)
Engage a panel of “expert judges of your attitude object” to rate each item on a scale from 0 (extremely negative) to 10 (highly positive)
For each item calculate the mean score by adding up the value each judge gave and dividing by number of judges

12. Thurstone method (Thurstone, 1931)
The mean now becomes the score a participant receives if they tick that item
Randomize the list of items so that the low- scoring items are not clustered together. Next to each statement include a check-box for participant to check if they agree with the statement.
Create 2 masters: one to complete the survey, the other with the mean scores if they agree with that statement. Add up values of each checked item after participant completes.

13. Thurstone method (Thurstone, 1931)
Example:
Please tick any of the following statements with which you agree:
Companies should pay full salary for maternity leave (9.8)
Companies should provide more women’s washrooms (6.2)
Women are less reliable employees because they leave through pregnancy (2.1)
(scores in brackets do not appear to the respondent)

14. Semantic Differentials (Osgood et al. 1957)
Assess feelings & thoughts to build up picture of “typical person”
Choose the person or group of people to rate. Can be a description or pictures
Choose bi-polar adjectives to assess the description of picture (i.e. strong/weak, clean/dirty)
Decide what a high score indicates
Randomize so not all the positive traits are on the left or right

15. Semantic Differentials (Osgood et al. 1957)
i.e. good _ _ _ _ _ bad
weak _ _ _ _ _ strong
Have five or seven response spaces to rate their responses on a scale form good to bad above
Create 2 masters, leave one blank and score the other! Rate each item
Do item analysis as with Likert Scale
Create 2 masters as you did before.

16. Open & Closed Questions
Open Questions: give us qualitative data based on the richness of the response.
Closed or Fix-choice questions give the answers to the participant and they have to choose the most appropriate for them
Open: please state the main factors as to why you own a pet at present?

17. Open & Closed Questions
Closed or Fixed choice answers:
Please tick the main factors as to why you own a pet at present:
Companionship □
Security □
Affection □
Stress reliever □
It was a stray □
Other □

18. Interviews using Surveys
Structured:
Previously decided questions in a particular order. Might be Likert or fixed-choice.
Unstructured:
A number of topic areas to be covered and fixed questions or fixed-choice answers are not provided or elicited.

19. Factors affecting Interviews
ECC 511, Survey Design, Dr. Jill Taggart
Factors affecting Interviews
Structure of the questions: poorly formulated
No double barrelled questions
Do not introduce an assumption
Do not include jargon
Do not use leading questions
Do not use double negatives
No “catch all” questions
Double barrelled
i.e. do you think cats and dogs are good pets?
No assumptions: do you think genetically modified food is harmful to the environment
Jargon” do you agree with Freud’s notion of the significance of the mother figure on cognitive-emotive development
Leading: do you believe Advil is more harmful that aspirin?
Do you feel it is not true that not all children need a stable maternal figure…
Catch-all Tell me all you know about how HIV influenced your sexual behaviour. Interviewee will need further prompts to continue.

20. Face to Face or not?
Fewer socially desirable responses in telephone interviews.
Hayaski et al. (1999) discovered a much higher response rate from a telephone interview compared with face to face when discussing sex behaviour.

21. ECC 511, Survey Design, Dr. Jill Taggart
Your own survey
Create a survey with 6 items
Use a 5 point likert scale
Test your survey on your classmates
Do an items analysis
Report on findings