1. Unit 6 Research and data collection techniques
Observation has been dealt with extensively in the ”Focus on the Learner” course. The tutor may decide to do some revision, for which a video sample of a class/classes would be best, so that students can practice with real data. Those who, for whatever reason, have not done “Focus on the learner”, may refer to the slides therein, as it is also available from the same location.
© 2015 Dávid & Ryan

2. Investigating student feedback 2 (based on homework)
Discussion: sample M.Ed. Thesis
What were the questions?
How were the research questions operationalised?
What was done to achieve reliability? And what was done in the design process so that a high degree of validity could later be achieved and validity could properly be assessed?
How was the research triangulated?
What interfering factors were predicted?
Procedural rigour goes a long way to substitute making reliable generalisations from samples.
Reseach design part of author’s thesis is attached in the relevant directory, but tutors could choose any other mixed type of research as well.
© 2015 Dávid & Ryan

3. A discussion of instruments and data collection techniques
To some extent, in “Focus on the Learner”
Observation
Interviews
Questionnaires, feedback and tests
Introspection
Concurrent/retrospective?
Verbal protocols
Repertory grids
Text (discourse) analysis
Conversation Analysis
Diary studies
Case studies
Focus in subsequent slides/units on various techniques.
© 2015 Dávid & Ryan

4. Exposure or revision/ practice
Observation
Exposure or revision/ practice
Observation has been dealt with extensively in the ”OT-274 Focus on the Learner” course. The tutor may decide to do some revision, for which a video sample of a class/classes would be best, so that students can practice with real data. Those who, for whatever reason, have not done “Focus on the learner”, may refer to the slides therein, as it is also available from the same location.
© 2015 Dávid & Ryan

5. Interviews Kinds of interview: a continuum of interview types
Unstructured through semi-structured to completely structured (or free and guided)
Question types: closed and open ended
Should there be an interview schedule?
What is an interview schedule?
The value of writing a set of questions and sequencing them.
Semi-structured schedule highly appropriate.
Why?
Possibility of branching out from the scripted sequence of questions
© 2015 Dávid & Ryan

6. Evaluating a semi-structured schedule
Sample from researching judges’ work
Law-court judges? Oh, no , these judges are ….
Evaluate the judges interview schedule.
Evaluate the main line of questions.
Appropriate? Logical?
Sequencing OK?
What is the function of the „subquestions” in italics?
… asked to provide judgements for language tests, in terms evaluating the appropriacy of passmarks, recommending alternative passmarks, etc. With the help of identifying „borderline” students.
Subquestions are reminders for the interviewer and they may be by-passed, if they are not necessary.
Source for questionnaire: Dávid (1995)
© 2015 Dávid & Ryan

7. Semi-structured intview schedule (with points of departure and extensions italicized)
How well were you prepared for your role as judge?
Did you manage to follow my instructions? Why not?
Can you recap the judgement process? What exactly were you asked to do? Why can't you, do you think?
How would you evaluate the judgement process? What are your reasons for saying that?
Here is the Lancaster model for the judgements. Do you think we were following it? What exactly was missing?
Were my instructions in agreement with it?
What do you think of the model?
Which year's grammar and listening papers were better written, do you think? How do they compare for difficulty? Is this your own observation or other staff said so too?
In your opinion, how do different years -- 'cohorts' – of students compare for their English?
Have you observed any of the 3 years of students to have had special strengths or weaknesses? If so, what were these?
Look at my assessment of the test data. Any comment?
How do you think we could make improvements so that there is no such gap between the judges' estimates and the test performance of borderline students?
What would you have to do in order to improve judgements next year?
This study is conducted rather late. Do you think this can affect the quality/validity of the findings?
Having had this talk, how would you assess your understanding of the judgement procedure now, as opposed to the beginning of the judgement exercise?
Anything else to add?
Photocopies of the schedule may have to be provided.
Source: Dávid (1995)
© 2015 Dávid & Ryan

8. Recording what has been said
Why should it be done at all?
How many different ways of recording can you identify?
Notes?
_____________
Discuss the relative advantages (disadvantages) of each.
Because one would forget much of what has been said and it is all retrievable.
Audio, AV.
AV is best, obviously, because of the wealth of info, including non-language and it can also be transcribed.
© 2015 Dávid & Ryan