Unit 5. Qualitative and quantitative information, data and research Different beliefs Some say one has to choose either and discourage mixing the two. Some encourage only qualitative, while others encourage quantitative research. Conflict inevitable? Is there a real contradiction? What type of information are you likely to get from using different instruments? Refer back to earlier slides to remember them. Discuss each. Author thinks there is no real contradiction in different methodologies. Summarising vs. Explaining. © 2015 Dávid & Ryan

A typical mixed design Suppose you use a questionnaire to find out about student motivation at school. The results are very poor: the students you asked scored very low on motivation. Complete the table: Can you think of another research context in which the order of qual/quant. methods would be different? Round Research question Instrument Qual.? Quant.? 1 How motivated are the students? questionnaire 2 3 One good research question for round 2 is „Why are they not motivated?”, operationalised in a qualitative way, e.g. interviews. In another context, a qualitative round (doing masses of class observations) might be followed by a quantitative summary (frequency of frontal, pair and group working modes). © 2015 Dávid & Ryan

Trade-off Supposing you want to begin with a questionnaire again, producing largely quantitative data (percentages), the most obvious way forward is to collect qualitative data, which in turn may be followed by another quantitative round. Why are you likely to alternate between quantitative and qualitative phases? Quantitative data fairly obviously lends itself to summarizing, and establishing a cross section of practice, opinion, beliefs, etc. Qualitative information is very good at explaining, giving reasons for the quantitative data/info collected earlier. So, after all, it is the differing nature of these instruments that one recognizes in switching between qualitative and quantitative methods. © 2015 Dávid & Ryan

Interdependence Examples to illustrate the mutual dependence of either on the other. E. g. discourse analysis into group oral tests about patterns of interaction. A qualitative round is naturally followed by a quantitative round of research, and vica versa. Low-yield vs. high-yield questions (Underhill, 1992) partly misunderstood, partly misleading. Tutor may use any example that shows going from quantitative results to further qualitative data collection or vice versa. According to some interpretations, getting a numerical result is low yield – which is either misunderstanding or a misleading extension of the original. © 2015 Dávid & Ryan

Sample and population The problem of writing proficiency. Imagine a nationwide opinion poll: Would young generations with smart phones „forget” to write sooner or later? Can they ask all 10 million Hungarians? The need to draw up a sample of people to ask. Can they ask every 10th Hungarian? Random enough? Still unsatisfactory, why? Hungary, „reduced in scale”? Random sample needed. Illustrate interesting research topic. Answers: obviously not, possible but still very large endeavour, simple sample but potential respondents are a vague group. Difficulty of establishing the population. © 2015 Dávid & Ryan

Concept of generalizability a.k.a. transferability Possible to generalize (extrapolate) from the (results of the) sample to hypothetical but probable results for the population (all Hungarians). Important: populations may also be smaller, such as all secondary students. Examples: All primary students at the same age, all girl students at the same age, all girl students with the same social background, all girl students with the same background, but from single parent families. Notice a trend in these populations? Trend of pop. getting ever the smaller in this list. © 2015 Dávid & Ryan

Discussion Consider situations likely to produce incomplete data, e.g. 1/15 student missing the day you collect data. 7/15 missing? What would you do in remedy? What if no success? Class/group/pair activities. © 2015 Dávid & Ryan

Debate Are the notions of population and sample(s) any good for language teacher-researchers? Some say no, some say yes. Argue for and/or against. Class/group/pair activities. © 2015 Dávid & Ryan

Replication of research Suppose you want to replicate the research underlying the Világ-nyelv brochure. Given its weaknesses, you also want to improve on the design. What will be the likely result(s) of your survey? What type would the first results likely be? Qualitative or quantitative? What would you do next, in order to avoid the pitfalls? What is the most obvious way to continue your research? Qualitative or quantitative questions which ask about _____________ ? The construct: „The ability to speak foreign languages in different European countries, including Hungary” For the discussion, we ignore the practical difficulties one would incur doing this research. Class or paired discussion. Expected answers/likely results poor for Hungary/Hungarians: First results likely to be of the quantitative nature. Most obvious to continue a quantitative project with qualitative questions, e.g. why are Hungarians so far behind? Blank: reasons/explanations/context/circumstances. © 2015 Dávid & Ryan

Reflections on the debate Readers are not prepared to read lengthy treatises without the hope of being able to find something transferable (generalisable) to their own contexts. How does this affect the popularity of case studies, for example? So the researcher needs to state in the research design what group/population their findings might/should be transferable/generalisable to. Which part of the thesis should this best be done? Conclusions from the Bundesrat exampőle for small scale research? This is the reason why case studies are not very popular. Stress the importance of dealing with the problem in a thesis (or paper). © 2015 Dávid & Ryan

Should everything be generalisable? It should be possible to generalise to a large population from a large sample, if the researcher has avoided most traps/pitfalls/mistakes. But what about small-scale (classroom-based research)? 10-15 pupils/students? Difficult to generalise from one class to another, The small-scale researcher has a definite advantage: no need to generalise when it is possible to ask all students in a class: Sample = population. © 2015 Dávid & Ryan

Investigating student feedback (Reading to be set for homework) Read the sample M.Ed. thesis, make notes and prepare to answer these questions: What were the questions? How were the research questions operationalised? What was done to achieve reliability? And what was done in the design process so that a high degree of validity could later be achieved and validity could properly be assessed? How was the research triangulated? What interfering factors were predicted? Procedural rigour goes a long way to substitute making reliable generalisations from samples. Research design part of author’s thesis is attached in the relevant directory, but tutors could choose any other mixed type of research as well. © 2015 Dávid & Ryan