1. Mixed-methods data analysis Graduate Seminar in English Language Studies Suranaree, March 2011
Richard Watson Todd
KMUTT

2. Overview Pure quantitative research Pure qualitative research
Mixed-methods research
Collecting both QUANT and QUAL data using different instruments
Mixed-methods data analysis
Usually only QUAL data collected
Data is treated both quantitatively and qualitatively

3. Quantitative or qualitative?
QUANT – QUAL distinction in applied linguistics research
QUANT: data is numbers; uses statistics
Experimental research; surveys
QUAL: data is words; uses thematic or narrative interpretation
Conversation analysis; ethnography

4. Mixed-methods research
“A mixed methods study involves the collection or analysis of both quantitative and qualitative data in a single study with some attempts to integrate the two approaches at one or more stages of the research process” (Dörnyei, 2007)
Purposes:
Achieve a fuller understanding
Triangulate findings

5. Examples of mixed-methods research
Poor example
Research into attitudes: survey a large number and interview a predetermined small number of subjects
Purpose: unclear
Similar, slightly better example
Research into attitudes: survey a large number of subjects, then, selecting based on questionnaire responses, interview a small number
Purpose: follow-up on interesting results

6. Examples of mixed-methods research
An example of the opposite
Interview a small number to gain insights to design a questionnaire, then survey a large number
Purpose: informing instrument design
Another similar example
Research into beliefs: interview 4 teachers but survey 80 students
Purpose: accounting for practicality in using instruments

7. Examples of mixed-methods research
An example focusing on triangulation
Research into strategies: comparing results from different instruments
Much strategy research involves the use of SILL
SILL asks respondents to identify how often they use a particular strategy
Strategy use is context-dependent
Research question: Will recent context of learning change responses to SILL?

8. Examples of mixed-methods research
Single subject
Time 1: read academic articles
Time 2: read short stories for pleasure
Responded to SILL twice
Interviewed 4 times (background interview, after SILL responses, summary interview)

9. Examples of mixed-methods research
SILL responses
Showed major differences between 2 times e.g. “If I guess the meaning of a word, later I will check whether my guess is correct by using a dictionary.” rated Always at Time 1; Never at Time 2
Interview responses
Showed that recent learning contexts influenced different ratings
Triangulation to confirm results or triangulation to provide different perspectives

10. Mixed-methods data analysis
“The most common perception of mixed methods research is that it is a modular process in which qualitative and quantitative components are carried out either concurrently or sequentially. Although this perception is by and large true, it also suggests that the analysis of the data should proceed independently for the QUANT and QUAL phases and mixing should occur only at the final interpretation stage. This conclusion is only partially true … we can also start integrating the data at the analysis stage, resulting in what can be called mixed methods data analysis”
Dörnyei (2007)

11. Mixed-methods data analysis (MMDA)
From Dörnyei, MMDA means
Quantitising qualitative data
Qualitising quantitative data

12. Quantitising qualitative data
Quantitising is often done unconsciously
Conducting a keyword analysis
Use of IELTS scores in research
Quantitising helps a qualitative analysis by allowing a reliability check
Quantitising can be used to count and compare frequency of themes
Quantitising allows further statistical analysis of data, but information is always lost when converting QUAL to QUANT

13. Qualitising quantitative data
Not common
Narrative profile formation
Using quantitatively obtained questionnaire data in a qualitative description of a subject

14. More complex MMDA Nature of QUANT data Nature of QUAL data Concise
Allows further analysis (inferential statistics)
Provides summary information
Nature of QUAL data
Detailed and informative
Allows insight into cases
Provides in-depth information

15. More complex MMDA
What purposes can mixing QUANT and QUAL data analysis serve?
Illustration for insight
Concise summary to give overview
Preliminary overview to inform analysis
Providing a more well-rounded and more persuasive analysis

16. MMDA: Illustration for insight
In many QUANT studies, it is easy to get lost in the numbers and forget what they mean
If the numbers are derived from QUAL data, it is useful to give a QUAL example to concretise the QUANT findings
In Case 1, the original data is QUAL; this is quantitised for analysis; a QUAL example is given to concretise the data and to show how the quantitative analyses was applied

17. MMDA: Summarising for an overview
In some QUAL research (primarily involving categorisation or thematisation), the lengthy, detailed data make it difficult to see the overall pattern
It can be useful to provide a QUANT summary as an overview
In Case 2, the data is QUAL and analysed in a QUAL way, but the overall pattern of results is presented as QUANT

18. MMDA: Preliminary overview to inform analysis
In QUAL studies with large amounts of data, it is difficult for the researcher to ensure that all relevant issues have been identified
It is also difficult to see underlying patterns that can be drowned in the sheer quantity of data
It is useful to conduct a preliminary QUANT analysis to ensure all issues and underlying patterns are identified
In Case 3, QUAL data is treated qualitatively to find keywords which then inform a QUAL thematic analysis

19. MMDA: Providing a more well-rounded and more persuasive analysis
In QUAL studies with large amounts of data, restricting analysis to either QUANT or QUAL cannot provide a full picture of the data
QUAL provides detailed description of the data
QUANT provides generalisations of patterns to the whole data set
In Case 4, QUAL and QUANT analyses are used together to produce a fuller description of the data

20. Uses of MMDA Use Illustration for insight Summarise for overview
Inform analysis
Provide full picture
Pattern
QUANT → QUAL
QUAL → QUANT
Mix of QUANT and QUAL