1. Mixed Methods Research: Design and Procedures
by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor, Journal of Mixed Methods Research
University of Pretoria, October 21, 2008
© Please do not duplicate or use these slides without the express permission of the author.

2. Topics Introductions Positioning myself
A definition of mixed methods research
Steps in the process of designing a mixed methods study

3. Positioning myself
View research as set of interactive components; not always linear
Focus on rigorous data collection and analysis
Work as an applied research methodogist
Trained in quantitative, self-trained in qualitative, lst generation mixed methods writer
Serve as a consultant on mixed methods on projects
Work on projects in an order not in proposal format; then I reassemble into proper format

4. A few more thoughts about myself…
Published by Sage Publications, Pearson Education (Merrill Educ.) 4

5. A Definition of Mixed Methods Research

6. How would you combine two types of data?
Qualitative
Text Data
This is a sample of a text file of words that might be collected on interview transcripts, observation fieldnotes, or optically-scanned documents.
Quantitative Numeric Data

7. Framework for viewing perspectives on mixed methods
Quantitative Data
Qualitative Data
Mixed Methods
Method
Methodology
Paradigm
Perspective
Use of mixed methods
in other designs

8. A mixed methods researcher…
Collects both quantitative and qualitative data
“Mixes” them
“Mixes” them at the same time (concurrently) or one after the other (sequentially)
Emphasizes both equally or unequally

9. A definition
Mixed methods research is both a method and methodology for conducting research that involves collecting, analyzing, and integrating quantitative and qualitative research in a single study or a longitudinal program of inquiry.
The purpose of this form of research is that both qualitative and quantitative research, in combination, provide a better understanding of a research problem or issue than either research approach alone.

10. Collecting both quantitative and qualitative data
Quantitative data
Instruments
Checklists
Records
Qualitative data
Interviews
Observations
Documents
Audio-visual materials
Quantitative data
Close-ended scales
Attitudinal/behavioral scales
Behavioral checklists
Census, attendance records
Qualitative data
Open-ended responses
Semi-structured interviews
Semi-structured observations
Records/documents
Videotapes
Images such as pictures or drawings

11. Quantitative and qualitative data analysis
Quantitative analysis
Use statistical analysis,
For description
For comparing groups
For relating variables
Qualitative analysis
Use text and images,
For coding
For theme development
For relating themes
Quantitative analysis
Numeric data
Descriptive trend analysis
Hypothesis testing, effect size, interval estimates
Qualitative analysis:
Text/image data
Coding
Themes
Description
Interrelated themes to build a picture of the phenomenon.

12. Mixing or linking the data
Converge data:
Qual
Results
Quan
Connect data:
Qual
Quan
Results
Embed the data:
Quan data
Qual data

13. Typical situations in which mixed methods is used…
To compare results from quantitative and qualitative research
To use qualitative research to help explain quantitative findings
To explore using qualitative research and then to generalize findings to a large population using quantitative research
To develop an instrument because none are available or useful
To augment an experiment with qualitative data

14. What is the reason for using mixed methods?
The insufficient argument – either quantitative or qualitative may be insufficient by itself
Multiple angles argument – quantitative and qualitative approaches provide different “pictures”
The more-evidence-the-better argument – combined quantitative and qualitative provides more evidence
Community of practice argument – mixed methods may be the preferred approach within a scholarly community
Eager-to-learn argument – it is the latest methodology
“Its intuitive” argument – it mirrors “real life” 14

15. Designing a Mixed Methods Study
Preliminary considerations
Creating a title
Posing a general question
Listing the types of data collection and analysis
Making explicit your worldview
Identifying your research design
Drawing a figure of your design
Writing a purpose statement
Writing research questions
Completing a research plan

16. Preliminary considerations (before you begin to design)
Research problem
Content – any topics
Fit the problem to mixed methods (arguments)
Access to both qualitative and quantitative data
Background and resources
Receptive audience

17. Create a working title Writing the title Short Topic Participants
Include the words “Mixed methods”
Neutral –neither quan or qual

18. Pose the general question to be answered
Write it as a question
Look to see how it is phrased
Make sure that it is specific enough and focused (an answerable question)
Ask yourself, “when I end the study, what question would like to have answered?”

19. List your types of data collection (a review)
Quantitative data (closed-ended)
Instruments
Behavioral checklists
Records
Qualitative data (open-ended)
Interviews
Observations
Documents
Audio-visual materials
Quantitative data
Close-ended scales
Attitudinal/behavioral scales
Behavioral checklists
Census, attendance records
Qualitative data
Open-ended responses
Semi-structured interviews
Semi-structured observations
Records/documents
Videotapes
Images such as pictures or drawings

20. Activity – List your sources of data
Quantitative Sources of Data
Qualitative Sources of Data

21. List your approach to data analysis (a review)
Quantitative analysis
Use statistical analysis,
For description
For comparing groups
For relating variables
Design-type
Qualitative analysis
Use text and images,
For coding
For theme development
For relating themes
Design-type
Quantitative analysis
Numeric data
Descriptive trend analysis
Hypothesis testing, effect size, interval estimates
Qualitative analysis:
Text/image data
Coding
Themes
Description
Interrelated themes to build a picture of the phenomenon.

22. Determine your worldview…
Postpositivism
Determination
Reductionism
Empirical observation and measurement
Theory verification
Constructivism
Understanding
Multiple participant meanings
Social and historical construction
Theory generation
Advocacy/Participatory
Political
Empowerment issue-oriented
Collaborative
Change-oriented
Pragmatism
Consequences of actions
Problem-centered
Pluralistic
Real-world practice oriented

23. Several stances on philosophy in mixed methods…
One paradigm (pragmatism, transformative) (Tashakkori & Teddlie, 2003; Mertens, 2003)
Multiple paradigms (dialectic perspective) (Greene, 2007)
Linking paradigms to design features) (Creswell & Plano Clark, 2007)
Epistemological stance (ontology, epistemology, axiology, methodology) (Guba & Lincoln, 2005)
Shared beliefs in a research field (Morgan, 2007)
What it is
How it informs your study

24. Make explicit your interpretive lens (theory)
Advocacy lens (feminist, racial, ethnic, disability, sexual orientation) (Mertens, 2003)
Social science lens (social science theory)
Components:
What is it
Who has used it in your field
How it will shape your study (rephrase your guiding research question, if a lens applies)

25. State your procedures (methods)
Procedures for handling your qualitative and quantitative data
Sequence – concurrent or sequential or both
Emphasis – emphasis on qualitative or quantitative
Sometimes both concurrent and sequential phases are used
Designs may include more than two phases
Think about using a simple, elegant design

26. Concurrent Mixed Methods Designs
Parsimonious designs (Creswell & Plano Clark, 2007)
Concurrent Mixed Methods Designs
Triangulation Design
QUAN Data & Results
Interpretation
QUAL Data & Results
Embedded Design
QUAN Pre-test Data & Results
QUAN Post-test Data & Results
Intervention
qual Process
Interpretation

27. Before-intervention qual After-intervention qual
Sequential designs
Explanatory Design
QUAN Data & Results
Interpretation
qual Data & Results
Following up
Exploratory Design
QUAL Data & Results
quan Data & Results
Interpretation
Building to
Sequential Embedded Design
Before-intervention qual
QUAN Intervention
Trial
After-intervention qual
Interpretation

28. Design options Design Name Equal priority QUAN emphasis QUAL emphasis
Concurrent, triangulation
QUAL+QUAN
QUAN+qual
QUAL+quan
Concurrent, embedded
n/a
QUAN(qual)
QUAL(quan)
Explanatory, sequential, quan first
QUANQUAL
QUANqual
quanQUAL
Exploratory sequential, qual first
QUALQUAN
qualQUAN
QUALquan
Sequential, embedded
(qual) QUAN
QUAN (qual)
(quan) QUAL
QUAL (quan)

30. Explanatory sequential design
Quant
itative
Quantitative
Data Analysis
Case Selection
Qualitative
Interpretation –
based on quan
ad QUAL results
Data Collection
(quan)
Data Analysis
(QUAL) +
Qualitative
Data Collection
Quantitative Data*
Quantitative Analysis
Case Selection
Qualitative Analysis
Interpretation
Selected 5 cases
maximally varying
Identified critical
months in which
smoking varied
Number of cigarettes
Graphic plot of CES -
Descri
ption of each
Why did changes in
CES - D6
D6 scores over time
case
smoking occur?
for each participant
Identification of life
Qualitative Data*
Graphic plot of
events occurring
cigarettes/day values
during critical
Semi -
structured
over time for each
months where
interviews, audio
participant
smoking increased or
recorded and
decreased
transcribed
Thematic analysis of
life events for each
* Data collected 10 times
case
over the course of a
calendar year for 40
Cross -
case thematic
participants
analysis
Source: Creswell, Plano Clark, Shope, McVea. (in progress)

31. Exploratory sequential design
Phase I Qualitative Research - Year 1
Unstructured Interviews -
50 participants
8 observations at the site
16 documents
Qualitative Data Collection
Qualitative Data Analysis
Text Analysis: Using QSR N6
Development of codes and themes
for each site
Qualitative Findings
Phase II Quantitative Research - Year 2
Create approximately a 80-item
instrument plus demographics
Quantitative Instrument Development
Administer survey to 500 individuals
Determine factor structure of items and
conduct reliability analysis for scales
Quantitative Test of the Instrument
Quantitative Results
Determine how groups differ
using ANOVA test
Exploratory sequential design

32. and analysis of qualitative (before, during, after trial)
Embedded research design
Experiment
Intervention
QUAN
Data collection
Pre-test
QUAN
Data collection
Post-test
Process – collection
and analysis of qualitative
data
(before, during, after trial)

33. Mixing the quan and qual data
Type of Mixing
Type of Design
Why Mixing Occurs
Where Mixing Occurs in Research Process
Connecting
Sequential
One phase builds on the other
Between data analysis (Phase 1) and data collection (Phase 2)
Merging
Concurrent
Bring results together
After analysis of both quan and qual – typically in discussion
Embedding
Sequential or Concurrent
Either building or bringing results together
Either between phases or in discussion after analysis

34. Methodological issues
Concurrent Designs
Use strategies to explore contradictory findings
Use parallel questions
Select sub-sample of quantitative for qualitative
Be sensitive to bias from one data collection to the other
Sequential designs
In Explanatory Design, select qual sub-sample from quan sample
In Explanatory Design, consider alternatives for followup qual sampling
In Exploratory Design, samples can differ
In Exploratory Instrument Design, consider qual data analysis approaches for developing instrument

35. Write a purpose statement for a triangulation design
This mixed methods study will address _________________ (overall content-aim of the study). A triangulation mixed methods design will be used, and it is a type of design in which different but complementary data will be collected on the same topic. In this study, _________________ (quantitative instruments) will be used to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). Concurrent with this data collection, qualitative _______________ (type of qualitative data, such as interviews) will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting both quantitative and qualitative data are to bring together the strengths of both forms of research to ___________________ (e.g., compare results, validate results, corroborate results).

36. Write a purpose statement for a embedded design
This mixed methods study will address _________________ (overall content aim of the study). An embedded mixed method design will be used, and it is a design in which one data set provides a supportive, secondary role in a study based primarily on the other data set. The primary purpose of this study will use ________________ (quantitative instruments) to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). A secondary purpose will be to gather qualitative data _______________ (type of qualitative data, such as interviews) that will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting the secondary database is ________________ (e.g., to address different question, to provide support for the primary purpose).

37. Write a purpose statement for an explanatory design
This study will address _______________ (content-aim of the study). An explanatory mixed methods design will be used, and it will involve collecting qualitative data after a quantitative phase in order to explain or follow up on the quantitative data in more depth. In the first quantitative phase of the study, ______________ instrument data to be collected from _______________ (participants) at ___________ (research site) to test _______ (the theory) that explains why ______________ independent variables) relate to the ____________ (dependent variables). The second qualitative phase will be conducted because ______________ (intent of the qualitative phase). In this exploratory follow-up, the ______________ (central phenomenon) will be tentatively explored with ___________ (participants) at _____________ (the research site). The reason for the exploratory follow-up is to _______________ (e.g., to help explain or build upon initial quantitative results).

38. Write a purpose statement for an exploratory design
This study addresses _________________ (content-area of the study). The purpose of this exploratory sequential design will be to __________________ (e.g., develop an test an instrument, generate a taxonomy). The first phase of the study will be a qualitative exploration of ________________ (the central phenomenon) by collecting ___________________ (types of data) from _________________ (participants) at _____________ (the research site). The second quantitative phase will follow up on the qualitative phase for the purpose of __________(intent of this followup). In the quantitative phase, ___________ (instrument data) will be collected from ____________ (participants) at ______________ (research site). Quantitative research questions/hypotheses will be formulated after the completion of the initial qualitative phase. The reason for collecting qualitative data initially is that _____________________ (e.g., instruments are not available, variables are not known, there is little guiding theory).

39. Qualitative research questions
Qualitative central question
Begin with “what” or “how”
Focus on single phenomenon
Use exploratory verbs (discover, understand, explore)
Non-directional language
A general question (allowing participants’ perspectives to emerge)

40. Quantitative research questions
Can be hypotheses or questions
State variables – independent, dependent, mediating, covariates
Develop from theory
Use distinct measures for independent and dependent variables
Order variables from independent to dependent

41. Writing research questions/hypotheses in mixed methods research
Write qualitative research questions and write quantitative research questions/hypotheses
Also write a mixed methods research question
Write these questions separately

42. A new type of research question: A mixed methods question
Three ways to write this question:
Methodologically-focused:
To what extent do the qualitative results confirm the quantitative results?
Content-focused:
How do the interviews with adolescent boys support the quantitative results that their self-esteem changes during the middle school years?
Hybrid of quantitative and qualitative elements:
What results emerge from comparing the exploratory qualitative data about boy’s self-esteem with outcome quantitative instrument data measured on a self-esteem instrument?

43. Order the topics for your plan
Title
Introduction
(Problem)
Worldview/theoretical lens
Audience
Purpose Statement
Research Questions
(Literature Review)
Methods
Type of Mixed Methods Design (also add definition of mixed methods)
Types of Data Collection
Types of Data Analysis
Sequence/Emphasis/ Mixing Procedures
Figure of procedures
Anticipated methodological issues
Ethical issues anticipated
Validity issues
Researcher resources and skills
References, Appendices

44. Additional resources Books:
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Creswell, J. W. (2008). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage Publications.
Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass.
Mertens, D. M. (2005). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.
Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications.

45. Additional resources Articles and Chapters:
Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2),
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp ). Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological issues in conducting mixed methods research. In M.M. Bergman (Ed.), Advances in mixed methods research. London: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11 (3),
Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed Methods Research, 1,
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40,

46. Mixed Methods Research: Design and Procedures
by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor, Journal of Mixed Methods Research
University of Pretoria, October 21, 2008
© Please do not duplicate or use these slides without the express permission of the author. 46 46