1. Mixed Methods Research: State of the Art (What Has Developed In Mixed Methods)
by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln
Co-editor, Journal of Mixed Methods Research, and
Co-Director, Office of Qualitative and Mixed Methods Research
© Please do not duplicate or use these slides without the express permission of the author.

2. The Story of Mixed Methods in the Last 20 Years
Highlight four developments
Why I can tell the story
First generation mixed methods – reflect on my role in story
Balanced – background in quan and qual
5 books on research methods
Applied research methodologist
Focused on education, social and human sciences
Journal editor
What I am trying not to do
“Fix” the history
Discourage discourse
Speak for all disciplines
Limit the discussion to a US perspective
Cover all developments

3. Four Developments (last 20 years)
Increasing interest in and advocacy for mixed methods
Evolving understanding of what is mixed methods research
Emerging philosophical perspectives as a foundation for mixed methods
Developing designs and innovative techniques

4. Development #1: Increasing Interest in and Advocacy for Mixed Methods
Key Developments
Research “movement”
Federal funding interest
Discipline interest
International interest
Books
Journals
Conferences
Critics

5. Mixed Methods as a “Movement” “The emergence of mixed
methods as a third
methodological movement
in the social and behavioral
sciences began during the
1980’s.” (p. 697)
-Tashakkori &
Teddlie (2003)

6. New
NIH Awards Using
Mixed Methods

7. Development of NIH guidelines (1999)
“Combining qualitative and quantitative
methods has gained broad appeal in
public health research.” (1999)
- Mentioned several
approaches for combining
qualitative and
quantitative research
- Advanced considerations for deciding
what model to use (literature
available, prior studies,
realistic design, expertise)
- Suggested to describe each
method thoroughly

8. Social Work Summer Institute on Mixed-Methods Research
NIH Summer Institute: The Design and Conduct of Qualitative and Mixed-Method Research in Social Work and Other Health Professions
August 4 -8, 2004 
Table of Contents
Course Objectives
Dates and Location
Background
Concept
Institute Co-Chairs
Faculty/Presenters
Speaker Presentations
Eligibility Requirements
Number of Participants
Costs
Application Process and Receipt Date
Additional Information
Social Work Summer Institute
on Mixed-Methods
Research
Course Objectives
To provide a thorough grounding in the design and conduct of qualitative and mixed method research to social work researchers and other health professionals interested in developing competence in the planning, design, and execution of these techniques and become more successful in the NIH extramural research program.

9. National Science Foundation Workshop, 2003, Includes Mixed Methods Research
Provided general
guidelines for developing
qualitative research
projects
-Several papers
addressed mixed
methods research

10. Books on
Mixed Methods
Research
14+ books on mixed methods research
have been written
since 1988

11. Two New Books From
Sage Publications

12. A New Journal Journals Devoted to Mixed Methods: Fieldwork Quality and
Quantity
Mixed Method
Approaches
Editors: John W. Creswell
and Abbas Tashakkori
Managing Editor: Vicki L.
Plano Clark

13. Conferences
Mixed Methods Conference, Cambridge, England, July 21-24, 2008
Proposed mixed methods conference, Sydney, Australia, 2009
Discipline conferences with mixed methods papers

14. Critics (Creswell, 2007)
Is there a post-positivist leaning to mixed methods?
Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10,
Giddings, L. S. (2006). Mixed-methods research: Positivism dressed in drag? Journal of Research in Nursing, 11(3),
Holmes, C. A. (2007). Mixed(up) methods, methodology and interpretive frameworks. Contributed paper for the Mixed Methods Conference, Cambridge, University, July 10-12, 2006.
What are the limits for mixed methods research?
Sale, J. E. M., Lohfeld, L. H., Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Quality and Quantity, 36,
Leahey, E. (2007). Convergence and confidentiality? Limits to the implementation of mixed methodology. Social Science Research, 36,
Is there a dominant discourse in mixed methods research?
Freshwater, D. (2007). Reading mixed methods research: Contexts for criticism. Journal of Mixed Methods Research, 1(2),

15. Development #2: Evolving Understanding of What is Mixed Methods Research
Key Developments
The emergence of diverse perspectives - Two strands to “mixing” to “methodology” to “mixed methods within designs”
Incorporation of perspectives into current definitions
Questions arise about what is “mixing?”
Recent assessment of the value of mixed methods research (or mixing)

16. Framework for Viewing Perspectives on Mixed Methods
Quantitative Data
Qualitative Data
Mixed Methods
Method
Methodology
Paradigm
Perspective
Research Design
Procedures

17. Mixed methods is… a Method, a Design, a Methodology
Identifying a research problem
Reviewing the literature
Identifying a purpose and stating questions
Views of knowledge
Assumptions
Collecting data
Analyzing and interpreting data
Reporting and evaluating the study

18. Mixed Methods within Designs
Ethnography
Case Study Research
Narrative research
Experimental
research
Qual Quan
Qual----- Quan
Qual-----Quan
Qual---- Quan

19. What is this Method Called?
Multi-method
Triangulation
Integrated
Combined
Quantitative and qualitative methods
Multi-methodology
Mixed methodology
Mixed-method
Mixed research
Mixed methods

20. A Definition
Mixed methods research is a methodology for conducting research that involves collecting, analyzing, and integrating (or mixing) quantitative and qualitative research (and data) in a single study or a longitudinal program of inquiry. The purpose of this form of research is that both qualitative and quantitative research, in combination, provide a better understanding of a research problem or issue than either research approach alone.
Aspects of this definition:
Collecting both quantitative and qualitative data
Mixing the data (from merging in which both quan and qual loses identity to keeping them distinct but connected)
Single vs multiple-phase project
Raises issues: how, what, where, why, value (of mixing)

21. Mixing (How) Converge data: Results Connect data: Embed the data: Qual
Quan
Connect data:
Qual
Quan
Results
Embed the data:
Quan data
Qual data

22. Mixing What Where Data (Methods) Qualitative and quantitative research
Philosophical assumptions
Where
Throughout the process of research
Data collection
Data analysis
Interpretation

23. Mixing – Why? (Bryman, Qualitative Research, 2006)
Validity – to corroborate quan and qual data
Offset – offset weaknesses of quan and qual and draw on strengths
Completeness – more comprehensive account that qual or quan alone
Process – quan provides outcomes; qual, the processes
Different question – quan and qual answer different questions
Explanation – qual can explain quan results or vice-versa
Unexpected results – surprising results from one, other explains
Instr development – qual employed to design instr, then instr tested
Sampling – one approach facilitates sampling from other approach
Credibility – both approaches enhance integrity of findings
Context – qual provides context; quan provides general.
Illustration – qual data helps develop “depth” for quan data
Utility – more useful to practitioners
Confirm – quan tests qual generated hypotheses
Diversity of views – relationship and meaning; researcher/participant views
Enhancement – augmenting or building on one form of data with the other

24. Value of “Mixing”
“What does mixed methods research provide that either quantitative or qualitative research alone does not provide?”
Indicators of “yield” (O’Cathain, Murphy, & Nicholl, JMMR, 2007)
Integration as a measure of “yield”
Publications as a measure of “yield”
My speculations:
Results are useful
Stronger impact on policymakers
Cost-effective
Better solutions through interdisciplinary input
More evidence for a study

25. Development #3: Emerging Philosophical Perspectives as a Foundation for Mixed Methods Research
Key developments:
There are many stances taken by researchers.
“Purist” stance - Paradigms are incompatible (fixed, unmoveable), paradigms relate to methods, therefore mixing is impossible.
“Dialectic” stance - Multiple paradigms are possible and can be used, but honor and make explicit the paradigms.
“Single paradigm” stance - There is a single paradigm for mixed methods research: pragmatism, transformative.
“Design” stance - Multiple paradigms can be used – relate them to your design.
“Community” stance – source of belief systems come out of shared meanings and understanding

26. Philosophical Assumptions Behind Mixed Methods Research
Worldview or philosophy (e.g., attitudes and beliefs about knowledge, such as constructivism, post-positivism)
Theoretical lens (e.g., feminist, racial)
Methodological approach (e.g., experimental, survey, ethnography, mixed methods)
Methods of data collection (e.g. interviews, focus groups)
Worldview or philosophy – these refer to beliefs or issues regarding epistemology
These are theories such as ones from the social sciences, feminist theory , racial theory
This is the research approach that is used to conduct the study such as experimental research, survey, ethnography, grounded theory
Examples of methods of data collection include interviews, checklists, instruments, interviews, mixed methods
Adapted from Crotty M. (1998).

27. Four Worldviews Postpositivism Determination Reductionism
Empirical observation and measurement
Theory verification
Constructivism
Understanding
Multiple participant meanings
Social and historical construction
Theory generation
Advocacy/Participatory
Political
Empowerment issue-oriented
Collaborative
Change-oriented
Pragmatism
Consequences of actions
Problem-centered
Pluralistic
Real-world practice oriented

28. Purist Stance Smith & Heshusius, (1986) “Incompatibility thesis”
“The contention of this
paper is that the claim of
compatibility, let alone one
of synthesis, cannot be
sustained.” (p. 4)

29. “Dialectic Stance” Greene (2007)
Paradigms – philosophical assumptions,
knowledge, methodology, values
Paradigms are different – but are
social constructions – not sacrosanct
Differences should be respected,
intentionally used together
Used together, tensions achieve
dialectical discovery of enhanced,
reframed, or new understandings.

30. “Single” Paradigm Stance
Pragmatism (Tashakkori & Teddlie, 2003)
13 writers endorse
Based on Dewey, Rorty, Pierce, others (Americans)
Focus on research question
What “works,” consequences
Multiple methods – large toolkit
Transformative (Mertens, 2003)
Transform lives of underrepresented, marginalized groups
Incorporate this focus into all phases of research

31. “Design” Stance (Creswell & Plano Clark, 2007)
Multiple paradigms can be used
Different paradigms can provide the foundation for different phases of research
Often post-positivist paradigm provides the foundation for quantitative aspects of research
Often constructivist/advocacy paradigm influences qualitative aspects of research
Often pragmatism paradigm influences both qualitative and quantitative aspects of research
This does not suggest that certain paradigms
are linked to certain methods (paradigm-method fit)

32. “Community” Stance (Morgan, 2007)
Espouses the pragmatic approach
Shifts conversation to the importance of
shared meanings and joint actions between researchers.
“To what extent are two people (or two research fields) satisfied that they understand each other, and to what extent can they demonstrate the success of shared meaning by working together on common projects.” (Morgan, 2007, p. 67)

33. Development #4: Developing Designs and Innovative Techniques
Key Developments
Evaluation designs
Notation
Typologies, and a parsimonious set
Diagrams of procedures
Complex evaluation models
Reconceptualizing designs
The practice of research
Emergence of innovations in procedures
Unusual blends
Methodological issues
Data analysis techniques
Presentation techniques

34. Early Designs (Greene, Caracelli, & Graham, 1989) Triangulation
(convergence,
corroboration)
Complementarity
(clarify results from
one method to the
other)
Development
(one method informs
the other method)
Initiation (paradox,
contradiction)
Expansion (extends
breadth of inquiry)

35. Preliminary Design Considerations (Morse, 1991)
Approach
Type
Purpose
Limitations
Resolutions
QUAL + quan
Simultaneous
Enrich description of sample
Qualitative sample
Utilize normative data for comparison of results
QUAL
Sequential
Test emerging H, determine distribution of phenomenon in population
Draw adequate random sample from same population
QUAN + qual
To describe part of phenomena that cannot be quantified
Quantitative sample
Select appropriate theoretical sample from random sample
QUAN
To examine unexpected results
quan
qual

36. Concurrent Mixed Methods Designs
Parsimonious Designs (Creswell & Plano Clark, 2007)
Concurrent Mixed Methods Designs
Triangulation Design
QUAN Data & Results
Interpretation
QUAL Data & Results
Embedded Design
QUAN Pre-test Data & Results
QUAN Post-test Data & Results
Intervention
qual Process
Interpretation

37. Before-intervention qual After-intervention qual
Sequential Designs Mixed Methods Designs
Explanatory Design
QUAN Data & Results
Interpretation
qual Data & Results
Following up
Exploratory Design
QUAL Data & Results
quan Data & Results
Interpretation
Building to
Sequential Embedded Design
Before-intervention qual
QUAN Intervention
Trial
After-intervention qual
Interpretation

38. Helpful Tips for Drawing this Visual of the Design:
Give a title to the visual model.
Choose either horizontal or vertical layout for the model.
Draw boxes for quantitative and qualitative stages of data collection, data analysis and interpretation of the study results.
Use capitalized (QUAN) or small letters (quan) to designate priority of quantitative and qualitative data collection and analysis.
Use single-headed arrows to show the flow of procedures in the design.
Specify procedures for each quantitative and qualitative data collection and analysis stage.
Specify expected products or outcomes of each quantitative and qualitative data collection and analysis procedure.
Use concise language.
Make your visual diagram simple.
Size your visual diagram to one page.
Source: Ivankova, N. et al. (2006)

39. Example of
A Diagram

40. Complex Mixed Methods Evaluation Designs (Nastasi, Hitchcock,
Sarkar, Burkholder,
Varjas & Jayasena, 2007)
Study of mental health
interventions for youth
in Sri Lanka

41. Concurrent,
Sequential,
Recursive
Qual/Quan
Phases
(Nastasi et al.,
2007)

42. Reconceptualizing Designs
Three ways to look at mixed methods designs:
Typological approach (Bryman, 1988; Creswell & Plano Clark, 2007)
Systems theory (Maxwell & Loomis, 2003)
Synergistic approach - Hall & Howard (in press), “A Synergistic Approach: Conducting Mixed Methods Research with Typological and Systemic Design Considerations”
Combined effect greater than separate parts
Position of equal value of qual and quan
Ideology of difference – protect paradigm differences
Researcher as collaborator and relationship with design – skilled in both quan and qual or part of interdisciplinary team

43. The Practice of Conducting A Mixed Methods Study
Cordon & Hirst (in press) “Implementing a mixed methods approach to explore the financial implications of death of a life partner” – Social Policy Researchers, UK
Why they chose mixed methods
Reasons for their choice of design
Sampling decisions
Concept map
Discussions among team members

44. Technique - Designing a Quantitative Survey Based on Qualitative Findings
QUAL data analysis
Quotes
Codes
Themes
Quan data analysis
instrument development
Items on a survey
Variables on a survey
Scales on a survey

45. Technique- Matrix with Quantitative and Qualitative Data
Count
Adj.Count**
Row Pct
Column Pct
Patients
N=2
Physicians
N=4
Medical Assistants
Themes
Familiarity
With the Form 13
6.5
41.9
5.8 17
4.25
27.4
3.0 19
4.75
30.7
5.3 49
15.5
100.0
Reactions to the Form 23
11.5
22.2
10.2
100
25.0
48.3
17.4 61
15.3
29.5
16.9
184
51.75
Use for Managing Depression 67
33.5
38.6
29.8
177
44.25
51.0 36
9.0
10.4
10.0
280
86.75
Changes to the Form
115
57.5
37.5
51.1
196
49.0
32.0
34.0
187
46.8
30.5
51.7
498
153.3
Situational Use of the Form 7
3.5
8.9
3.1 86
21.5
54.4
14.9 58
14.5
36.7
16.1
151
39.5
225
112.5
576
144.0
361
90.3

46. Technique –Raising and Addressing Methodological Issues in Designs
Creswell, Plano Clark, Garrett (in press)
Contradictory findings
Data merging
Sample selection
Sample size
Introducing bias
Time

47. Technique – Unusual Blends of Data Collection
Qualitative
Post-modern philosophy and
Longitudinal data and
Qualitative data and
Qualitative themes matched
Qualitative discourse data
Quantitative
Quantitative data
Longitudinal data
Secondary survey data
With items on survey to produce new quan variables
Survey data

48. Other Developments: International emphases Discipline differences
Research team practices
The emergence of bilingual nomenclature (will employ this thinking in the workshop this afternoon as we “title” projects)

49. Four Developments (last 20 years)
Increasing interest in and advocacy for mixed methods
Evolving understanding of what is mixed methods research
Emerging philosophical perspectives as a foundation for mixed methods
Developing designs and innovative techniques

50. References
Bryman, A. (1988). Quantity and quality in social research. London: Routledge.
Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research 6(1),
Cordon, A., & Hirst, M. (in press). Implementing a mixed methods approach to explore the financial implications of death of a life partner. Journal of Mixed Methods Research.
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (in press). Methodological issues in conducting mixed methods research designs. In M. Bergman (Ed.), Advances in mixed methods research. London: Sage.
Crotty M. (1988) The foundations of social research: Meaning and perspective in the research process.
London: Sage.
Freshwater, D. (2007). Reading mixed methods research: Contexts for criticism. Journal of Mixed Methods Research, 1(2),
Giddings, L. S. (2006). Mixed-methods research: Positivism dressed in drag? Journal of Research in Nursing, 11(3),
Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274.
Hall, B., & Howard, K. (in press). A synergistic approach: Conducting mixed methods research with typological and systemic design considerations. Journal of Mixed Methods Research.
Holmes, C. A. (2007). Mixed(up) methods, methodology and interpretive frameworks. Contributed paper for the Mixed Methods Conference, Cambridge, University, July 10-12, 2006.
Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10,
Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3–20.
Leahey, E. (2007). Convergence and confidentiality? Limits to the implementation of mixed methodology. Social Science Research, 36,
Maxwell, J., & Loomis, D. (2003). Mixed methods design: An alternative approach. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp ). Thousand Oaks, CA: Sage.

51. References (Cont’d)
Mertens, D. M. (2003). Mixed methods and the politics of human research: The transformative-emancipatory perspective. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 135–164). Thousand Oaks, CA: Sage.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1),
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120–123.
Nastasi, B. K., Hitchcock, J., Sarkar, S., Burkholder, G., Varjas, K., & Jayasena, A. (2007). Mixed methods in intervention research: Theory to adaptation. Journal of Mixed Methods Research, 1(2),
National Institutes of Health (2004). NIH Summer Institute: The design and conduct of qualitative and mixed-method research in social work and other health professions, August 4-8, 2004.
National Institutes of Health. Office of Behavioral and Social Sciences Research (1999). Qualitative methods in health research: Opportunities and considerations in application and review. Washington D.C.: NIH.
National Science Foundation (2003). Workshop on scientific foundations of qualitative research. Washington D.C.: NSF.
O’Cathain, A., Murphy, E., & Nicholl, J. (2007). Integration and publications as indicators of “yield” from mixed methods studies. Journal of Mixed Methods Research 1(2),
Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage.
Sale, J. E. M., Lohfeld, L. H., Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Quality and Quantity, 36,
Smith, J. K., & Heshusius, L. (1986). Closing down the conversation: The end of the quantitative-qualitative debate among educational inquirers. Educational Researcher, 15(1), 4-12.
Tashakkori, A., & Teddlie, C. (Eds.). (2003a). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.

52. Mixed Methods Research: State of the Art (What Has Developed In Mixed Methods)
by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln
Co-editor, Journal of Mixed Methods Research, and
Co-Director, Office of Qualitative and Mixed Methods Research
© Please do not duplicate or use these slides without the express permission of the author.