Planning a Mixed Methods Study by John W. Creswell, Ph.D., Department of Educational Psychology, University of Nebraska-Lincoln Co-editor, Journal of Mixed Methods Research, and Co-Director, Office of Qualitative and Mixed Methods Research Presentation made to Social Work-Nursing Graduate Students, University of Michigan, February 5, 2008 © Please do not duplicate or use these slides without the express permission of the author.

“Products” of Planning Dissertation or masters thesis Proposal for external funding Planning the steps that will be undertaken in the research Focus on a dissertation or masters thesis Let’s not get stuck – value in seeing the entire planning process

Our Agenda: View typical contents for a proposal for mixed methods research Select steps in the process of planning a study Design a mixed methods study as a group using one example in a “guided design” approach Assemble the final plan in order of standard presentation of a plan Individuals can plan their own studies as we go through “guided design” Share the final plans at the end of the session

Outline of the Structure of a Proposal for a Dissertation or Thesis (Creswell & Plano Clark, 2007, p. 155) Title (foreshadows mixed methods study and type of design) Introduction The research problem Past research on the problem Deficiencies in past research and one deficiency related to a need to collect both quantitative and qualitative data The audiences that will profit from the study Purpose The purpose or study aim of the project (use the scripts) and reasons for mixed methods The research questions Quantitative, qualitative, mixed methods research questions/hypotheses Philosophical foundations for using mixed methods and the type of design (optional) Literature Review (include quantitative, qualitative, and mixed methods studies, if they are available) Methods A definition of mixed methods research The type of design used, and its definition Challenges in using this design Examples of using the type of design (in your field, if possible) Reference and inclusion of a visual diagram Quantitative data collection and analysis Qualitative data collection and analysis Mixed methods data analysis procedures Validity approaches in both quantitative and qualitative research Researcher’s Resources and Skills Potential Ethical Issues Timeline for Completing the Study References/Appendices with instruments/protocols

Components of a Plan that We will Design Title (foreshadows mixed methods study and type of design) Introduction The research problem Past research on the problem Deficiencies in past research and one deficiency related to a need to collect both quantitative and qualitative data The audiences that will profit from the study Purpose The purpose or study aim of the project and reasons for mixed methods The research questions/hypotheses Quantitative, qualitative, mixed methods research questions/hypotheses (create separate sections for each set of questions Philosophical foundations for using mixed methods and the type of design (optional) Literature Review (include quantitative, qualitative, and mixed methods studies, if they are available) Methods A definition of mixed methods research The type of design used, and its definition Challenges in using this design Examples of using the type of design (in your field, if possible) Reference and inclusion of a visual diagram Quantitative data collection and analysis Qualitative data collection and analysis Mixed methods data analysis procedures Validity approaches in both quantitative and qualitative research Researcher’s Resources and Skills Potential Ethical Issues Timeline for Completing the Study References/Appendices with instruments/protocols

Components That We Will Design (in order that we will address them ) Title The research problem – Content intent Content issue(s) Reasons for mixed methods Researcher’s resources and skills Definition of mixed methods Qualitative data collection and analysis Quantitative data collection and analysis Research questions (qualitative, quantitative, mixed) Type of design Visual diagram Philosophical foundation Purpose statement

Designing a title Topic Writing the title Short Identify participants and site Consider a neutral title, neither quantitative or qualitative in wording Include the words “mixed methods”

Title

The Research Problem Intent of the study What are you trying to accomplish in this study? One sentence Narrow down to one overriding goal Problem What content issue are you addressing? Real life problem Deficiency in the literature Practice needs of audience All of the above

Intent of the Study

Research Problem (Issue)

Reasons for Mixed Methods Why are you planning to use mixed methods in your study? What purpose (research objective) will it accomplish? Why do you plan on gathering both quan and qual data? Think about this from a research methods perspective.

Reasons for Conducting a Mixed Methods Study (Bryman, Qualitative Research, 2006) Validity – to corroborate quan and qual data Offset – offset weaknesses of quan and qual and draw on strengths Completeness – more comprehensive account that qual/qual alone Process – quan provides outcomes; qual, the processes Different question – quan and qual answer different questions Explanation – qual can explain quan results or vice-versa Unexpected results – surprising results from one, other explains Instr development – qual employed to design instr, then instr tested Sampling – one approach facilitates sampling from other approach Credibility – both approaches enhance integrity of findings Context – qual provides context; quan provides general. Illustration – qual data helps develop “depth” for quan data Utility – more useful to practitioners Confirm – quan tests qual generated hypotheses Diversity of views – relationship and meaning; researcher/participant views Enhancement – augmenting or building on one form of data with the other

Reasons for Conducting Mixed Methods

Research Skills and Resources Are you qualified to conduct a mixed methods study? Quantitative coursework Qualitative coursework Personal experience conducting a mixed methods study Support from your committee/support from your field Time and resources

Research Skills and Resources

Definition of Mixed Methods How would you describe this approach? Focus on methods? Focus on methodology? Focus on paradigm/worldview? Focus on application to traditional design? Aspects I would include: Definition Citations to studies that have employed it Major characteristics of the design

Methodology – Definition of Mixed Methods Research Mixed methods research is a design for collecting, analyzing, and mixing both quantitative and qualitative research (or data) in a single study or series of studies to understand a research problem. (adapted from Creswell and Plano Clark, 2007)

Sites/Individuals to be Studied Quantitative sample Qualitative sample Sampling

Types of Quantitative and Qualitative Data to be Collected Quantitative data Instruments Checklists Records Qualitative data Interviews Observations Documents Audio-visual materials Quantitative data Close-ended scales Attitudinal/behavioral scales Behavioral checklists Census, attendance records Qualitative data Open-ended responses Semi-structured interviews Semi-structured observations Records/documents Videotapes Images such as pictures or drawings

Data Collection Site to be studied Participants to be studied Forms of data collection Quan Qual

Quantitative and Qualitative Data Analysis Quantitative analysis Use statistical analysis, For description For comparing groups For relating variables Qualitative analysis Use text and images, For coding For theme development For relating themes Quantitative analysis Numeric data Descriptive trend analysis Hypothesis testing, effect size, interval estimates Qualitative analysis: Text/image data Coding Themes Description Interrelated themes to build a picture of the phenomenon.

Data Analysis Qualitative Quantitative

Research Questions Quantitative questions Qualitative questions Describing variables Relating variables Comparing variables Think in terms of i.v., d.v. at least (narrowed, closed-ended) Theory Qualitative questions Exploring a single phenomenon General, open-ended Mixed methods questions What will you learn by “mixing” Combine quan and qual and what you hope to achieve by it (reasons)

Research Questions Quantitative questions Qualitative questions Mixed method question

Type of Mixed Methods Design Will the quan and qual data be collected: Timing Sequentially? – one builds on the other Concurrently? – both are collected at the same time Emphasis Quan emphasized? Qual emphasized? Mixing Merging? Connecting? Embedding?

Concurrent Mixed Methods Designs Triangulation Design QUAN Data & Results Interpretation QUAL Data & Results Embedded Design QUAN Pre-test Data & Results QUAN Post-test Data & Results Intervention qual Process Interpretation

Before-intervention qual After-intervention qual Sequential Designs Mixed Methods Designs Explanatory Design QUAN Data & Results Interpretation qual Data & Results Following up Exploratory Design QUAL Data & Results quan Data & Results Interpretation Building to Sequential Embedded Design Before-intervention qual QUAN Intervention Trial After-intervention qual Interpretation

Worldview or Paradigm for Study Postpositivism Determination Reductionism Empirical observation and measurement Theory verification Constructivism Understanding Multiple participant meanings Social and historical construction Theory generation Advocacy/Participatory Political Empowerment issue-oriented Collaborative Change-oriented Pragmatism Consequences of actions Problem-centered Pluralistic Real-world practice oriented

Purpose Statement Most important statement in your project plan Follow one of my scripts to get started (can adapt later) Elements of scripts: Intent (from earlier) Include qualitative purpose statement (central phenomenon, data collection, participants, site) Include quantitative purpose statement (variables, data collection, participants, site) Reason for a mixed methods study

Explanatory Sequential QUAN qual The intent of this two-phase, sequential mixed methods study will be to ______________________ (mention content objective of the study). In the first phase, quantitative research questions or hypotheses will address the ___________ relationship or comparison of ___________ (independent) and ___________ (dependent) variables with ___________ (participants) at ______________ (the research site). Information from this first phase will be explored further in a second qualitative phase. In the second phase, qualitative interviews or observations will be used to probe significant _______________ (quantitative results) by exploring aspects of the _____________ (central phenomenon) with ______________ (a few participants) at ____________ (research site). The reason for following up with qualitative research in the second phase is to __________________ (e.g., better understand and explain the quantitative results).

Exploratory Sequential QUAL quan The intent of this two-phase, sequential mixed methods study will be to _______________ (mention content objective of the study). The first phase will be a qualitative exploration of a ____________ (central phenomenon) by collecting ___________ (types of data) from __________ (participants) at ___________ (research site). Findings from this qualitative phase will then be used to test ________ (a theory, research questions, or hypotheses) that ____________ (relate, compare) ____________ (independent variable) with _____________ (dependent variable) for _________ (sample of population) at ___________ (research site). The reason for collecting qualitative data initially is that ______________ (e.g., instruments are inadequate or not available, variables are not known, there is little guiding theory, or few taxonomies).

Concurrent QUAL + QUAN The intent of this concurrent mixed methods study is to __________________ (content objective of the study). In the study, ___________ (quantitative instruments) will be used to measure the relationship between _____________ (independent variable) and ____________ (dependent variable). At the same time, the _____________ (central phenomenon) will be explored using _______________ (qualitative interviews or observations) with ___________ (participants) at _______________ (the research site). The reason for combining both quantitative and qualitative data is to better understand a research problem by converging both quantitative (broad numeric trends) and qualitative (detailed views) data.

Embedded QUAN(qual) or QUAL(quan) This mixed methods study will address _________________ (overall content aim of the study). An embedded mixed method design will be used, and it is a design in which one data set provides a supportive, secondary role in a study based primarily on the other data set. The primary purpose of this study will use ________________ (quantitative instruments) to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). A secondary purpose will be to gather qualitative data _______________ (type of qualitative data, such as interviews) that will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting the secondary database is ________________ (e.g., to address different question, to provide support for the primary purpose).

Planning a Mixed Methods Study by John W. Creswell, Ph.D., Department of Educational Psychology, University of Nebraska-Lincoln Co-editor, Journal of Mixed Methods Research, and Co-Director, Office of Qualitative and Mixed Methods Research Presentation made to Social Work-Nursing Graduate Students, University of Michigan, February 5, 2008 © Please do not duplicate or use these slides without the express permission of the author.