1. Mixed Methods: Focus on Dissertations
Ryan Rominger, Ph.D., LCPC-PIT
Associate University Research Chair
Center for Leadership Studies and Organizational Research
Adjunct Research Faculty:
Sofia University, CIIS
Renew Wellness Center,
therapist-in-training

2. Mixed Methods Agenda for today: Dissertation Examples
Context for today’s discussion
Main ‘types’ of MM designs
Those pesky, yet informative, symbols & boxes
Dissertation Examples
Questions

6. Mixed Methods Assumptions
Often, philosophical foundation is pragmatism (use the method that gets the data which answers your questions)
If not, know what it is and how it influences method strands
Combining Qual and Quant help minimize problems within both
But if poor quality, can include problems of both
Combining helps ‘triangulate’ phenomenon to better understand what is happening
If not well coordinated, rather than 2 blind people feeling same elephant, can be 1 feeling elephant, another feeling a lion – i.e., need to have purposeful, meaningful coordination = ALIGNMENT

7. 5 Purposes of mixed method research
Triangulation – to confirm results, look at phenomenon from different angles, seeks convergence and corroboration
Complementarity – elaborate and clarify complex phenomena, gain information about different facets
Development – results from one method helps form the next method of inquiry
Initiation – not always intentional, used to look at dissonance between results that leads to generativity
Expansion – used to increase scope, different methods/strands do not measure same phenomenon
Eckhardt, A. L. (2013). Fatigue as a symptom of coronary heart disease. Doctoral dissertation.

8. Mixed Methods – main models
Sequential mixed methods
Researcher “seeks to elaborate on or expand on the findings of one method with another method” (C&PC, p. 14)
Quant  Qual or Qual  Quant
Concurrent mixed methods
Researcher “merges quantitative and qualitative data” (C&PC, p. 14) collection
Collects both forms of data at same time
Integrates the information during analysis & interpretation

9. Mixed Methods – main models
Transformative mixed methods
Uses overarching theoretical lens (or umbrella) which provides framework (C&PC, p. 15)
Collect multiple types of data to answer questions about the topic
Particularly important in interdisciplinary or transdisciplinary studies
Ex: In psychology, lens could be queer theory, investigating the expression of gender among transgender individuals in early adolescence

10. Mixed Methods – Research questions
Think about order of data collection.
If sequential, ask first RQ first, second second
If concurrent, ask RQ based on weight or importance – if quant more heavily weighted, start w/quant research hypothesis. If qual more heavily weighted, start with qual research question. Or, use ‘general’ RQ first.
Include a specific MM question.
Addresses mixing of data; could focus on procedures or content
At what point do you merge strands?
Data collection, analysis, synthesis, conclusions

11. Mixed Methods qual MM Strategies QUANT
Important concepts: Timing, Weighting, Mixing, Theory
+ = concurrent data collection
= sequential data collection
Capitalization = weight/priority
Quan/Quant = quantitative
Qual = qualitative
QUANT/qual = qualitative embedded in quantitative design
QUANT
qual

12. Mixed Methods - Graphics
QUANT
Data collection
Data analysis
qual
Sequential Explanatory Designs
Interpretation of entire analysis
Example: Studying “significant life events” during college of recent college graduates. QUANT – 500 college students complete survey. qual – 30 “exemplar” students chosen and interviews about their experience. Interpretation done after all data collected and analyzed.

13. Mixed Methods - graphics
Concurrent Embedded Design (b) #1
QUANT
QUAL #2
qual
quant
Analysis of Findings
Analysis of Findings
Ex #1: Efficacy based RCT experiment on meditation with 6th graders to help increase focus (QUANT). Interview several of these students to find out what it was like for them (qual).

14. Decision Based Learning Model for Mixed Methods Designs
Would a purely quant study answer your RQ?
Yes
use Quant no
would a purely qual study answer your RQ?
yes
use QUAL
Consider a MM design

15. Do you need one set of data to inform the collection of the next set?
yes
Use Sequential designs no
Will one data strand be primary?
use embedded designs
use a concurrent design

16. Will quantitative data be primary?
Will Qualitative data be primary?
yes
It is likely Exploratory no
Will quantitative data be primary?
It is likely Explanatory
Then quant and qual are equally weighted

17. Dissertation examples
Burrows, T. J. (2013). A preliminary rubric design to evaluate mixed methods research. Virginia Polytechnic Institute and State University.
Method:
Exploratory multi-phase mixed-method content analysis study expert participants. Qual then quant.
RQ: What are the criteria for evaluating mixed-method research?
Critique:
Clearly written, and both method and topic are MM research=alignment.
Used visual representations of the method and timing of data collection and analysis.
Methods chapter organized by phase of study, with clear descriptions of analysis. Themes from phase 1 resulted in rubric to be tested in phase 2.

18. Rubric for evaluating MM Research

20. Dissertation examples
Eckhardt, A. L. (2013). Fatigue as a symptom of coronary heart disease. University of Illinois at Chicago.
Method:
Series of articles/manuscripts combined into ‘dissertation’.
partially mixed sequential dominant status design (QUAN → qual).
QUAN with 102 participants completing several measures (fatigue, mood, quality of life, etc.), qual with purposive sample subset who were interviewed about fatigue.
Critique:
**Great description of different types of mixed methods, including examples of different types of MM designs, qualities of MM research.
Provides a unique approach to a dissertation (several manuscripts) with in- depth discussion of MM designs, theory, and paradigms of research especially with regard to mixing of paradigms (positivist and constructivist).

22. O’Cathain, A. (2010). Assessing the quality of mixed methods research: Toward a
comprehensive framework. In A. Tashakkori & C. Teddlie (Eds.), SAGE Handbook of
Mixed Methods in Social and Behavioral Research (pp ). Los Angeles: SAGE.
As cited in Eckhardt, A. L. (2013). Fatigue as a symptom of coronary heart disease.
University of Illinois at Chicago.

23. Dissertation examples
Wright, J. B. (2018). Physician executives – A mixed-methods examination of leadership characteristics and how physicians learn to lead. University of Charleston-West Virginia.
Method:
Called a explanatory sequential mixed-methods interpretive phenomenological analysis, with goal of determining physician characteristics connected to good leadership
Surveyed and interviewed 6 physician executives to discuss discuss “what competencies, thought processes, and techniques are important for future physician executives in healthcare to lead effectively”
Critique:
Quant ‘strand’ is weak with only 6 participants; limits conclusions of group analysis used in dissertation. In MM it is best to have strong quant and qual strands.
Writes of ‘phenomenological’ interviews, but only a single one-hour interview with each participant. Maybe better described as hermeneutical RM, although have seen IPA with only 1 interview.

24. Dissertation examples
Stephan, K. P. (2017). Does Mathletics, a supplementary digital math tool, improve student learning and teaching methods at three private Catholic schools in Florida? – A mixed methods study. Creighton University.
Method:
Called a convergent, parallel mixed methods study.
Quant strand used comparison groups (6th grade classes) and Mathletics groups, comparing Iowa Test of Basic Skills (outcome).Qual strand used interviews with teachers.
Critique:
Similar to program evaluation models, as this is essentially a outcome based program evaluation to assess whether implementation would benefit upstream and downstream constituents.
Clear research questions for each strand, convergent process after separate parallel data collection and analysis, and Qual definitely expanded the Quant results. Member checking, comparison table, etc. to improve validity.
No graphic representation.

25. Dissertation examples
Boyle, E. D. (2018). Executive selection process of U.S. defense contractors: A mixed methods study. University of Phoenix.
Method:
Called a convergent, parallel mixed methods study. (No graphic)
2 pilot tests and main study. Main study had 37 in quant strand, and qual strand had 3 participants. Researcher did not meet his goals of 159 in quant and 6-12 in qual after 3 months of participant recruitment.
Critique:
Strong argument for use of mixed method design. Strong, statistical argument for number needed in quant strand. However, did not meet participant number goals, which could have made study stronger.
While survey, part 2 of which used Likert scales, was created by the author (thus no prior reliability and validity), there was a theoretical base for the questions AND author conducted 2 pilot tests to test content validity and reliability.
Qual results provided confirmation of quant results.

26. Mixed Methods Challenges Can take more time
Can take more resources (money, personnel)
Need skills for both Qual and Quant methods
Individual
Team
Trouble determining when to merge/synthesize strands
Trouble with unclear research questions – note suggestion to have one overall ‘mixed method’ RQ and within that, method specific RQs
Some combinations do not mix well/easily
Methods – classical phenom & large scale quant
Theories – positivist (RCT) and constructivist (narrative)

27. Questions?
Next Webinar:
DBL Model & Software
April 4:00 AZ time