1. + Using Mixed Methods Research Designs for Research in Teaching and Learning Dr. Elizabeth G. Creamer, Virginia Tech Dr. Beth Mac Donald, Utah State University Julaine Fowlin, Virginia Tech

2. + Goals for the Session 1. Model ways to design a feasible mixed methods research study. 2. Review strategies to maximize the benefits of a mixed methods approach. 3. Consider challenges to designing and conducting a mixed methods dissertation. 2014 CIDER Conference, February 5-7, 2014 2

3. + Agenda 1. Introductions and introductory remarks – Creamer Model 1: Exploratory/Confirmatory Study 2. Model 2 and Example from Math Education – MacDonald 3. Model 3 and Example from Distance Education – Fowlin 4. Challenges to Using Mixed Methods in Dissertation Research 2014 CIDER Conference, February 5-7, 2014 3

4. + Key Elements of the Definition of MMR Both a method and a methodology Contains both a qualitative (e.g. emergent or inductive) and a quantitative (e.g. deductive, hypothesis testing) strand Requires MIXING or the integration of the qualitative and quantitative strands in on or more phases of the research process: Design Sampling Data Collection Analysis Inferences and conclusions 2014 CIDER Conference, February 5-7, 2014 4

5. + Arches as a Metaphor for Mixing 2014 CIDER Conference, February 5-7, 2014 5

6. + Combining QUANT and QUAL Strengths for a Stronger Study PhasesQUANTITATIVEQUALITATIVE DesignWhat research questions What, how, and why research questions Data CollectionNumbersWords SamplingPotential generalizability Pursue extreme, negative, or exemplary cases AnalysisConfirmatoryExploratory 2014 CIDER Conference, February 5-7, 2014 6 From E. G. Creamer (in progress), Introduction to fully integrated mixed methods research. SAGE Publisher.

7. + TRIANGULATION: The Most Common MM Design 2014 CIDER Conference, February 5-7, 2014 7 DATA COLLECTION IN ONE PHASE – OFTEN A SURVEY QUANT DATA- Likert Items QUAL DATA – open ended questions

8. + TRIANGULATION DESIGN – TAKING ADVANTAGE OF STRENGTHS DESIGN – Adds potential to ask how or why questions ANALYSIS If open-ended questions are analyzed inductively, adds an exploratory component Potential to show relationships between QUAN and QUANT data Potential to analyze potential predictive power of QUAL variables 2014 CIDER Conference, February 5-7, 2014 8

9. + The MM Development Design 2014 CIDER Conference, February 5-7, 2014 9 QUAL INTERVIEWS DEVELOP A SURVEY PILOT TEST THE SURVEY Dissertation Your Next Life

10. + Mixed Methods Research and Mathematics Education Rely on pragmatic paradigms which offer a blend of worldviews (i.e. radical and social constructivism). Explains learning a process, not a product Learning is investigated through interactions between: Student and Student Students and Teachers Students and Tasks 2014 CIDER Conference, February 5-7, 2014 10

11. + Model #2: Mathematics Education 2014 CIDER Conference, February 5-7, 2014 11 Analysis of QUAL Survey Data Analysis of QUAN Survey Data Treatment Groups and Control Group Blended Experiences to influence PST knowledge and beliefs Blended Conclusions describe how differing experiences changed PST content knowledge and PST beliefs Research Purpose and Paradigms are blended to support Pragmatic perspective QUAN and QUAL Survey given to all groups

12. + Mixed Methods Mathematics Education Research: Maximizing the Benefits of Mixed Methods Research 2014 CIDER Conference, February 5-7, 2014 12 PHASEQUALITY ISSUE DESIGN QUALITYPragmatic research questions are more closely aligned to real issues at the classroom level. Designs alter traditional experimental designs to better investigate blended research questions. DATA & ANALYSIS QUALITY Data and Analysis are collected separately, but conclusions are drawn from a blend of both analysis. INFERENCE QUALITYData are triangulated and confirmed through cohesive discussions from multiple perspectives.

13. + What Mathematical Knowledge Matters and What Evidence Counts? “Qualitative researchers have much to learn from large-scale test developers; large-scale test developers need lessons learned in qualitative research to succeed. More crossover needs to occur between these two separate enterprises” (Hill, Sleep, Lewis, & Ball, 2007, pp. 151). Hill, Sleep, Lewis, & Ball (2007). Assessing teachers’ mathematical knowledge. In F. K. Lester, Jr. (Ed.) Second handbook of research on mathematics teaching and learning (pp. 111-156). Charlotte, NC: Information Age Publishing 2014 CIDER Conference, February 5-7, 2014 13

14. + Mixed Methods and Distance Education Provides a more holistic picture of the learning situation Allows for advancement to the field through instrument and framework development Allows for comprehensive program evaluations 2014 CIDER Conference, February 5-7, 2014 14

15. + Model # 3 : Distance Education Research 2014 CIDER Conference, February 5-7, 2014 15 Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20. Quant Data Collection Survey N=278 RQ: What factors? Quant Data Collection Survey N=278 RQ: What factors? Quant Data Analysis Quant Data Analysis Mixing Quant results used to select Qual participants and create interview protocol QUAL Data Collection Case Study N=4 RQ: Why those factors? QUAL Data Analysis Mixing Interpretation using Quant and Qual results

16. + Challenges in Mixed Methods Research Meaningful Mixing- Creamer Blending of Paradigms-MacDonald Issues with Priority in MM Designs-Fowlin 2014 CIDER Conference, February 5-7, 2014 16