1. About this Template In the past decades, mixed methods research has been increasingly applied to the dominant research fields of social, behavioral or human sciences. It was widely recognized as the third research approach or paradigm aligning with the quantitative and qualitative research methods. Creswell and Clark (2007) defined mixed methods research clearly: a research design or methodology with philosophical assumptions as well as methods of inquiry, in which both quantitative and qualitative data were collected, analyzed and mixed in a single study. A mixed methods study is most suitable if a single method failed to develop a comprehensive perspective to pose good research questions based on certain phenomena. It could also enhance validity, offset weaknesses of either approach alone, and assort the divergent views (Greene, 1989; Creswell & Clark, 2007; Tashakkori & Teddlie, 2010). However, the researchers could not reach a consensus about rigorous criteria to evaluate and ensure the high quality of mixed methods studies. Grounded on the previous studies on evaluation of mixed methods, this research is intended to develop and validate three criteria. The criteria can be used to evaluate mixed methods studies, assist future researchers in identifying what mixed methods research is, and clarify what crucial elements mixed methods research studies should include. There are some preliminary considerations for qualifying as a good mixed methods study. Typology is regarded as one of the most valuable criteria used by various researchers in mixed methods design because it can provide the paths and types with an organizational structure when they try to decide upon their research procedure. It is also a useful pedagogical tool to help legitimize Mixed methods and distinguish mixed methods from either a quantitative or qualitative design (Teddlie & Tashakkori, 2009, p.139). rationale for the use of mixed methods will clarify and justify why a mixed methods approach is necessary or superior to others (O’ Cathain et al., 2008). The amount of mixing or integration will provide a better understanding of the research problem than a single method (Creswell & Clark, 2007). The three criteria were employed to evaluate not only the content of each methodological component but also the quality and appropriateness of each component under the context of each study. Table 2. Evaluation Results for Three Studies. Introduction References Three Criteria to Evaluate Mixed Methods Studies Yingqi Wang, Educational Research and Evaluation, Virginia Tech Result Literature Review Articles Typology of Mixed Methods Design Rationale of Mixed Methods Amount of Integration (mixing) Total Wittink, Barg & Gallo, 2006. 1 needs work0 poor2 meets standard3 needs work Ivankova & Stick, 2007. 3 excellent2 meets standard3 excellent8 excellent Siwatu, 2011.3 excellent0 poor3 excellent6 meets standard As mentioned in the previous discussion about the typology of the mixed methods, due to the multiple classifications of mixed methods designs, there exists a problem of pigeonholing a particular mixed method design based on a specific typology. Therefore, the researcher could not place one of three studies in my typology as it failed to clearly introduce its study design. The evaluation results based on the second criterion for the rationale show that only one study chose to provide a rationale of adopting the mixed methods approach. When evaluating the amount of mixing in the studies, the researcher found that the mixing occurred most frequently during the inferential or interpretation stage and experiential stage. In the future study, more samples will be introduced to test the generalizability of the criteria Discussion Table 1. Rubric for Evaluation of Mixed Methods Studies. Criteria0. Poor 1. Needs work 2. Meets standard 3. Excellent Typology of Mixed Methods Designs No information related to the typology of mixed methods design can be identified. The typology of mixed methods designs is incomplete and/or unfocused. The typology of mixed methods designs is clearly but partially stated. The typology of mixed methods designs is completely and explicitly stated. Rationale of Mixed Methods No information related to the rationale of mixed methods design can be identified. The rationale of mixed methods is incomplete and /or unfocused. The rationale of mixed methods is clearly but partially stated. The rationale of mixed methods is completely and explicitly stated. Amount of Integration (mixing) No information related to the mixing of mixed methods design can be identified. Mixing occurs in the one stage in the study. Mixing occurs in the two stages in the study. Mixing occurs in the three stages in the study. 8 out of 9 points, the strongest and most highly qualified mixed method study among the three. Ivankova & Stick (2007) 6 points, the research has met the basic standards of a mixed methods study and got 6 points by losing 3 points for lacking of the statement of a rationale. Siwatu’s (2011) 3 points, which means this study may need more work to achieve higher quality as mixed method research. Wittink et al. (2006) Typology Even though there existed over 40 different types of mixed methods designs, we could generally classify them into five major types including triangulation, explanatory, exploratory, embedded designs and others. Rationale The excellent rationale should be explicitly and clearly introduce the reason(s) for adopting the mixed methods approach. Amount of mixing Excellent mixed methods study should have evidence to show that there is a certain amount of mixing during the process of design, data collection, data analysis, and interpretation (Creswell & Clark, 2007). Method Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage. Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational evaluation and policy analysis, 11(3), 255-274. Ivankova, N. V., & Stick, S. L. (2007). Students’ persistence in a distributed doctoral program in educational leadership in higher education. Research in Higher Education, 48 (1), 93-135. O’Cathain, A., Murphy, E., & Nicholl, J. (2008). The quality of mixed method studies in health services research. Journal of Health Services Research Policy, 13 (2), 92-98. Siwatu, K. (2011). Preservice Teachers' Culturally Responsive Teaching Self-Efficacy-Forming Experiences: A Mixed Methods Study. Journal Of Educational Research, 104(5), 360-369. doi:10.1080/00220671.2010.487081 Teddlie, C., A. & Tashakkori. (2009). Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Thousand Oaks, CA: Sage. Wittink, M. N., Barg, F. K., & Gallo, J. J. (July/August 2006). Unwritten rules of talking to doctors about depression: Integrating qualitative and quantitative methods. Annals of Family Medicine, 4(4), 302-3 Three studies from the educational field were evaluated by the three criteria to test their validity. Based on the three 4-level mixed methods criteria, we can place the three studies on three levels of quality (See table 2). In the present study, each component of a mixed methods study is assessed by assigning a score (from zero to 3 points) according to 4 levels of each criterion ( See table 1).