ALP CONFERENCE / June xx, COMMUNITY COLLEGE RESEARCH CENTER June 14, 2013 Nikki Edgecombe, Community College Research Center Hunter Boylan, National Center for Developmental Education, Appalachian State University David Crook, University Dean for Institutional Research and Assessment, City University of New York The 5 th Annual Conference on Acceleration in Developmental Education Baltimore, MD Translating Research to Inform Developmental Education Policy and Practice

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER The body of research on developmental education is growing It can be hard to discern the quality of research and significance of findings Research findings may not be actionable or may inspire a potential range of actions Considerations for translating research to practice How do you effectively use research to inform developmental education improvement efforts? What do you need to consider in doing so?

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Types of research What kind of information are they designed to generate? What are their benefits and limitations? Translating research into practice Framing the process Lessons from researchers and policymakers What are your experiences translating research into practice? Today’s Presentation

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Impact studies Program evaluations Implementation analysis Developmental education research may include: Other common forms of research include practitioner research and descriptive outcomes analyses.

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Impact Studies PurposeMeasure the effectiveness of developmental education Research DesignRigorous experimental designs including randomized control trials and regression discontinuity that allow the researcher to “test” a treatment by comparing outcomes of similar individuals and infer causality LimitationsResults may not be generalizable to other populations or contexts RCTs can be challenging to administer and expensive and require large sample sizes Regression discontinuity results only apply to subjects near the treatment/no treatment cut point Unable to explain why or how a treatment works ExampleAttewell, P. A., Lavin, D. E., Domisna, T., & Levey, T. (2006). New evidence on college remediation. Journal of Higher Education, 77(5), 886–924.

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Program Evaluations PurposeExamine the implementation and/or effectiveness of an intervention implemented at a single institution or across multiple sites Research DesignPotential range of quantitative and qualitative research methods Quantitative methods may include experimental or quasi-experimental analyses or descriptive statistics and typically measure impacts Qualitative evaluations may require a variety of data collection activities, including interviews, focus groups, observations, surveys and document review and typically assess processes and context LimitationsResults may have limited generalizability, particularly the qualitative outcomes Challenging to identify context-specific features that affect implementation and outcomes Results likely to vary depending on maturity of intervention ExamplesVisher, M., Weiss, M. Weissman, E., Rudd, T. & Wathington, H. (2012). The effects of learning communities for students in developmental education. A synthesis of findings from six community colleges. MDRC. Edgecombe, N., Jaggars, S. S., Baker, E. D., & Bailey, T. (2013). Acceleration through a holistic support model: An implementation and outcomes analysis of Community College Research Center.

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Implementation Analyses PurposeExamine how reforms are implemented and the implications of those processes for outcomes Research DesignPotential range of qualitative and quantitative research methods, though more likely qualitative Qualitative analyses document the opportunities and challenges associated with initiating, refining, expanding, and institutionalizing reforms LimitationsTypically do not provide evidence on effectiveness Empirical results may not be generalizable though conceptual frameworks may be broadly applicable ExamplesEdgecombe, N., Cormier, M., Bickerstaff, S. & Barragan, M. (2013). Strengthening developmental education reforms: Evidence on implementation efforts from the Scaling Innovation project. Community College Research Center. Quint, J., Byndloss, D. C., Collado, H., Gardenhire, A., Magazinnik, A., Orr, G., Welbeck, R., Jaggars, S. S. (2011). Scaling up is hard to do: Progress and challenges during the first year of the Achieving the Dream Developmental Education Initiative. MDRC.

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Translating Research into Practice: Teaching and Learning Research TopicFindingInterpretationPractice Students’ understanding of developmental math There are few students able to demonstrate conceptual understanding of mathematics; yet those who do are more likely to answer problem correctly Increased conceptual understanding may lead to better learning outcomes in developmental math Provide ongoing training to teachers on implementing curriculum that explicitly requires students to draw connections between mathematical concepts and procedures

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Translating Research Into Practice: Individual Reform Research TopicFindingInterpretationPractice Holistic developmental education model Program participation has short-term benefits but is not associated with longer-term outcomes like persistence or accumulation of college-level credits Single-semester interventions may not provide the breadth or depth to to influence long-term outcomes Implement developmental education reforms in conjunction with other long-term student support strategies

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER Translating Research Into Practice: Institutional Policy Research TopicFindingInterpretationPractice Assessment and placement policy Placement tests are associated with “severe” placement errors, particularly under-placements Policy must address vulnerabilities in dominant assessment and placement system and not assign students to remediation who could be successful in college courses Implement a placement system using multiple measures, not a single test score

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER It is important to weigh the strengths and limitations of the research as you utilize it to improve developmental education Most developmental education reforms have generated only small and short- term effects, which may be attributable to: Concessions made for pragmatic reasons that can undermine implementation Propensity to implement small-scale and minimally disruptive reforms A lack of sustained, targeted supports that span students’ college careers We need a stronger understanding of teaching and learning in developmental education; therefore, systematic pedagogical experimentation and documentation is essential Translating Research Into Practice Takeaways

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER How would you describe the state of the research on developmental education? Where do we have strong understandings? What areas need further exploration? How do you use research on developmental education in your work? If you were advising policymakers and practitioners, what recommendations would you have in terms of the most effective ways to use research to help to improve the outcomes of students referred to developmental education? What risks or pitfalls are inherent in that process? Questions for Panelists

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER How does research shape the decisions you make? What types of research do you find most helpful? What types of research would you like to see more of and why? Questions for Discussion

JUNE 14, 2013 COMMUNITY COLLEGE RESEARCH CENTER JUNE 14, 2013 For more information Please visit us on the web at where you can download presentations, reports, and briefs, and sign-up for news announcements. We’re also on Facebook and Twitter. Community College Research Center Institute on Education and the Economy, Teachers College, Columbia University 525 West 120th Street, Box 174, New York, NY Telephone: