1. Developing a Program of Postsecondary Academic Instruction Over the Transforming Lives Network Improving Evidence of Impact through a National Study of Postsecondary Academic Programming in State prisons Dr. Stephen Meyer, Principal Investigator RMC Research Corporation Cindy Borden & Penny Richardson, Field Investigators Northstar Correctional Education Services Dr. Stephen Steurer, Project Director Correctional Education Association CEA Leadership Forum March 10, 2008

2. This Session Research in Correctional Education Research in Correctional Education Promotion of Rigorous Research in Education Promotion of Rigorous Research in Education Current State of Correctional Postsecondary Academic Programming Current State of Correctional Postsecondary Academic Programming College of the Air and the Transforming Lives Network College of the Air and the Transforming Lives Network Overview of the Study Overview of the Study More Detailed Information about Incentives, Data Collection Activities, Timeline, and Expectations of Participating Sites More Detailed Information about Incentives, Data Collection Activities, Timeline, and Expectations of Participating Sites

3. Research in Correctional Education

4. Example: Chappell (2004) Review of Postsecondary CE and Recidivism Studies Conducted in the 1990s Review of Postsecondary CE and Recidivism Studies Conducted in the 1990s Found a correlation (.31) between PSCE and recidivism Found a correlation (.31) between PSCE and recidivism Only 15 studies met criteria for inclusion. These studies had substantial limitations: Only 15 studies met criteria for inclusion. These studies had substantial limitations: –Selection bias; control groups for only 3 studies –Failure to distinguish between secondary and postsecondary –Different definitions of PSCE and recidivisim Chappell, C.A. (2004). Post-Secondary Correctional Education and Recidivism: A Meta-Analysis of Research Conducted 1990-1999. Journal of Correctional Education, June.

5. Example: Wilson, Gallagher, & MacKenzie (2000) Meta analysis of 33 experimental and quasi- experimental evaluations of education, vocation, and work programs. Meta analysis of 33 experimental and quasi- experimental evaluations of education, vocation, and work programs. Found that program participants recidivated at a lower rate than nonparticipants Found that program participants recidivated at a lower rate than nonparticipants “The generally weak methodological character of these studies, however, prevents attributing this observed effect on criminal behavior to the activities of the programs.” “The generally weak methodological character of these studies, however, prevents attributing this observed effect on criminal behavior to the activities of the programs.” Wilson, D.B., Gallagher, C.A., & MacKenzie, D.L. (2000). A Meta-Analysis of Corrections-Based Education, Vocation, and Work Programs for Adult Offenders. Journal of Research in Crime and Delinquency, 37(4).

6. Research in CE: Limitations Post-treatment outcomes Post-treatment outcomes Recidivism as primary outcome Recidivism as primary outcome Small study samples Small study samples Quasi-experimental designs Quasi-experimental designs Limited statistical controls Limited statistical controls Limited measurement of program components Limited measurement of program components See Lewis, J. (2006). Correctional education: Why it is only promising. Journal of Correctional Education, 57(4).

7. Promotion of Rigorous Research in Education

8. Policy Emphasis on Rigorous Research Focus on evidence-based policy and practice Focus on evidence-based policy and practice –Comprehensive School Reform Demonstration program in 1990s –No Child Left Behind (2002) Requires scientifically valid and readily interpretable syntheses of research on practical and replicable programs and policies Requires scientifically valid and readily interpretable syntheses of research on practical and replicable programs and policies

9. What Works Clearinghouse Established by the US Department of Education to be a central source of scientific evidence of what works in education Established by the US Department of Education to be a central source of scientific evidence of what works in education Conducts reviews that include only studies that meet standards for scientific evidence Conducts reviews that include only studies that meet standards for scientific evidence

10. What Works Clearinghouse Considerations Relevance to the review Relevance to the review –Were the intervention and outcome measures properly defined? Generality of findings Generality of findings –Was the intervention tested on relevant participants and environments? Precision of outcomes Precision of outcomes –Could accurate effect sizes be derived from the study report? Clarity of causal evidence Clarity of causal evidence –Was the intervention the cause of the change in the outcome?

11. What Works Clearinghouse Study Eligibility To be eligible for WWC review, a study must be a randomized controlled trial or an eligible quasi-experiment. – –Random assignment=“gold standard” of scientifically based research – –Comparison groups=“silver standard”

12. Rigorous Criteria Result in Few Studies for Review - Example Review of over 1,300 studies that examined the effect of teacher professional development on student achievement (Yoon et al., 2007) Review of over 1,300 studies that examined the effect of teacher professional development on student achievement (Yoon et al., 2007) Found that only nine met What Works Clearinghouse standards for rigorous evidence! Found that only nine met What Works Clearinghouse standards for rigorous evidence! Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest.

13. Promoting More Rigorous Research The Institute of Education Sciences (IES) The Institute of Education Sciences (IES) –Established in 2002 to “provide rigorous evidence on which to ground education practice and policy” –Has the goal of “identifying, developing, and validating effective education programs, practices, policies, and approaches as well as understanding the factors that influence variation in their effectiveness such as implementation.”

14. IES 2007 Grants Postsecondary Education added as new research topic Postsecondary Education added as new research topic Only three proposals funded in FY 2007 (from over 50 applications) Only three proposals funded in FY 2007 (from over 50 applications)

15. IES Priorities for Study Design Clearly defined interventions Clearly defined interventions Random Assignment – Cluster Randomized Trials Random Assignment – Cluster Randomized Trials Study design and sample size that provides adequate statistical power Study design and sample size that provides adequate statistical power Process (implementation) measures Process (implementation) measures

16. Benefits of Random Assignment Participants have equal chances of being assigned to each of the conditions Differences between conditions cannot be attributed to pre-existing differences between participants and/or selection biases Provides evidence of causality

17. Benefits of This Study Results are expected to provide valuable information about the effectiveness and impact of a widely available postsecondary academic delivery model that can increase access, persistence, and completion of postsecondary education by incarcerated youth. Results are expected to provide valuable information about the effectiveness and impact of a widely available postsecondary academic delivery model that can increase access, persistence, and completion of postsecondary education by incarcerated youth. Because access to this type of curriculum by incarcerated youth is limited, development of this model and documentation of its impact has great potential to improve postsecondary correctional education. Because access to this type of curriculum by incarcerated youth is limited, development of this model and documentation of its impact has great potential to improve postsecondary correctional education. Study design will provide evidence that meets WWC standards Study design will provide evidence that meets WWC standards

18. State of PS Academic Programming and College of the Air and the Transforming Lives Network

19. Current State of Correctional Postsecondary Academic Programming Programs on rise, recovering from loss of Pell grants Programs on rise, recovering from loss of Pell grants IYO age/funding cap legislative changes IYO age/funding cap legislative changes 60% state/73% federal offenders have earned pre-requisite HSD/GED 60% state/73% federal offenders have earned pre-requisite HSD/GED 5% nationwide have access to academic post secondary 5% nationwide have access to academic post secondary Slow migration from vocational to academic Slow migration from vocational to academic Politically unpopular Politically unpopular Lack of funding primary barrier Lack of funding primary barrier

20. College of the Air and the Transforming Lives Network Nationwide satellite broadcast owned and operated by CEA Nationwide satellite broadcast owned and operated by CEA Three 22-credit certificates culminating in A.A. Three 22-credit certificates culminating in A.A. $325/three-credit course (no out-of-state tuition) offered by Milwaukee Area Technical College $325/three-credit course (no out-of-state tuition) offered by Milwaukee Area Technical College 200 level or above for transferability 200 level or above for transferability $1195 annual site fee for unlimited TLN use $1195 annual site fee for unlimited TLN use Pioneered by Wisconsin DOC Pioneered by Wisconsin DOC

21. Overview of the IES Study

22. The College of the Air curricula delivered via the Transforming Lives Network (COA/TLN) has great potential to increase access, persistence, and completion of courses by incarcerated youth leading to postsecondary degrees. The College of the Air curricula delivered via the Transforming Lives Network (COA/TLN) has great potential to increase access, persistence, and completion of courses by incarcerated youth leading to postsecondary degrees. This study is designed to obtain impact data to determine the efficacy of this approach. This study is designed to obtain impact data to determine the efficacy of this approach.

23. Research Questions 1. To what extent does College of the Air delivered via the Transforming Lives Network (COA/TLN): a. Increase rates of participation in postsecondary and other academic programming? b. Improve participants’ academic achievement outcomes and progress toward postsecondary academic degrees? c. Improve participants’ achievement motivation and educational aspirations? d. Affect post-release employability for participants? e. Affect institutional outcomes, such as institutional climate and recidivism*? 2. To what extent do aspects of the COA/TLN curriculum and its delivery, institutional support, participant engagement, and participant characteristics affect outcomes? *Defined as reincarceration as a result of a new crime or parole violation.

24. Logic Model

25. Timeline Three-year study, beginning fall 2008 Three-year study, beginning fall 2008 Fall and spring data collection through spring 2011 Fall and spring data collection through spring 2011

26. Sample Forty-four prisons that serve a high concentration of youth offenders Forty-four prisons that serve a high concentration of youth offenders –Have the infrastructure in place to provide postsecondary academic instruction –Have not offered COA/TLN in the past –Provide IYO or other funds for PS academic programs –Willing to be randomly assigned to treatment or control condition as part of the study –Have a population that will provided a minimum of approximately 15 study participants beginning PS academic programming in fall 2008 who are: (1) between the ages of 18 and 25; (2) with a release date between 1 and 5 years; (3) in possession of a high school diploma or equivalent; and (4) whose tuition costs are paid using IYO or other grant funding.

27. Cluster Randomized Trial Random assignment of prisons within states to offer either experimental or control condition Random assignment of prisons within states to offer either experimental or control condition –If assigned to experimental condition, COA/TLN must be provided as the primary institution-sponsored PS academic curriculum for the duration of the study. –If assigned to control condition, alternative PS academic programming must be made available for the duration of the study. COA/TLN must not be provided for postsecondary. –Options for sites with strong PS academic programs already in place

28. More Detailed Information about Incentives, Data Collection Activities, Timeline, and Expectations of Participating Sites

29. Incentives COA/TLN sites COA/TLN sites –First Year: Reimbursement of up to $1,500 TLN equipment purchase costs/installation and the $1,000 annual site fee. –Years 2 and 3: Reimbursement of the $1,000 annual site fee. Control sites Control sites -$2,500 during the first year and $1,000 each subsequent year to be applied toward TLN equipment/installation at the conclusion of the study.

30. Data Collection Data collection activities conducted by research team at each site Data collection activities conducted by research team at each site Informed Consent Informed Consent CAAP CAAP College Student Survey College Student Survey Site Coordinator Survey Site Coordinator Survey Institutional Data Institutional Data College Student Follow-up Interview College Student Follow-up Interview Case Studies (interviews, focus groups, observations) Case Studies (interviews, focus groups, observations)

31. Data Collection: CAAP Critical Thinking Test Upon participant recruitment each fall (2008, 2009, 2010) and each subsequent spring through 2011 Upon participant recruitment each fall (2008, 2009, 2010) and each subsequent spring through 2011 Nationally normed, standardized, multiple-choice test that measures students’ skills in clarifying, analyzing, evaluating, and extending arguments Nationally normed, standardized, multiple-choice test that measures students’ skills in clarifying, analyzing, evaluating, and extending arguments

32. Data Collection: College Student Survey Upon participant recruitment each fall (2008, 2009, 2010) and each subsequent spring through 2011 Upon participant recruitment each fall (2008, 2009, 2010) and each subsequent spring through 2011 Measures curriculum and instructional delivery; academic engagement; achievement motivation; progress toward postsecondary degree; educational aspirations; institutional support; institutional climate; respondent characteristics Measures curriculum and instructional delivery; academic engagement; achievement motivation; progress toward postsecondary degree; educational aspirations; institutional support; institutional climate; respondent characteristics

33. Data Collection: Site Coordinator Survey Spring 2009, 2010, 2011 Spring 2009, 2010, 2011 Measures: available postsecondary programs and inmate participation; eligibility criteria and incentives for participation in postsecondary programs; curriculum and instructional delivery; institutional support; institutional climate; respondent characteristics Measures: available postsecondary programs and inmate participation; eligibility criteria and incentives for participation in postsecondary programs; curriculum and instructional delivery; institutional support; institutional climate; respondent characteristics

34. Data Collection: Institutional Data Spring 2009, 2010, 2011 Spring 2009, 2010, 2011 Information about: Information about: –Study Participants – Education background, course and credit completion, sentence length, release date, custody level, transfers, post-release contact information, recidivism –Facility - Size, inmate demographics, recidivism rates, availability of and participation in academic programs, educational funding and staffing, education policies –Curriculum - Course content, expected time commitment, course sequence and credential, delivery mode(s)

35. Data Collection: College Student Follow-Up Interview Spring 2009, 2010, 2011 Spring 2009, 2010, 2011 For released inmates For released inmates Measures of progress toward postsecondary degree, achievement motivation, educational aspirations, employment status, respondent characteristics Measures of progress toward postsecondary degree, achievement motivation, educational aspirations, employment status, respondent characteristics

36. Data Collection: Case Study Sites Random sample of 10 COA/TLN and 5 control sites Random sample of 10 COA/TLN and 5 control sites Site visits during 2008-09, 2009-10, and 2010-11 academic years Site visits during 2008-09, 2009-10, and 2010-11 academic years Interviews with site coordinators, focus groups with participating students, and observation of the learning environment and instructional activities Interviews with site coordinators, focus groups with participating students, and observation of the learning environment and instructional activities Additional information about Additional information about –clarity, value, and relevance of instruction; –factors that facilitate or impede progress; –factors affecting motivation, course completion; –postsecondary and career aspirations; –support mechanisms associated with pursuing coursework; and –state and local policies affecting participation.

37. Timeline

38. Participating Sites - Requirements Have the infrastructure in place to provide postsecondary academic instruction Have the infrastructure in place to provide postsecondary academic instruction Have not offered COA/TLN in the past Have not offered COA/TLN in the past Provide IYO or other funds for PS academic programs Provide IYO or other funds for PS academic programs Willing to be randomly assigned to treatment or control condition as part of the study Willing to be randomly assigned to treatment or control condition as part of the study Have a population that will provided a minimum of approximately 15 study participants beginning PS academic programming in fall 2008 who are: (1) between the ages of 18 and 25; (2) with a release date between 1 and 5 years; (3) in possession of a high school diploma or equivalent; and (4) whose tuition costs are paid using IYO or other grant funding. Have a population that will provided a minimum of approximately 15 study participants beginning PS academic programming in fall 2008 who are: (1) between the ages of 18 and 25; (2) with a release date between 1 and 5 years; (3) in possession of a high school diploma or equivalent; and (4) whose tuition costs are paid using IYO or other grant funding.

39. Participating Sites - Expectations Site Coordinator for each site Site Coordinator for each site –Assist with security clearance –Group students for data collection activities –Provide access to student records/ institutional databases –Provide information to track released offenders Two site visits annually Two site visits annually –Fall (August or September) –Spring (April or May) 1-2 day visit; all data collection activities conducted by research staff 1-2 day visit; all data collection activities conducted by research staff

40. Next Steps Identification of final sample ASAP Identification of final sample ASAP Initial Northstar visit by June Initial Northstar visit by June Identify Site Coordinator Identify Site Coordinator Inform administrators Inform administrators Secure MOU’s Secure MOU’s Random assignment of sites in June or earlier Random assignment of sites in June or earlier

41. Dr. Stephen Meyer, Principal Investigator RMC Research Corporation meyer@rmcdenver.com Cindy Borden & Penny Richardson, Field Investigators Northstar Correctional Education Services northstar@ekit.com Dr. Stephen Steurer, Project Director Correctional Education Association ssteurer@ceanational.org meyer@rmcdenver.com northstar@ekit.com ssteurer@ceanational.org meyer@rmcdenver.com northstar@ekit.com ssteurer@ceanational.org Contact Information