Success is what counts. A Better Way to Measure Community College Performance: An Achieving the Dream Cross-State Data Initiative 2009 SHEEO/NCES Network Conference and IPEDS Workshop May 21, 2009 J. Keith Brown North Carolina Community College System

Success is what counts. Presentation Summary : Overview of Achieving the Dream and the Cross- State Data Workgroup Recommendations for an Alternative Set of Measures for Community College Performance Introduction of Intermediate Milestones and Final Measures of Student Performance Implications for Policy and Practice

Success is what counts. Achieving the Dream Overview National initiative to help more community college students succeed (earn degrees, earn certificates, or transfer) Particularly concerned about student groups that have faced the most significant barriers to success, including low-income students and students of color

Success is what counts. 82 Institutions in 15 States AR, CT, FL, HI, MA, MI, NC, NM, OH, OK, PA, SC, TX, VA, WA

Success is what counts. Achieving the Dream Values Student-centered Equity and excellence Culture of evidence, inquiry, accountability, and shared responsibility

Success is what counts. Cross-State Data Workgroup Initial States Connecticut Florida North Carolina Ohio Texas Virginia States Joining Arkansas Massachusetts New Mexico Oklahoma South Carolina Washington

Success is what counts. Data Workgroup Goals Develop a set of indicators to:  More effectively track student performance  Evaluate the effectiveness of interventions  Learn from the strengths of other community college systems

Success is what counts. Test Drive: Six States Pilot Better Ways to Measure and Compare Community College Performance

Success is what counts. Recommended Changes: Prior Enrollment Current Federal MethodATD Work Group Method First-time-in-college students onlySame as federal method Intent at time of Enrollment Current Federal MethodATD Work Group Method Only students seeking a certificate or degree Same as federal method Enrollment Status Current Federal MethodATD Work Group Method Full-time students onlyFull-time and part-time students

Success is what counts. Timeframe Current Federal MethodATD Work Group Method Three years (150% of “normal time” to completion) Six years Recommended Changes: Success Outcomes Current Federal MethodATD Work Group Method Earned degree or certificate- Earned degree or certificate - Transferred without an award - Enrolled in year 6 with at least 30 hrs

Success is what counts. Tracking transfer students within the 2-year sector Current Federal MethodATD Work Group Method Reporting is based on individual college; doesn’t track outcomes of students who transfer to another college; simply reported as transferred out Reporting is based on statewide community college system; tracks outcomes within the system (therefore across colleges) Recommended Changes:

Success is what counts. Controlling for factors associated with success Current Federal MethodATD Work Group Method Part-time students excluded from analysis; no disaggregation of results by age at initial enrollment Disaggregated results by part- time and full-time status and age at initial enrollment Recommended Changes:

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Results of Cross-State Comparison

Success is what counts. Next Steps: Developing Intermediate Benchmarks to Measure Student Progress

Success is what counts. Intermediate milestones to track students First-Year Milestones Persisted fall to spring Passed 80% or more of attempted hours Earned 24 or more hours Second-Year Milestones Persisted fall to fall Completed developmental math by year 2 Earned 48 or more hours Third-Year Milestones Passed gatekeeper English or higher by year 3 Passed gatekeeper math or higher by year 3

Success is what counts. First-Year Milestones Returned spring semester 16% increase in final success outcomes for full-time students; 28% for part-time students Earned 24 credits (full-time) or 18 credits (part-time) by the end of the first year 25% increase in final success outcomes for full-time students; 66% for part-time students Passed 80% of credits attempted 12% increase in final success outcomes for full-time students; 46% for part-time students

Success is what counts. Second-Year Milestones Returned in fall of second year 23% increase in final success outcomes for full- time students; 53% for part-time students Returned and earned 42 credits (full-time) or 24 credits (part-time) by the end of the second year 32% increase in final success outcomes for full- time students; 49% increase for part-time students

Success is what counts. Second-Year Milestones Passed developmental mathematics course by the end of the second year 84% increase in final success outcomes for full- time students; 110% increase for part-time students Passed developmental English course by the end of the third year 17% increase in final success outcomes for full- time students; 39% increase for part-time students

Success is what counts. Third-Year Milestones Passed “gatekeeper” mathematics course by the end of the third year 45% increase in final success outcomes for full- time students; 147% increase for part-time students Passed “gatekeeper” English course by the end of the third year 17% increase in final success outcomes for full- time students; 59% increase for part-time students

Success is what counts. Tracking toward final success measures Fourth- and Sixth-Year Measures Award of less than associate’s degree w/o transfer Award of associate’s degree or higher w/o transfer Award of less than associate’s degree and transferred Award of associate’s degree or higher and transferred Transferred w/o an award Still enrolled with 30 or more college hours Total success rate

Success is what counts. Next Steps…finishing data runs Run benchmarks at state and institutional levels Disaggregate and analyze performance by: o academic readiness o income o ethnicity o gender Identify and document promising interventions

Success is what counts. What does all this mean and what are the policy implications?

Success is what counts. Context of the Data Reflects the mission of colleges/state systems Examples: age distribution, award distribution Reflects differences in state priorities/policies Example: college transfer policy Not all measures pertain to all students Example: gatekeeper math course

Success is what counts.

Context of the Data Reflects the mission of colleges/state systems Examples: age distribution, award distribution Reflects differences in state priorities/policies Example: college transfer policy Not all measures pertain to all students Example: gatekeeper math course

Success is what counts. Impact of state policy on outcomes Differences in transfer patterns reflect policy differences o Encouraging transfer after earning a degree e.g. FL – 69% transfer after degree & 7% before o Encouraging transfer without a degree e.g. TX – 25% transfer without degree o Absence of strong transfer policies e.g. OH – 22% transfer after degree & 6% before o Balanced approach to transfer e.g. NC – 16% transfer after degree & 14% before

Success is what counts. Context of the Data Reflects the mission of colleges/state systems Examples: age distribution, award distribution Reflects differences in state priorities/policies Example: college transfer policy Not all measures pertain to all students Example: gatekeeper math course

Success is what counts. Implications of the Data: College Perspective Identification of at-risk students Student advising Review of policies/practices Examples:  Course taking sequence/timing  Drop/add policies

Success is what counts. Implications of the Data: State/System Perspective Policy development to improve student success Review of regulations Performance indicators Development/refinement of student database Benchmarking

Success is what counts. Implications of the Data: National Perspective More appropriate measure of student success: accountability Financial aid policy Expand the body of knowledge on successful community colleges/practices

Success is what counts. Acknowledgements The following individuals are gratefully acknowledged for their contributions to this presentation: Chris Baldwin: Jobs for the Future Pat Windham: Florida Donna Jovanovich: Virginia Corby Coperthwaite: Connecticut

Success is what counts. Achieving the Dream Success is what counts. ww.achievingthedream.org