Audit Committee May 16, 2007 Meeting Student Success Measures Project Update

Objectives How reliable are the student success measures that are reported to the Board of Trustees? How accurate is the underlying student record data or other source data? Are colleges and universities complying with board policies related to student success, such as admissions, transfer, and graduation? Are the information technology support systems operating as intended? ISRS Degree Audit Reporting System (DARS) Course Applicability System (CAS)

Background Student Success Measures Measures developed using a consultative process by the Institutional Research Directors. Definition of Success includes Students who Re-enroll Transfer Receive an academic award in an approved MnSCU program

Three Parts of the Project Part I: System-wide Student Success Measures Preliminary results being presented today. Part II: Institutional Student Success Measures Testing in progress Part III: Student-level Success Work to be completed in summer

Part 1 – System-wide Student Success Measures Planning completed Discussions with Research Unit, ITS Division and DARS/CAS staff Tested system-wide statistical sample System-wide data analysis in progress

Part 1 – System-wide Student Success Measures Statistical sample 456 sample items 95% confidence level with 2% margin of error 540,891 student records in population (detail data that supports student success measures)

Reliability Testing Exceptions being Corrected by Research Unit Part 1 – System-wide Student Success Measures - Preliminary Conclusions Reliability Testing Exceptions being Corrected by Research Unit Error in identifying some students with degrees. Issue identified in how transfer students are determined. Clearinghouse data based on FERPA directory information. Correction in progress to use alternative methods for identifying transfer students, including using MnSCU ISRS data.

Part 1 – System-wide Student Success Measures - Preliminary Conclusions Comparison of Success Rates on Full-time Undergraduate Students – Adjusted for Exceptions Status for Fall 1999 – 2005 Cohorts Original Student Success Measure % Exceptions added to Success % Graduated 27.27% 31.02% Transferred 18.18% 19.25% Retained 19.79% Overall 65.24% 69.52% Note: The effect varies significantly on the transfer out exceptions by college or university.

Part 1 – System-wide Student Success Measures - Preliminary Conclusions Dashboard Reliability - Limitations Admission status for transfer students not reliable prior to Fall 2005. Solution = Combine undergraduate regular and transfer Race/Ethnicity data incomplete in earlier cohorts. Solution = Be cautious about interpreting trends based on counts. Trends in success should be reasonable. Combining all admission statuses is difficult to interpret. Solution = Default on full-time undergraduates as indicator.

Part 1 – System-wide Student Success Measures – Assumption Questions Unresolved questions about Student Success Measures Assumptions Should academic standing be a criteria for defining success, particularly for transfer out students? Should students who have not met “success” criteria but have substantially completed be considered successful?

Part 1 – System-wide Student Success Measures – Assumption Questions Possible definition of Good Academic Standing: 2.0 cumulative Grade Point Average 66.67 percent cumulative completion rate (earned credits/attempted credits)

Part 1 – System-wide Student Success Measures – Assumption Questions Comparison of Success Rates on Full-time Undergraduate Students - Adjusted for Academic standing for transfer out students Status for Fall 1999 – 2005 Cohorts Original Student Success Measure % Exceptions added to Success % Poor Academic Standing Deleted from Success % Graduated 27.27% 31.02% Transferred 18.18% 19.25% 8.02% Retained 19.79% Overall 65.24% 69.52% 58.29%

Part 1 – System-wide Student Success Measures – Assumption Questions Possible Substantial Completion Criteria for Students who have not met “Success” criteria. Good Academic Standing For Undergraduate Students – must have completed 60% of program (2 years – 36 earned credits and 4 years – 72 earned credits) For Graduate Students - must have completed 60% of program (20 earned credits)

Part 1 – System-wide Student Success Measures – Assumption Questions Comparison of Success Rates on Full-time Undergraduate Students – Added Students with Substantial Completion Status for Fall 1999 – 2005 Cohorts Original Student Success Measure % Exceptions added to Success % Poor Academic Standing Deleted from Success % Substantial Completion Added to Success % Graduated 27.27% 31.02% Transferred 18.18% 19.25% 8.02% Retained 19.79% Substantial Completion - 4.28% Overall 65.24% 69.52% 58.29% 62.57%

Other Assumptions being reviewed Part 1 – System-wide Student Success Measures – Analysis of other Assumptions Other Assumptions being reviewed Only Fall Cohorts Tracked 75% of total entrants are Fall entrants 87% of university entrants are Fall entrants 96% of FT university entrants are Fall entrants 66% of college entrants are Fall entrants 82% of FT college entrants are Fall entrants Graduate Success Any degree awarded tracked as a success Certificates - 5% of graduates in sample had less than 10 credits Only first degree tracked

Preliminary Results – Other Observations Excessive ISRS access to enter or modify academic awards Over 1,200 employees have access to enter academic awards in ISRS Data inconsistencies/complexities Consortium credits Academic Suspensions

Next Steps Complete Testing at Colleges and Universities on Student Success Measures and DARS Complete Surveys at each College and University Complete system-wide data analysis Analyze Results Final Report – September 2007