1. Audit Committee May 16, 2007 Meeting Student Success Measures Project Update

2. Objectives
How reliable are the student success measures that are reported to the Board of Trustees?
How accurate is the underlying student record data or other source data?
Are colleges and universities complying with board policies related to student success, such as admissions, transfer, and graduation?
Are the information technology support systems operating as intended?
ISRS
Degree Audit Reporting System (DARS)
Course Applicability System (CAS)

3. Background Student Success Measures
Measures developed using a consultative process by the Institutional Research Directors.
Definition of Success includes Students who
Re-enroll
Transfer
Receive an academic award in an approved MnSCU program

4. Three Parts of the Project
Part I: System-wide Student Success Measures
Preliminary results being presented today.
Part II: Institutional Student Success Measures
Testing in progress
Part III: Student-level Success
Work to be completed in summer

5. Part 1 – System-wide Student Success Measures
Planning completed
Discussions with Research Unit, ITS Division and DARS/CAS staff
Tested system-wide statistical sample
System-wide data analysis in progress

6. Part 1 – System-wide Student Success Measures
Statistical sample
456 sample items
95% confidence level with 2% margin of error
540,891 student records in population (detail data that supports student success measures)

7. Reliability Testing Exceptions being Corrected by Research Unit
Part 1 – System-wide Student Success Measures - Preliminary Conclusions
Reliability Testing Exceptions being Corrected by Research Unit
Error in identifying some students with degrees.
Issue identified in how transfer students are determined.
Clearinghouse data based on FERPA directory information.
Correction in progress to use alternative methods for identifying transfer students, including using MnSCU ISRS data.

8. Part 1 – System-wide Student Success Measures - Preliminary Conclusions
Comparison of Success Rates on Full-time Undergraduate Students –
Adjusted for Exceptions
Status for Fall 1999 – 2005 Cohorts
Original Student Success Measure %
Exceptions added to Success %
Graduated
27.27%
31.02%
Transferred
18.18%
19.25%
Retained
19.79%
Overall
65.24%
69.52%
Note: The effect varies significantly on the transfer out exceptions by college or university.

9. Part 1 – System-wide Student Success Measures - Preliminary Conclusions
Dashboard Reliability - Limitations
Admission status for transfer students not reliable prior to Fall 2005.
Solution = Combine undergraduate regular and transfer
Race/Ethnicity data incomplete in earlier cohorts.
Solution = Be cautious about interpreting trends based on counts. Trends in success should be reasonable.
Combining all admission statuses is difficult to interpret.
Solution = Default on full-time undergraduates as indicator.

10. Part 1 – System-wide Student Success Measures – Assumption Questions
Unresolved questions about Student Success Measures Assumptions
Should academic standing be a criteria for defining success, particularly for transfer out students?
Should students who have not met “success” criteria but have substantially completed be considered successful?

11. Part 1 – System-wide Student Success Measures – Assumption Questions
Possible definition of Good Academic Standing:
2.0 cumulative Grade Point Average
66.67 percent cumulative completion rate (earned credits/attempted credits)

12. Part 1 – System-wide Student Success Measures – Assumption Questions
Comparison of Success Rates on Full-time Undergraduate Students -
Adjusted for Academic standing for transfer out students
Status for Fall 1999 – 2005 Cohorts
Original Student Success Measure %
Exceptions added to Success %
Poor Academic Standing Deleted from Success %
Graduated
27.27%
31.02%
Transferred
18.18%
19.25%
8.02%
Retained
19.79%
Overall
65.24%
69.52%
58.29%

13. Part 1 – System-wide Student Success Measures – Assumption Questions
Possible Substantial Completion Criteria for Students who have not met “Success” criteria.
Good Academic Standing
For Undergraduate Students – must have completed 60% of program (2 years – 36 earned credits and 4 years – 72 earned credits)
For Graduate Students - must have completed 60% of program (20 earned credits)

14. Part 1 – System-wide Student Success Measures – Assumption Questions
Comparison of Success Rates on Full-time Undergraduate Students –
Added Students with Substantial Completion
Status for Fall – 2005 Cohorts
Original Student Success Measure %
Exceptions added to Success %
Poor Academic Standing Deleted from Success %
Substantial Completion Added to Success %
Graduated
27.27%
31.02%
Transferred
18.18%
19.25%
8.02%
Retained
19.79%
Substantial Completion -
4.28%
Overall
65.24%
69.52%
58.29%
62.57%

15. Other Assumptions being reviewed
Part 1 – System-wide Student Success Measures – Analysis of other Assumptions
Other Assumptions being reviewed
Only Fall Cohorts Tracked
75% of total entrants are Fall entrants
87% of university entrants are Fall entrants
96% of FT university entrants are Fall entrants
66% of college entrants are Fall entrants
82% of FT college entrants are Fall entrants
Graduate Success
Any degree awarded tracked as a success
Certificates - 5% of graduates in sample had less than 10 credits
Only first degree tracked

16. Preliminary Results – Other Observations
Excessive ISRS access to enter or modify academic awards
Over 1,200 employees have access to enter academic awards in ISRS
Data inconsistencies/complexities
Consortium credits
Academic Suspensions

17. Next Steps
Complete Testing at Colleges and Universities on Student Success Measures and DARS
Complete Surveys at each College and University
Complete system-wide data analysis
Analyze Results
Final Report – September 2007