1. Student Success and  Institutional Metrics:  Recent Clearinghouse Research on  Student Completions
AACRAO SEM Conference
November 11, 2013
Plan:
A quick “about NSC” – because it’s important to know who we are and why we have these data.
About measures – institutional and student
Some examples from recent RC pubs
More on why this matters and how we can go further.
Douglas T Shapiro, PhD
Executive Research Director
National Student Clearinghouse Research Center

2. About the National Student Clearinghouse Research Center
NSC: Exchanging student enrollment and degree data to simplify institutions’ compliance and verification functions
Since 1993
Non-profit 501(c)(6)
NSC Research Center: Tracking Student Educational Pathways to benefit the education community
Student-level outcomes for institutions, K-12 schools, states, etc.
Published reports and benchmarking for education policy and practice
Since 2010
Non-profit 501(c)(3)

3. About the National Student Clearinghouse Research Center
A unique view of higher education as a system
> 120 Million unique students tracked
In 2013:
95% of all enrollments at Title IV degree-granting institutions
99% at 2 & 4-year public institutions
93% at private non-profits
67% at for-profits
Institutions participate voluntarily and at no cost to share data; low or no cost to track students
> 120 M unique students
> 2 B enrollment and degree records
Enrollment records updated monthly.
NO: course, credit or grade info.
Reducing institutional burden is our basic business model – in our DNA.

4. Student Success and Institutional Metrics
Institutional success and student success are different things
Institutions and students have fundamentally different views of education
This has some important effects:
We mislabel a lot of student success as institutional failure
We try to shoehorn student pathways into institutional metrics
We don’t know what to do with shifting student goals
We can’t craft policies and practices that support students
We don’t know how successful higher education is, as a system
Measuring Student and Institutional Success…
We hear a lot about institutional metrics these days. Obama wants to create a new set of ratings based on institution-level data.
I’m here to talk about some metrics of student success. But we mean that in different ways, so I want to spend a minute talking about the differences.
The problem is that institutional success and student success are different things, even when we’re talking about student outcomes. And that’s because institutions and students increasingly have fundamentally different views of education:
Institutions tend to view education as something that happens at a time and a place (namely, now & here). They tend to see students through a very narrow window: during this academic term, on this campus. Or during 150% of program time, in this program. The institution is the home, where you are meant to start and finish and everything in between. They don’t see what happened before, or after.
Students, and I’m speaking based on their observable behaviors, increasingly view education as a pathway, as a career, as something that happens in many institutions at many times, and often in different institutions at the same time. They’re stopping in and out for employment, or family, or life, taking courses online, or across town, or in another state. The institution is more like a stepping stone than a home, a place where you stay for a time to make progress towards your goals, or to adjust your goals, and then continue on the path.
So, the kind of measures that are important to institutional success, and the various accountability requirements that are imposed on them, are typically those that pertain to students’ successful attainment of goals during that narrow window: access, cost, retention and graduation rates.
The kind of measures that that are important to student success are those that inform us about the pathways: like persistence and completion rates anywhere along the path, not just within a single window.
Our focus on the institutional measures has some important effects:
1. We mislabel a lot of student success as institutional failure. For example, when a student who completes her degree at the second or third institution in her path, looks like a dropout to the first.
2. We expend a lot of effort trying to shoehorn student pathways into institutional metrics, not to increase learning or value to the student, but just because that’s what we measure. For example, when a college tries to find former students who’ve already moved on, just so they can retroactively award “reverse transfer” degrees.
3. We don’t know what to do with shifting student goals. How does a college measure its contribution to a student’s education when the result is a new pathway leading elsewhere?
4. We – institutions, policy makers, funders -- don’t really know how to craft policies and practices that support students along the pathways as they are, rather than as tradition says they should be, because we are stuck in the window view.
5. We don’t really know how successful higher education is, as a system, at helping students attain their goals. For example, when President Obama last week said that only 58% of students who started college at a 4-year institution complete a bachelor’s degree within 6 years. We all know that’s not accurate, but by how much?
So, I want to start diving into some data to show you what I mean, what the reality of Student Educational Pathways looks like today.
These are all examples from recent published reports from the RC. And most of the information isn’t new, in the sense that NCES has done small-sample studies for years, like BPS, B&B, NPSAS, showing much the same trends. Those are very reliable, but they’re infrequent, costly, and unable to offer detailed institution-level and peer benchmarking.
So the following shows some of what we’ve been doing, that “census”-based, timely, cost-effective, and available at the institution level…

5. Up to one quarter of fall-to-fall persistence takes place at a different institution
Student persistence, Institutional attrition.
All starting students, from Signature Report #1

6. Transferring and Moving
Transfer & Mobility: One-Third of Students Enroll in a Different Institution within First 5 Years or a Degree (whichever comes first)
Prevalence of Transfer and Mobility Among All Students in Entry Cohort, Fall 2006 N %
Transferring and Moving
923,196
33.1%
Staying put
1,869,765
67.0%
Total
2,792,961
100.0%
This is a big number. 1 in 3 students enrolls in more than one institution before earning their first degree.
Same rate, regardless of FT or PT, Public or Private, 2-yr or 4yr starting instn. (exception is the for-profits)

7. That’s Half of Each Institution’s Students (on average)
Institution A
Institution B
Unique Students
Institution Students
Here’s where you can really appreciate the unique view of higher ed as a system that NSC data provides:
Imagine there’s only 3 students in the world.
2 of them enroll at your institution, and the OTHER 2 enroll at the institution across town…
The mobile student (1/3) is the same person, but each institution counts him separately as ½ of “their” students.
Institution A
Institution B

8. Focus on College Completion Rates
College completion is a key college success outcome.
Yet, institutions and policymakers have limited knowledge of the rates of completion for students who follow non-traditional postsecondary pathways.
Conventional graduation rate calculations are institution based and only count students who finish at the starting institution.
Between this slide and next slide the presenter explains what IPEDS is and whom they count (only full-time) and also mentions what this report does differently. Emphasize that no number can be directly compared to that in IPEDS and that was intentional (this is a comprehensive look with more nuanced measures)
9/22/2018

9. Study Cohort and Outcome Definitions
Started with entering cohort of fall 2006
1.9 million unique students in all types of institutions (two-year and four-year, public and private)
Counted all completions (any degree or certificate) within 6 years (through May, 2012), regardless of the student pathway
Counted all students:
Full-time
Part-time
Mixed enrollment (enrolled both full time and part time)
Results reported for traditional-age students (age 24 or younger) and adult learners (age over 24)
9/22/2018

10. National Completion Rate 54% in 6 years
12% of the first-time students graduated at a different institution from where they started
Of 1.9 million students starting in fall 2006, at all institution types.
The 12% elsewhere is fairly consistent across all institution types (2 yr/4yr, pulbic/private. BUT lower at for-profits).
So, the first answer to “how far off was Obama’s number?” is, add 12%...
9/22/2018

11. 6-year Student Outcomes by Starting Institution Type
Completing College Report, November 2012
9/22/2018

12. Gains in Completions from Graduating Elsewhere
Draw attention to All Students mentioning that part-time and mixed enrollment students are included here
9/22/2018

13. That’s more than one fifth of all student success, being counted as institutional failure.
9/22/2018

14. Fewer than Half of Students Enroll Exclusively Full Time
Fewer than Half of Students Enroll Exclusively Full Time. Most have mix of full and part time.
Conventional indicators of student type by looking only at first term status? Bah!
Overall: 51% Mixed, 7% PT, 42% FT
9/22/2018

15. Completion Rate by Enrollment Intensity
More about how enrollment intensity was defined in this study. 67% of mixed enrollment group started as FT (n=963,672*.67=645,660). As% of all who started FT it is 45%. In traditional graduation measures this 45% will be considered as FT when they didn’t keep their FT status. This is also the group that had highest gain in completion from graduation elsewhere.
9/22/2018

16. Completion Rate by Age at First Entry
9/22/2018

17. Six-Year Progress by Age and Enrollment Intensity
Completion risks for traditional-age part-timers though again it should be emphasized this study period wasn’t enough for part-timers at four-year institutions.
9/22/2018

18. Students who started at 4-year Public Institutions
WOW! 80% completion rate for Exclusively FT traditional age students. 82% if you include 2-yr degrees.
Look at Adults: HIGHER completion rates than traditional age among PT and Mixed!
9/22/2018

19. Students who started at 2-year Public Institutions
Enrollment status makes a big difference in outcomes. Notice the gap in completion rates between FT and everybody else among younger students. Whereas for older students, the gaps are much smaller – PT/Mixed are much more par for the course.
ALSO, FT Adults are MORE likely to complete at starting Instn than traditional aged. But less likely to complete overall because fewer transfer to 4yr.
How many students know the odds of success based on their own plans? Work pattern?
9/22/2018

20. 6-Year Bachelors Completions for Fall 2006 Two-Year Starters (N=631,171)
Nearly two thirds of those who completed a 4-yr degree did so without first obtaining a two-year degree.
Community colleges do not receive any “credit” for the success of these students under traditional graduation rate measures.
9/22/2018

21. Benchmarking and Peer Analysis
State level completion rates published in February 2013
Institution level rates will be available by request in 2014
Peer group rates available by request
Increasingly popular pathway – students looking for more economical alternatives.
9/22/2018

22. Focus on Community College Transfer
Only 15% of students who start at a 2-year institution complete at a 4- year within six years.
This report looks at the completion rates for a transfer cohort.
What percent of those who transfer complete a 4-yr degree?
What enrollment behaviors correspond to success?
Increasingly popular pathway – students looking for more economical alternatives.
9/22/2018

23. Private for-profits are over-represented here, by a factor of about 2
Private for-profits are over-represented here, by a factor of about 2.5, relative to first-time student distribution at 4-yr instns. (67%, 29% and 4%)

24. Notice fewer FT and more PT and Mixed than the starting cohorts

25. Outcomes of Students who Transferred from 2-yr to 4-yr Institutions in 2005/06, 6 years after transfer
26.6%
4.0%
7.8%
61.6%
N=320,911

26. Look how much better the PT do, relative to FT – Much less difference here than in 4yr starters.

27. 6-Year Outcomes by Associate’s Degree Status
9/22/2018

28. 6-year Outcomes by Type of Destination Institution
Higher completion at publics – reverse of what happens for first-time students.
How many 2-year institutions are able to advise their students about the relative success of these pathways?
9/22/2018

29. Outcomes by Institution Type and Enrollment Intensity
Oddly, Mixed: Publics do better with FT and Mixed Enrollments. Privates better with PT.

30. 6-year Outcomes by Timing of Transfer, Since Last 2-year Enrollment
Stopping out is always a bad idea…
Timing of transfer, from last 2-yr enrollment
9/22/2018

31. 6-year Outcomes by Number of Terms Enrolled Before Transfer
Not sure what to make of this one… ?
Students who transfer quickly are often thought to be better prepared…?
Number of Terms enrolled before transfer
9/22/2018

32. Focus on Reverse Transfer
20% of students who start at a 4- year institution will later enroll at a 2-year within 6 years.
How much of this is transfer vs. swirl?
What are the outcomes?
Increasingly popular pathway – students looking for more economical alternatives.
9/22/2018

33. 4-yr to 2-yr Transfer and Mobility: Out of 1
4-yr to 2-yr Transfer and Mobility: Out of 1.2M students who started at 4-yr institutions in fall 2005
Subsequent enrollment at 2-year institutions
(within 6 years or first degree) N %
Regular Terms
178,846
14.4%
Summer Terms
67,231
5.4%
Total
246,078
19.8%
(different cohort, tracking period and cohort definition than previous slide) What’s important is the proportion of regular terms vs. summer
Tracked each student for up to 6 years or first degree
Identified reverse transfer or mobility:
Any enrollment in a 2-yr institution prior to first degree completion

34. These are very different pathways
28%
Rebooters: Change of goals -- either stay in 2-yr or switched to a different 4-yr
Rechargers: regain academic footing, then return to origin
Superchargers: Summer courses to accelerate progress to degree… Higher graduation rates upon return than those who never left.
17%

35. N=1.2M (83,935 who left and returned; 962,481 who never left)
Superchargers – high performers use CC courses to accelerate progress to 4-yr degree
Rechargers – lower performers use CC semesters to regain academic footing
Rebooters – looking for a better fit, use CC semesters to re-assess or recalibrate before finding a new 4-yr institution

36. Key Numbers
Mobility
1/3 of all students transfer or enroll ensewhere. 1/2 of all enrollees.
Persistence
15% of persistence among 4-yr starters occurs somewhere else % at 2-yr institutions.
Completion
12% of all starters complete somewhere else.
Success
22% of all completions are not at the starting institution. 34% among 2-yr starters.

37. More Key Numbers Transfer Completion
15% of 2-year starters complete a 4-year degree within 6 years.
60% of 2 to 4 transfers complete degree within 6 years of transfer
Transfer Behaviors
Higher completion with Associate’s degree.
Higher completion at Public 4-years
Higher completion with 4 to 6 terms at 2-year institution
Higher completion with immediate transfer

38. Better Institutional Measures of Student Success
Instead of misclassifying student success as institutional failure, measure institutional contributions to student success anywhere along the path.
Credit institutions for serving the needs of students who transfer in, transfer out, or enroll part time.
Provide current and prospective students with complete information about available pathways to success.
Provide policy-makers with realistic view of how the higher education system serves students.
If you’re a community college providing lots of summer calculus classes to those “superchargers” from the local university, helping them to accelerate their path, you should be able to measure that and compare it to peers.
You should be able to advise a student on the odds of success if he transfers to the state flagship with or without your associates’ degree, given his GPA and major.
So they don’t have to focus all their attention on your traditional-pathway students only.
Students, institutions and policymakers need information about all pathways to student success, not just the traditional.

39. Improving Institutional Practice by Balancing the Needs of Consumer Information and Accountability
Institutional success and student success are different, but not incompatible.
Equip institutions with information to devise and implement policies that align them.
Student-level outcomes, like persistence, transfer and completion anywhere, linkable to native academic and demographic data:
What institutions do students arrive from? Leave for?
How do they compare to trends for similar institutions?
How do their paths and outcomes compare to similar students elsewhere?

40. NSC Research Center Provides
Student-level tracking through cohort queries
Institution-level and peer-level comparisons to published national, state and sector benchmarks
Institutions control their own results
Work with associations and RACs to develop, and to maintain low institutional burden
e.g., some current workforce initiatives (WICHE 4-states, and Census/LED), but there are still challenges.

41. Research Center Snapshots and Reports: http://research
Doug Shapiro: