Mason Burley ERDC SLDS Conference Olympia, WA May 21, 2014 Big Data, Big Obstacles Assessing the Impact of Statewide College Financial Assistance Programs WASHINGTON STATE INSTITUTE FOR PUBLIC POLICY

Debrief Strategy Obstacles Mission Washington State Institute for Public Policy May 21, 2014 Slide 2 of 28 Started in 1969 Available to WA undergrads Available to public/private, two/four students Award amounts linked to tuition Need-based (median family income) Means tested (prorated award) Subject: State Need Grant (SNG) program

Debrief Strategy Obstacles Mission Washington State Institute for Public Policy Slide 3 of 28 Problem: Resident undergraduate tuition growth Sector Tuition and fees (2000–01) Tuition and fees (2011–12) Public research$3,984$9,993 Public regional$3,092$6,969 Private four-year$15,597$31,618 Community and technical college $1,756$3,542 May 21, 2014

Debrief Strategy Obstacles Mission Washington State Institute for Public Policy Slide 4 of 28 State Need Grant—Total expenditures ( ) May 21, 2014

Debrief Strategy Obstacles Mission Washington State Institute for Public Policy Slide 5 of 28 State Need Grant eligible students May 21, 2014

Debrief Strategy Obstacles Mission Washington State Institute for Public Policy Slide 6 of 28 Mission: Are SNG dollars effective? 2012 HB 2127 “Determine to what extent this [SNG] program has increased access and degree attainment for low-income students and to determine whether the funding for the state need grant has been utilized in the most efficient way possible to maximize the enrollment and degree attainment of low-income students” May 21, 2014

Debrief Strategy Obstacles Mission Washington State Institute for Public Policy Slide 7 of 28 Mission: Are SNG dollars effective? The demographics of recipients, including, but not limited to, gender, race, and income The effect on enrollment rates of low-income students at the different institutions of higher education and whether these students attend full-time or part-time The effect on persistence, performance, degree completion, and time-to-degree completion An inventory of the types of degrees, by field of study The interplay of the State Need Grant program with other forms of financial aid May 21, 2014

Debrief Strategy Obstacles Mission Washington State Institute for Public Policy Slide 8 of 28 Operatives: Washington State Institute for Public Policy Non-partisan research at legislative/board direction Board of Directors Senator Jeanne Kohl-Welles, Co-Chair Representative Maureen Walsh, Co-Chair Senator Joe FainRepresentative Ruth Kagi Senator Karen FraserRepresentative Larry Springer Senator Mark SchoeslerRepresentative Hans Zeiger Richard Rodger, Senate Staff Dir.Jill Reinmuth, House Staff Dir. David Schumacher, OFMGovernor Appointee Sandra Archibald, UWRodolfo Arévalo, EWU Thomas “Les” Purce, TESCRobert Rosenman, WSU May 21, 2014

Results Strategy Mission Obstacles Washington State Institute for Public Policy Slide 9 of 28 Research obstacles: Different sources of aid Aid categoryResearchRegional Private four-year Community/ technical colleges Total Federal grants—Pell23%25%14%33%28% State Need Grant28%25%18%15%20% Institutional and other grants16%8%29%3%9% Work study2% Federal loans18%26%21%15%18% Need unmet14% 16%31%23% Need unmet average (dollars) $ 3,349 $ 2,914 $ 6,384 $ 4,921 $ 4,676 Average need per student $ 22,852 $ 19,982 $ 38,910 $ 15,655 $ 19,877 Calculated need equals cost of attendance minus expected family contribution Note that not all students receive award/loan in each category Private loans not included May 21, 2014

Results Strategy Mission Obstacles Washington State Institute for Public Policy Slide 10 of 28 Research obstacle: Differences among students Student background and profile 64% – Community and technical college students 44% – Independent, over age 25 (37% dependent under age 25) 84% – Income under half of state median family income 16% – Independent and married Enrollment level 58% – Enrolled full-time, full year 18% – Enrolled full-time, part year 33% – Summer enrollment May 21, 2014

Results Strategy Mission Obstacles Washington State Institute for Public Policy Slide 11 of 28 Research obstacle: Comparison group …may not be comparable Higher family incomes Late filers More likely to enroll part-time Unserved SNG students… …don’t stay “unserved” 40% receive SNG in following year …have long term outcomes that are unknown first year unserved population grew May 21, 2014

Results Strategy Mission Obstacles Washington State Institute for Public Policy Slide 12 of 28 Data challenge: Timing Persistence: on-going enrollment Academic year completion (fall to spring) Fall-to-fall continuation Academic progress Accumulation of credits for on-track graduation Graduation Degree or certificate completion within 6 years (3 years for associates) Electronic databases available from 2005 (fall) forward Final report due January 2014 Potential outcomes May 21, 2014

Results Strategy Mission Obstacles Washington State Institute for Public Policy Slide 13 of 28 Data challenge: Linking data Unit Record Report (WSAC) CTC enrollment (SBCTC) Public four-year (PCHEES-COP) Private four-year (12/15 colleges) Private career (6/11 schools) Education Research and Data Center (ERDC) WSIPP SNG research data Linked, de-identified May 21, 2014

Results Strategy Mission Obstacles Washington State Institute for Public Policy Slide 14 of 28 Data challenge: Standardized data Comparability of public/private college data Cooperation Definitions Institutional aid IPEDS measures Institutional crossover First-time enrollment Academic progress Credit accumulation, satisfactory academic progress May 21, 2014

Results Mission Obstacles Strategy Washington State Institute for Public Policy Slide 15 of 28 Difference in SNG amount, but similar profile/aid Permits reliable inference on SNG effect Approach: Take advantage of “discontinuities” in SNG design Median family income range Income limits (family of four) Percent of State Need Grant award 0-50% $41, % 51-65% $53,000 75% 66-70% $57,500 50% 71%+ineligible May 21, 2014

Results Mission Obstacles Strategy Washington State Institute for Public Policy Slide 16 of 28 Average State Need Grant and Pell by MFI If SNG has an effect, we would expect to see it here May 21, 2014

Results Mission Obstacles Strategy Washington State Institute for Public Policy Slide 17 of 28 Financial aid as a percentage of COA by MFI range and sector: May 21, 2014

Results Mission Obstacles Strategy Washington State Institute for Public Policy Slide 18 of 28 Know thy data…. Acronym soup COA, EFC, SAP, Need, CIP…. Sector-specific definitions Student classification Outcome milestones Verification and transparency May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 19 of 28 Effects* of a 25% reduction SNG on outcomes May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 20 of 28 4% change in expected fall-to-fall enrollment for CTC students 1,791 students (of total 47,134 enrolled) may change enrollment decision based on $600 change in SNG amount Fall-to-fall enrollment A 25% change in the SNG award is associated with…. 2% change in expected fall-to-fall enrollment for public baccalaureate students 378 students (of total 17,171 enrolled) may change enrollment decision based on $2,000 change in SNG amount May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 21 of 28 5% change in expected degree completions for CTC students Total degrees earned may change for 436 entering students (of 10,895 first-time students) with a $600 change in SNG amount Degree completion A 25% change in the SNG award is associated with…. 8% change in expected degree completions for public baccalaureate students Total degrees earned may change for 439 entering students (of 5,289 first-time students) with a $2,000 change in SNG amount May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 22 of 28 Finding 1: Effect of need-based grants on subsequent fall enrollment (elasticity) Weighted average = May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 23 of 28 Finding 2: Longitudinal (cross sector) outcomes CTC Earned awardNo award Type of student Transfer to four- year No transfer Transfer to four- year Prepared transfer ready Still enrolled Not enrolled Total Academic 329 (15%) 313 (14%) 166 (7%) 64 (3%) 374 (17%) 998 (44%) 2,250 Workforce 71 (4%) 524 (29%) 46 (3%) 408 (22%) 127 (7%) 651 (36%) 1,826 May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 24 of 28 Finding 2: Longitudinal (cross sector) outcomes Baccalaureate Sector Completion at starting institution Complete at different institution Still enrolled Not enrolled Total Four- year college Two- year college Public research 765 (64%)23 (1%)33 (2%)69 (5%) 289 (24%) 1,179 Public regional 445 (43%)48 (4%)53 (5%)92 (8%) 394 (38%) 1,032 Private four- year 239 (56%)20 (4%)11 (2%)23 (5%) 127 (30%) 420 May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 25 of 28 Finding 3: Variation in “need” Pell Grant receipt among SNG students by income level ( ) MFI rangeReceived PellPell ineligible Total students with SNG 0-50%61,255 (99%)692 (1%)61, %4,440 (77%)1,339 (23%)5, %2,459 (65%)1,345 (35%)3,804 Total68,154 (95%)3,376 (5%)71,530 May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 26 of 28 Alternative state awarding formulas State Recipients ( ) Expenditures (millions) Model WA70,376$223Shared responsibility (MFI) MN103,544$168Shared responsibility (Pell) NC (UNC)61,952$133Shared responsibility (Pell, tax) OR43,111$77Shared responsibility (Pell, tax) PA171,702$413Percentage of need Shared responsibility model award amount=COA-(student+family+govt Pell) Percentage of need declines as EFC increases May 21, 2014

Mission Strategy Obstacles Results Washington State Institute for Public Policy Slide 27 of 28 Finding 3: Variation in “need” Hypothetical state examples—students served and award amounts State formula Estimated students served Estimated students unserved Estimated average grant Maximum grant WA72,060*28,450*$3,460$10,868 MN81,31915,549$3,143$10,450 NC73,474-$2,514$3,800 OR96,653-$1,382$1,950# PA98,257-$1,883$3,700 May 21, 2014

LESSONS LEARNED 28 of 28 Statewide programs do not always include statewide information Within/between differences An integrated database can be used to test eligibility and awarding formulas Right team and structure necessary for program evaluation – THANKS to ERDC, WSAC, and institutional partners May 21, 2014

THANK YOU Questions ? Mason Burley