From a Culture of Guesstimates to a Culture of Evidence

About Speakers Blaine Bennett Blaine Bennett Dick Whipple Dick Whipple Hector Gonzales Hector Gonzales Michael Taft Michael Taft Michael Nguyen Michael Nguyen

About SWTJC Large service area Large service area 11 Counties (16,812 sq. miles) 11 Counties (16,812 sq. miles) HSI - 82% HSI - 82% 50%+ need remediation 50%+ need remediation 42% financial aid 42% financial aid AtD 2005 – Round 1 AtD 2005 – Round 1

About AtD AtD Goals AtD Goals Increase percentage who: Increase percentage who: Succeed in dev ed classes Succeed in dev ed classes Enroll / succeed in gatekeeper classes Enroll / succeed in gatekeeper classes Return Return Graduate Graduate

Fall 2005 – Data difficult to acquire. Need to know what’s going on (AtD data driven) with students. Fall 2005 – Data difficult to acquire. Need to know what’s going on (AtD data driven) with students. Began development of a student data warehouse - Estudias Began development of a student data warehouse - Estudias Wanted to study the dev-ed to gateway process more closely Wanted to study the dev-ed to gateway process more closely (e.g. How long does it take a student to go through dev-ed and complete the gateway course?) (e.g. How long does it take a student to go through dev-ed and complete the gateway course?) In The Beginning…

Data and Questions????? Is data readily accessible for real time planning & decision making? Is data readily accessible for real time planning & decision making? Can advisors readily access a comprehensive look at a student? Can advisors readily access a comprehensive look at a student? Can cohorts of students be easily identified ? Can cohorts of students be easily identified ? Can gaps in student progress be readily identified? Can gaps in student progress be readily identified?

Example: Completion Rates? Hard to get basic info from Datatel Hard to get basic info from Datatel Solution: Estudias Data Warehouse Solution: Estudias Data Warehouse

Developmental placement? Developmental placement? Success rates? Duration? Success rates? Duration? By course, faculty, cohort, high school, placement score, modes of instruction By course, faculty, cohort, high school, placement score, modes of instruction Which are the gateway courses? Which are the gateway courses? Post-gateway success? Post-gateway success? Probability, time-to-degree/transfer Probability, time-to-degree/transfer Interventions? Interventions? What kind? What audience? Do they work? What kind? What audience? Do they work? Access to Data Leads to More Questions

Levels were tied with course enrollment to gauge a student’s capacity for success in a course. Levels were tied with course enrollment to gauge a student’s capacity for success in a course. Prior to Estudias, a student/advisor would see something like this in the course description: Prior to Estudias, a student/advisor would see something like this in the course description: Student must be at math level 1 to enroll; i.e., Accuplacer 30 or greater. Student must be at math level 1 to enroll; i.e., Accuplacer 30 or greater. Before Estudias

Developmental prerequisites at this time were best guesstimates! Developmental prerequisites at this time were best guesstimates! Before Estudias

From guesstimates to evidence Placement ranges were not based on refined data analysis Placement ranges were not based on refined data analysis Cutoff scores hard to remember, many tests, course completions Cutoff scores hard to remember, many tests, course completions Data were not fine-grained / diagnostic Data were not fine-grained / diagnostic

Looking at class and test history, impossible to see where student is in dev ed sequence Too Much Data, Not Enough Information

Solution Logistical regression on scores Logistical regression on scores Incorporate grades Incorporate grades Higher grades = higher scores Higher grades = higher scores Bottom Line  Test History Class History

Example Susan Smith Accuplacer 78 (high level 2) Accuplacer 78 (high level 2) Aaron Abel Accuplacer 63 (barely level 2) Accuplacer 63 (barely level 2) Although these students are both in Level 2, they are very different students

Solution: new level based on better data The score accurately reflects the fact that each of these students has completed one developmental class and more precisely illustrates the differences between them Old Level New Score Aaron Abel Accuplacer 63 (barely level 2) Passed Level 2 with a D Passed Level 2 with a DLevel2 Low Level 200 Susan Smith Accuplacer 78 (almost level 3) Passed Level 2 with a A Passed Level 2 with a ALevel2 High Level 275

Data and Questions????? Can advisors readily access a comprehensive look at a student? Can advisors readily access a comprehensive look at a student? Can cohorts of students be easily identified ? Can cohorts of students be easily identified ? Can gaps in student progress be readily identified? Can gaps in student progress be readily identified? Is data readily accessible for real time planning & decision making? Is data readily accessible for real time planning & decision making?

Example: Bubble Student

Find Bubble Students Credits > 50 GPA > 2.5 Math: Level 2 (high)

Compare Cohorts Started 2005 FA Performance of: Cohort 1: Placed into Level 2 Cohort 2: Placed into Gateway How many completed Gateway over time?

Examine Cohort in Detail Cohort 1: 2005FA, placed into Level 2. How many: Enrolled in Level 2 Completed Level 2 Enrolled Gateway Completed Gateway

Examine Cohort in Detail How many: Enrolled in Level 2, Completed Level 2, Enrolled Gateway, Completed Gateway

Questions Blaine Bennett Blaine Bennett Dick Whipple Dick Whipple Hector Gonzales Hector Gonzales Michael Taft Michael Taft Michael Nguyen Michael Nguyen Slides at: