2. Measuring the Effects of Delayed or Avoided Developmental Coursework A Suggested Approach for Assessing the Effectiveness of Pre-College Courses T.M. Wright Columbia-Greene Community College

4. Context

5. 99.6% of two-year public institutions offer pre- college courses Often referred to a remedial or developmental education (distinction between the two at CGCC) First ‘remedial courses’ were offered at Harvard College in 1657 where students took remedial courses in Latin

6. Growing Phenomenon –Nationwide 36% of entering freshmen in 1985 increased to 41% by 1999

7. Source: NYSED Survey, 1998

9. Two-thirds of students requiring pre-college preparation in one subject only are deficient in math

10. Context Continued

11. National Trend to transfer developmental/remedial mission to the two-year sector either by –De jure CUNY Initiative, Florida, California, others –De facto Open access and expanded support services offered at a lower cost by two-year sector –Two-year sector share approaching 90% (2003) Costly Endeavor –Michigan Study in 2000 found that the cost to the state was $600 million (post secondary and private industry) –If the costs nationwide are comparable, then remedial education for basic skills costs about $16.6 billion annually in the United States.

12. The Debate

13. Are inadequate state high school graduation standards to blame for so many academically under prepared students? ACT Survey of Faculty (2006) How well do you think your state’s standards prepare students for college-level work in your content area? Percent reporting “Very Well” or “Well” –Post Secondary Faculty »Writing 33% »Reading 37% »Math 42% –High School Faculty »Writing 76% »Reading 72% »Math 79%

14. Do the outcomes of pre-college courses justify the costs? –Lower mandated class sizes and reduced loads for remedial & developmental faculty plus enhanced support services result in a higher cost per credit hour CGCC max class size for pre-college course=15 Full load for developmental faculty = 4 courses CGCC max class size for college-level course=22 Full load for faculty teaching college level = 5 courses –Developmental & remedial classes have higher withdrawal and failure rates

16. –Academically under prepared students have lower retention and graduation rates

18. Typical Approach to Measuring Developmental Course Outcomes

19. Problems with The Traditional Approach to Assessing Pre-College Course Outcomes Control group not representative of the treatment group –Need to examine post-developmental course outcomes for similar treatment and control groups in terms of academic preparation and social characteristics Unrealistic Expectations –Intent of pre-college courses is to increase the probability of success but not to erase a history of poor academic preparation Placement tests cannot measure motivation or other affective variables in academic learning. “ if we simply compare the performances of remedial versus non-remedial students in terms of educational outcomes, the former group will perform far worse than the latter group due mainly to pre-college differences rather than to the program itself “ (Bettinger & Long, 2005).

20. The Columbia-Greene Study Design & Sample Funded by the Institute of Community College Development

21. Ex Post Facto, Quasi Experimental design Study population includes –all first-time students enrolled for 9 or more credits who first entered CGCC between fall 2000 and fall 2005 with 15 or more accumulated credits by fall 2006 –Tested for developmental (50% pre-college, 50% college material) but not remedial (100% pre college) courses in English, math, or both –Separated into Treatment Group – those that took the required developmental course within their first two semesters and subsequently enrolled in a follow-on college-level course Control Group – those that did not take the required developmental course within their first two semesters and enrolled in a follow-on college-level course

22. Control Group further broken down into subgroups –Those that tested or waived out of the requirement Advisor may waive EN100 requirement based upon review of writing sample (WAIVE) Student tests out of the course on the first day –Those that delayed taking the required course(s) beyond their first two semesters (DELAY) but did not take a follow on course –Those that managed to avoid the requirement altogether. Took follow on course w/o taking developmental (AVOID) Did drop then add in first week Advisor error allowed student to enroll in follow-on course (student did not take or failed the required developmental course)

23. Developmental English

24. Developmental Math

25. How the need for Pre College Coursework is Determined at CGCC

26. Course Placement at CGCC Using COMPASS CGCC Uses the ACT COMPASS tests to make placement decisions for selected first-time students: –Students with a non-Regents diploma, including students from other states. – Students with a Regents diploma but whose grades are weak or inconsistent (set at < 76). – Students with a GED. – Students without a high school diploma or GED. – Students who are or have been home-schooled. – Transfer students who have not successfully completed (C or better) college-level or skill building coursework in English and math. The COMPASS tests provide an objective measure of students’ academic achievement and readiness for college and incorporates curriculum-based tests of educational development in: –English (writing) –mathematics –reading.

27. COMPASS Placement Score Probability of Success Chart for Establishing Cutoff Points 100 0

28. Group Means for COMPASS Writing Cumulative GPA

29. Group Means for COMPASS Writing Earn Rate

30. Group Means for COMPASS Math Cumulative GPA

31. Group Means for COMPASS Math Earn Rate

32. The Columbia-Greene Study Results

33. Outcomes Measures Included in the Study –Grade performance on follow-on English & math courses % A-B & F or W –One year retention –CUM GPA and Credits Earned –Earn Rate (Credits Earned/Credits Attempted) –Three-year Graduation Rate

34. Developmental English Only Cumulative GPA

35. Developmental English Only Grade in EN 101 - Composition Percent

36. Developmental English Only One Year Retention Rate 2005-06 Percent

37. Developmental Math Only Cumulative GPA

38. Developmental Math Only Grade in Follow on Math Courses Percent

39. Developmental Math Only One Year Retention Rate 2005-06 Percent

41. Developmental Students Three-Year Graduation Rate Percent

42. Conclusions Effectiveness of developmental courses is questionable as control group exhibits better academic outcomes on most indicators Non academic factors seem to play a large role (Control Group’s Savvy Student) More confident Finds creative ways around requirement –Obtains advisor waiver –Drop/Add ploy –Transfer in from another institution Need to examine other factors such as # tutoring hours, class attendance, goal commitment

43. Discussion/Questions

44. Exam Question

45. What is the difference between a dead dean and a dead skunk lying in the middle of the road?

46. Skid marks in front of the skunk!!!