1. Using Demographic Data to Predict Students’ Achievement in DSPM Courses Daryl Stephens, ETSU stephen@etsu.edu TNADE October 27, 2005

2. Students take developmental studies program (DSP) courses for many reasons: Forgot material learned in high school by the time they took entrance tests Long gap between high school and matriculation Didn’t take high school seriously Didn’t consider going to college until later First generation college student (Salter & Noblett, 1994)

3. Most students (~90%) at ETSU take MATH 1530, Probability and Statistics, as their mathematics course to satisfy core curriculum requirements. In previous years, this course had a high failure rate.

4. The Problem Placement procedures for TBR schools have changed. Pre-2000: ACT 22 → take AAPP 2000-2002: COMPASS replaces AAPP 2002: ACT score alone determines placement for students with score < 3 years old; COMPASS alone for others

5. The Problem State funding is static. Placement decisions based on one test. Enrollment caps may happen in the future. Is there a way to augment the placement process by predicting a student’s chance of success or failure in developmental or core math classes?

6. Purpose Develop models to predict success of students in –DSPM 0800 (Elementary Algebra) –DSPM 0850 (Intermediate Algebra) –MATH 1530 (Probability and Statistics) Use multiple regression to develop the models using readily obtainable information

7. Importance MATH 1530 traditionally had high failure rate Developmental math students are at a greater risk of failure and dropping out Specific developmental studies program advisors done away with in budget crunch of 2003

8. Importance Previous similar studies on developmental students used additional instruments which cost money. Very few universities require probability and statistics for math credit for graduation. Most similar studies on core math deal with college algebra, precalculus, or math survey courses.

9. Assumptions Prediction is possible Self-reported data are correct Information in SIS is correct

10. Limitations Different demographics and course requirements from other institutions, so work only applies to ETSU Data only collected for fall; spring and summer grades are probably different

11. Definitions DSPM 0800: Elementary algebra –Arithmetic review, algebraic representations, linear equations in one and two variables DSPM 0850: Intermediate algebra –Exponents, polynomials, factoring, MATH 1530: Probability and statistics –“Stat mansion” and “stat cave”

12. Definitions COMPASS (Computerized Adaptive Placement Assessment and Support System –Prealgebra and algebra sections –Reading and writing sections –Replaced AAPP

13. Previous Research

14. Developmental Studies History Special programs at Harvard in 17 th century Preparatory department at University of Wisconsin, 1849 Morrill Acts, late 19 th century, establish land grant colleges and extend access to higher education to more people

15. Developmental Studies History Preparatory departments widespread in early 20 th century (350 in 1915) GI Bill of Rights brings in veterans with needs for auxiliary services 1960s-70s: Increase of women, students of color, first-generation students, students with learning disabilities; open-access community colleges established

16. Tennessee DSP History TBR establishes formal developmental studies program in 1984. Defining Our Future (2001): “operate more efficiently and with more limited resources” Move 0700-level courses to community colleges

17. Related Research Developmental Math Core Math (almost nothing on P&S) What factors are related to success in the courses? Three broad categories: –Academic –Demographic –Affective

18. What variables predict course success in dev. math? High school GPA, at least for traditional students Scores on the ACT or SAT — sometimes Placement tests (e.g. COMPASS, ASSET, Accuplacer, CPT, PTT, CLAST) — sometimes GED math scores — as adjunct to other placement scores

19. Course success in dev. math Number of high school mathematics courses taken — sometimes High school math GPA – some courses College GPA Attendance Study habits Age or length of time since last math class

20. Course success in dev. math (cont’d) Gender (some studies) Race Math anxiety level not related to grade! Attitude Perception of success, engagement in class Paying attention and interacting with instructor

21. Success in core classes Very little research on success in classes like MATH 1530, so other courses examined ACT/SAT math scores (usually) COMPASS, TASP, local placement tests High school GPA for college algebra (multiple studies) and calculus (but not precalculus)

22. Success in core classes (contd.) HS math GPA – mixed results HS percentile rank Number and difficulty of HS math classes taken Whether a math class taken senior year Students who didn’t take intermediate algebra scored sig. higher in college alg.

23. Success in core classes (contd.) Age in some cases Time since last math course Gender? Yes in 3, no in 5 studies Full-time vs. part-time (1 study) Class meeting time (1 study) No difference in resident vs. commuter, campus activity

24. Success in core classes (contd.) Attitude Learning styles Self-concept

25. Research Questions

26. Relationship Questions Is there a relation between course grade and … ACT (DSPP) math scores? ACT (DSPP) reading scores? COMPASS intermediate algebra scores? COMPASS reading scores? Number of college preparatory math classes taken in high school? High school GPA?

27. Regression Questions Can a regression equation be found to predict final course grade based on these items? From SIS: ACT composite ACT math ACT reading ACT English High school GPA Age on first day of class From Survey: Number of high school math classes taken Number of years since last HS math class

28. Regression Questions Can a regression equation be found to predict final course grade based on these items? From SIS: COMPASS writing COMPASS reading COMPASS prealgebra COMPASS intermediate algebra Age on first day of class From Survey: Number of high school math classes taken Number of years since last HS math class

29. Method Collect information about courses taken in high school and year of last high school math class from students via survey Obtain other information from SIS Use Pearson correlation for relationship questions Use stepwise multiple regression for grade prediction questions

30. Initial Placement ACT (DSPP) Math section – < 17 (SAT < 440): DSPM 0800 – 18 (SAT 450): DSPM 0850 – >19 (SAT > 450): college level math COMPASS –Prealgebra score < 29: DSPM 0700 –Prealgebra 30-99 and algebra 20-27: DSPM 0800 –Algebra 28-49: DSPM 0850 –Algebra 50-99: college level math

31. Data considerations Not counted: grades of W, WF, I FN grade not counted to be consistent with some other ETSU studies Online sections, RODP sections not included

32. Surveys returned DSPM 0800: 149 / 304 (49%) –No night, off-campus, or ITV sections DSPM 0850: 214 / 455 (47%) –Included Kingsport, night, ITV MATH 1530: 631 / 1074 (59%)

33. Results

34. Grade Distribution Table 3 * Grades of C-, D+, and D are not allowed in developmental studies courses. **Not used in answering the research questions. Grade DSPM 0800 DSPM 0850 MATH 1530 A357570 A-171848 B+141735 B21 98 B-81136 C+91334 C132589 C-**35 D+ **42 D **28 F19 49 FN**10858 I**011 W**365 WF**003 Total149214631 Mean2.7852.9912.419 SD1.29931.20751.1526

35. ACT Math vs. Grade ACT Math DSPM 0800DSPM 0850MATH 1530 r0.025.322**.481** GPTSp0.80200 N102169499

36. ACT Reading vs. Grade ACT Reading DSPM 0800 DSPM 0850 MATH 1530 r0.0470.164* GPTSp0.6420.0330 N102169497

37. COMPASS Int. Alg. vs. Grade COMPASS Intermediate Algebra Score DSPM 0800 DSPM 0850 MATH 1530 r0.1070.326*0.133 GPTSp0.4480.0290.348 N524552

38. COMPASS Reading vs. Grade COMPASS Reading Score DSPM 0800 DSPM 0850 MATH 1530 r0.2220.2060.118 GPTSp0.1940.3230.513 N362533

39. High School Math Courses (Most Common) Course080008501530 Prealgebra473767 Foundations 2 162628 Algebra 1 123181518 Algebra 2 111182556 Geometry102176544 Precalculus627211 Trig/Adv. Alg. 427133 Prob. & Stat. 4672 Calculus2488 N149214633

40. Course Grade vs. Number of College Prep Classes Taken HSMATH (All students) DSPM 0800 DSPM 0850 MATH 1530 r-0.0680.191**0.265** GPTSp0.4340.007.000 N135198564 Mean # taken2.523.113.73 SD taken1.0340.9551.115

41. Course Grade vs. Number of College Prep Classes Taken HSMATH (Students with ACT Scores) DSPM 0800 DSPM 0850 MATH 1530 r-0.237*0.193*0.298** GPTSp0.0170.012.000 N102168499 Mean # taken2.843.283.85 SD taken1.23331.21.034

42. Course Grade vs. Number of College Prep Classes Taken HSMATH(Students with COMPASS scores) DSPM 0800 DSPM 0850 MATH 1530 r-0.0610.1310.171 GPTSp0.6710.3920.226 N514552 Mean # taken1.882.532.85 SD taken1.1031.2541.036

43. HSGPA vs. Grade HS Overall GPA DSPM 0800DSPM 0850MATH 1530 r.389**.333**.445** GPTSp.000 N105176500

44. Regression Using ACT DSPM 0800 (95 students) Model 1: ŷ = –.589(HSMATH) + 4.599(p =.001) Model 2: ŷ = –.765(HSMATH) + 1.009(HSOGPA) + 2.298(p <.001)

45. Regression Using ACT DSPM 0850 (160 students) Model 1: ŷ =.364(ACTM) – 3.238(p <.001) Model 2: ŷ =.301(ACTM) +.662(HSOGPA) – 4.111 (p <.001)

46. Regression Using ACT MATH 1530 (475 students) Four models (p <.001 in each case; r 2 ranging from.233 to.353): ŷ =.134(ACTM) –.521 ŷ =.092(ACTM) +.771(HSOGPA) – 2.181 ŷ =.103(ACTM) +.910(HSOGPA) +.108(AGE) – 4.953 ŷ =.089(ACTM) +.855(HSOGPA) +.105(AGE) +.025(ACTE) – 4.9

47. Regression Using COMPASS DSPM 0800 (33 students) – no equation possible DSPM 0850 (23 students with all sections) ŷ =.020 (COMPASS writing) + 1.374 (r 2 =.221, p =.024) DSPM 0850 (44 students with math scores) ŷ =.023 (COMPASS arithmetic) + 1.730 (r 2 =.190, p =.033)

48. Regression Using COMPASS MATH 1530 22 students with all COMPASS sections: no equation possible 51 students with just math sections: ŷ =.027(COMPASS arithmetic) +.671 (r 2 =.276, p <.001)

49. Observations, Conclusions, and Recommendations

50. Observations: 0800 Elementary algebra: only high school GPA showed a significant correlation with course grade; math preparation in high school showed a little bit of predictive value. Finding agrees with Long (2003) – ACT math score not significantly correlated with E.A. course grade

51. Observations: 0800 (cont’d) Odd finding: Why a negative correlation between HS math prep and grade in 0800?

52. Observations Intermediate algebra: course grade correlated with ACT math and reading, COMPASS intermediate algebra and reading, high school math preparation, high school GPA. ACT math correlation agrees with Lott (1990) HS math preparation agrees with 4 studies

53. Observations Probability and statistics grade was significantly correlated with ACT math and reading and HSGPA, but regression used ACT math and English, age, and HSGPA. Only COMPASS arithmetic score was in regression for students without ACT scores. Findings in line with 7 other studies of other core college math courses.

54. Recommendations For students with borderline placement scores, use the regression equation to fine-tune the placement. If student’s regression equation predicts a failing grade, give extra attention to that student (watch attendance, recommend tutoring, etc.), or consider placement in a lower level class if possible.

55. Future Research Repeat study with data from spring and summer. Historically, grades in developmental math classes are significantly higher in summer and lower in spring. (Unknown if true for 1530) Repeat treating repeaters separately, or include number of previous attempts as a variable

56. Future Research Is grade in 1530 affected by other college-level or developmental math experience? Characteristics of repeating students (mostly qualitative research) What if core curriculum at ETSU changes allowing other math courses to count for graduation? How would raising cutoff scores affect grades? (TBR uses 19, ACT recommends 22)

57. Future Research Qualitative study (suggested by a student): What kind of guidance were students given in selecting high school math classes? Why a negative correlation between HS math prep and 0800 grade?

58. For more information Entire dissertation is available online: http://faculty.etsu.edu/stephen/ http://faculty.etsu.edu/stephen/ Will be online from UTK in Jan. or Feb., but will stay on my faculty web page unless there’s a space crunch. This slide show is also on my faculty web page. Look for the “conference presentations” link.