Using Demographic Data to Predict Students’ Achievement in DSPM Courses Daryl Stephens, ETSU TNADE October 27, 2005

Students take developmental studies program (DSP) courses for many reasons: Forgot material learned in high school by the time they took entrance tests Long gap between high school and matriculation Didn’t take high school seriously Didn’t consider going to college until later First generation college student (Salter & Noblett, 1994)

Most students (~90%) at ETSU take MATH 1530, Probability and Statistics, as their mathematics course to satisfy core curriculum requirements. In previous years, this course had a high failure rate.

The Problem Placement procedures for TBR schools have changed. Pre-2000: ACT 22 → take AAPP : COMPASS replaces AAPP 2002: ACT score alone determines placement for students with score < 3 years old; COMPASS alone for others

The Problem State funding is static. Placement decisions based on one test. Enrollment caps may happen in the future. Is there a way to augment the placement process by predicting a student’s chance of success or failure in developmental or core math classes?

Purpose Develop models to predict success of students in –DSPM 0800 (Elementary Algebra) –DSPM 0850 (Intermediate Algebra) –MATH 1530 (Probability and Statistics) Use multiple regression to develop the models using readily obtainable information

Importance MATH 1530 traditionally had high failure rate Developmental math students are at a greater risk of failure and dropping out Specific developmental studies program advisors done away with in budget crunch of 2003

Importance Previous similar studies on developmental students used additional instruments which cost money. Very few universities require probability and statistics for math credit for graduation. Most similar studies on core math deal with college algebra, precalculus, or math survey courses.

Assumptions Prediction is possible Self-reported data are correct Information in SIS is correct

Limitations Different demographics and course requirements from other institutions, so work only applies to ETSU Data only collected for fall; spring and summer grades are probably different

Definitions DSPM 0800: Elementary algebra –Arithmetic review, algebraic representations, linear equations in one and two variables DSPM 0850: Intermediate algebra –Exponents, polynomials, factoring, MATH 1530: Probability and statistics –“Stat mansion” and “stat cave”

Definitions COMPASS (Computerized Adaptive Placement Assessment and Support System –Prealgebra and algebra sections –Reading and writing sections –Replaced AAPP

Previous Research

Developmental Studies History Special programs at Harvard in 17 th century Preparatory department at University of Wisconsin, 1849 Morrill Acts, late 19 th century, establish land grant colleges and extend access to higher education to more people

Developmental Studies History Preparatory departments widespread in early 20 th century (350 in 1915) GI Bill of Rights brings in veterans with needs for auxiliary services 1960s-70s: Increase of women, students of color, first-generation students, students with learning disabilities; open-access community colleges established

Tennessee DSP History TBR establishes formal developmental studies program in Defining Our Future (2001): “operate more efficiently and with more limited resources” Move 0700-level courses to community colleges

Related Research Developmental Math Core Math (almost nothing on P&S) What factors are related to success in the courses? Three broad categories: –Academic –Demographic –Affective

What variables predict course success in dev. math? High school GPA, at least for traditional students Scores on the ACT or SAT — sometimes Placement tests (e.g. COMPASS, ASSET, Accuplacer, CPT, PTT, CLAST) — sometimes GED math scores — as adjunct to other placement scores

Course success in dev. math Number of high school mathematics courses taken — sometimes High school math GPA – some courses College GPA Attendance Study habits Age or length of time since last math class

Course success in dev. math (cont’d) Gender (some studies) Race Math anxiety level not related to grade! Attitude Perception of success, engagement in class Paying attention and interacting with instructor

Success in core classes Very little research on success in classes like MATH 1530, so other courses examined ACT/SAT math scores (usually) COMPASS, TASP, local placement tests High school GPA for college algebra (multiple studies) and calculus (but not precalculus)

Success in core classes (contd.) HS math GPA – mixed results HS percentile rank Number and difficulty of HS math classes taken Whether a math class taken senior year Students who didn’t take intermediate algebra scored sig. higher in college alg.

Success in core classes (contd.) Age in some cases Time since last math course Gender? Yes in 3, no in 5 studies Full-time vs. part-time (1 study) Class meeting time (1 study) No difference in resident vs. commuter, campus activity

Success in core classes (contd.) Attitude Learning styles Self-concept

Research Questions

Relationship Questions Is there a relation between course grade and … ACT (DSPP) math scores? ACT (DSPP) reading scores? COMPASS intermediate algebra scores? COMPASS reading scores? Number of college preparatory math classes taken in high school? High school GPA?

Regression Questions Can a regression equation be found to predict final course grade based on these items? From SIS: ACT composite ACT math ACT reading ACT English High school GPA Age on first day of class From Survey: Number of high school math classes taken Number of years since last HS math class

Regression Questions Can a regression equation be found to predict final course grade based on these items? From SIS: COMPASS writing COMPASS reading COMPASS prealgebra COMPASS intermediate algebra Age on first day of class From Survey: Number of high school math classes taken Number of years since last HS math class

Method Collect information about courses taken in high school and year of last high school math class from students via survey Obtain other information from SIS Use Pearson correlation for relationship questions Use stepwise multiple regression for grade prediction questions

Initial Placement ACT (DSPP) Math section – < 17 (SAT < 440): DSPM 0800 – 18 (SAT 450): DSPM 0850 – >19 (SAT > 450): college level math COMPASS –Prealgebra score < 29: DSPM 0700 –Prealgebra and algebra 20-27: DSPM 0800 –Algebra 28-49: DSPM 0850 –Algebra 50-99: college level math

Data considerations Not counted: grades of W, WF, I FN grade not counted to be consistent with some other ETSU studies Online sections, RODP sections not included

Surveys returned DSPM 0800: 149 / 304 (49%) –No night, off-campus, or ITV sections DSPM 0850: 214 / 455 (47%) –Included Kingsport, night, ITV MATH 1530: 631 / 1074 (59%)

Results

Grade Distribution Table 3 * Grades of C-, D+, and D are not allowed in developmental studies courses. **Not used in answering the research questions. Grade DSPM 0800 DSPM 0850 MATH 1530 A A B B21 98 B C C C-**35 D+ **42 D **28 F19 49 FN**10858 I**011 W**365 WF**003 Total Mean SD

ACT Math vs. Grade ACT Math DSPM 0800DSPM 0850MATH 1530 r **.481** GPTSp N

ACT Reading vs. Grade ACT Reading DSPM 0800 DSPM 0850 MATH 1530 r * GPTSp N

COMPASS Int. Alg. vs. Grade COMPASS Intermediate Algebra Score DSPM 0800 DSPM 0850 MATH 1530 r *0.133 GPTSp N524552

COMPASS Reading vs. Grade COMPASS Reading Score DSPM 0800 DSPM 0850 MATH 1530 r GPTSp N362533

High School Math Courses (Most Common) Course Prealgebra Foundations Algebra Algebra Geometry Precalculus Trig/Adv. Alg Prob. & Stat Calculus2488 N

Course Grade vs. Number of College Prep Classes Taken HSMATH (All students) DSPM 0800 DSPM 0850 MATH 1530 r **0.265** GPTSp N Mean # taken SD taken

Course Grade vs. Number of College Prep Classes Taken HSMATH (Students with ACT Scores) DSPM 0800 DSPM 0850 MATH 1530 r-0.237*0.193*0.298** GPTSp N Mean # taken SD taken

Course Grade vs. Number of College Prep Classes Taken HSMATH(Students with COMPASS scores) DSPM 0800 DSPM 0850 MATH 1530 r GPTSp N Mean # taken SD taken

HSGPA vs. Grade HS Overall GPA DSPM 0800DSPM 0850MATH 1530 r.389**.333**.445** GPTSp.000 N

Regression Using ACT DSPM 0800 (95 students) Model 1: ŷ = –.589(HSMATH) (p =.001) Model 2: ŷ = –.765(HSMATH) (HSOGPA) (p <.001)

Regression Using ACT DSPM 0850 (160 students) Model 1: ŷ =.364(ACTM) – 3.238(p <.001) Model 2: ŷ =.301(ACTM) +.662(HSOGPA) – (p <.001)

Regression Using ACT MATH 1530 (475 students) Four models (p <.001 in each case; r 2 ranging from.233 to.353): ŷ =.134(ACTM) –.521 ŷ =.092(ACTM) +.771(HSOGPA) – ŷ =.103(ACTM) +.910(HSOGPA) +.108(AGE) – ŷ =.089(ACTM) +.855(HSOGPA) +.105(AGE) +.025(ACTE) – 4.9

Regression Using COMPASS DSPM 0800 (33 students) – no equation possible DSPM 0850 (23 students with all sections) ŷ =.020 (COMPASS writing) (r 2 =.221, p =.024) DSPM 0850 (44 students with math scores) ŷ =.023 (COMPASS arithmetic) (r 2 =.190, p =.033)

Regression Using COMPASS MATH students with all COMPASS sections: no equation possible 51 students with just math sections: ŷ =.027(COMPASS arithmetic) (r 2 =.276, p <.001)

Observations, Conclusions, and Recommendations

Observations: 0800 Elementary algebra: only high school GPA showed a significant correlation with course grade; math preparation in high school showed a little bit of predictive value. Finding agrees with Long (2003) – ACT math score not significantly correlated with E.A. course grade

Observations: 0800 (cont’d) Odd finding: Why a negative correlation between HS math prep and grade in 0800?

Observations Intermediate algebra: course grade correlated with ACT math and reading, COMPASS intermediate algebra and reading, high school math preparation, high school GPA. ACT math correlation agrees with Lott (1990) HS math preparation agrees with 4 studies

Observations Probability and statistics grade was significantly correlated with ACT math and reading and HSGPA, but regression used ACT math and English, age, and HSGPA. Only COMPASS arithmetic score was in regression for students without ACT scores. Findings in line with 7 other studies of other core college math courses.

Recommendations For students with borderline placement scores, use the regression equation to fine-tune the placement. If student’s regression equation predicts a failing grade, give extra attention to that student (watch attendance, recommend tutoring, etc.), or consider placement in a lower level class if possible.

Future Research Repeat study with data from spring and summer. Historically, grades in developmental math classes are significantly higher in summer and lower in spring. (Unknown if true for 1530) Repeat treating repeaters separately, or include number of previous attempts as a variable

Future Research Is grade in 1530 affected by other college-level or developmental math experience? Characteristics of repeating students (mostly qualitative research) What if core curriculum at ETSU changes allowing other math courses to count for graduation? How would raising cutoff scores affect grades? (TBR uses 19, ACT recommends 22)

Future Research Qualitative study (suggested by a student): What kind of guidance were students given in selecting high school math classes? Why a negative correlation between HS math prep and 0800 grade?

For more information Entire dissertation is available online: Will be online from UTK in Jan. or Feb., but will stay on my faculty web page unless there’s a space crunch. This slide show is also on my faculty web page. Look for the “conference presentations” link.