Multiple Measure Math Placement: An evolving-revolving credit approach Angela Garcia garciaangela@cochise.edu Math Department Chair, Cochise College Tammy Brewer brewert@cochise.edu Director of Testing Services, Cochise College “How the Math Department and Testing Center are working cooperatively to start using multiple measures for placing students into mathematics classes”

Sometimes the most brilliant and intelligent students do not shine in standardized tests because they do not have standardized minds. - Diane Ravitch Network for Public Education Co-founder & President Diane Ravitch is a Research Professor of Education at New York University and a historian of education. She is the Founder and President of the Network for Public Education (NPE). Diane Ravitch’s Blog is dianeravitch.net and has received more than 30 million hits. From 1991 to 1993, she was Assistant Secretary of Education and Counselor to Secretary of Education Lamar Alexander in the administration of President George H.W. Bush. She was responsible for the Office of Educational Research and Improvement in the U.S. Department of Education. As Assistant Secretary, she led the federal effort to promote the creation of voluntary state and national academic standards. From 1997 to 2004, she was a member of the National Assessment Governing Board, which oversees the National Assessment of Educational Progress, the federal testing program. She was appointed by the Clinton administration’s Secretary of Education Richard Riley in 1997 and reappointed by him in 2001. From 1995 until 2005, she held the Brown Chair in Education Studies at the Brookings Institution and edited Brookings Papers on Education Policy. Before entering government service, she was Adjunct Professor of History and Education at Teachers College, Columbia University.

Multiple Measure Math Placement: An evolving-revolving credit approach Background & Definitions Cochise College Plan to implement Multiple Measure Math placement Obstacles we encountered (Re)working Multiple Measure into the fine print Discussion: the future of this valuable commodity

Background and Definitions

Background and Definitions What are multiple measures Why are multiple measures important?

Background and Definitions What are multiple measures

Background and Definitions Why are multiple measures important? What we think What we’ve learned What we want "Shh. Shh. Shh. Shh. Do you hear that? It’s the winds of change." -Randall, Monsters INC.

accessibility Save time Trends We just know they are Completion $$ Money $$ Convenience I.e., why should we replace single measure placement

Background and Definitions Why are multiple measures important? Single subject exams limitations Overplacing Underplacing Studies suggest other indicators, e.g, high school GPA, may be better predictors of success Community College Research Center (CCRC) One CCRC study of a statewide community college system found that the ACCUPLACER severely misplaces 33 percent of entering community college students. In other words, based on their ACCUPLACER scores, one third of entering students were either "overplaced" in college-level courses and failed or "underplaced" in remedial courses when they could have gotten a B or better in a college-level course. Using students' high school GPA instead of placement testing to make placement decisions was predicted to cut severe placement error rates in half (to 17 percent) (Belfield & Crosta, 2012). (emphasis added) https://ccrc.tc.columbia.edu/Community-College-FAQs.html Community College Research Center (CCRC) One CCRC study of a statewide community college system found that the ACCUPLACER severely misplaces 33 percent of entering community college students. In other words, based on their ACCUPLACER scores, one third of entering students were either "overplaced" in college-level courses and failed or "underplaced" in remedial courses when they could have gotten a B or better in a college-level course. Using students' high school GPA instead of placement testing to make placement decisions was predicted to cut severe placement error rates in half (to 17 percent) (Belfield & Crosta, 2012). https://ccrc.tc.columbia.edu/Community-College-FAQs.html Community College Research Center (CCRC) One CCRC study of a statewide community college system found that the ACCUPLACER severely misplaces 33 percent of entering community college students. In other words, based on their ACCUPLACER scores, one third of entering students were either "overplaced" in college-level courses and failed or "underplaced" in remedial courses when they could have gotten a B or better in a college-level course. Using students' high school GPA instead of placement testing to make placement decisions was predicted to cut severe placement error rates in half (to 17 percent) (Belfield & Crosta, 2012). (emphasis added) https://ccrc.tc.columbia.edu/Community-College-FAQs.html WHAT WE LEARNED Note that this is not ACCUPLACER’s “fault” - they advocate for multiple measures and promote a “Placement Program” as opposed to “Placement Testing” where a single subject test is one piece of the puzzle https://ccrc.tc.columbia.edu/Community-College-FAQs.html CAPR Center for the analysis of post-secondary readiness Community College Data: http://communitycollegedata.com/articles/multiple-measures-for-college-placement/ MULTIPLE MEASURES FOR COLLEGE PLACEMENT: GOOD THEORY, POOR IMPLEMENTATION by Alexandros M. Goudas April 2017 Education commission

Background and Definitions Why are multiple measures important? The on-ramp to our guided pathway WHAT WE WANT

Cochise College plan to implement Math Multiple Measures

CC Plan to implement Math MM Math Department’s MM Development Process How we expected to use Math MM What we expected to see

CC Plan to implement Math MM Math Department’s MM Development Process

CC Plan to implement Math MM How we expected to use Math MM

CC Plan to implement Math MM What we expected to see

Obstacles

Obstacles Students faced Advisers faced Can we insert the Math Placement Path Cheat Sheet on a slide around here somewhere? I can’t figure it out.

Obstacles Students are unaware of what is needed from them.

Obstacles Adviser Survey Results

Obstacles Survey Results How do you define multiple measure placement? Estimate the percentage of students you place through methods other than the traditional placement test. Estimate the percentage of students who provide you with a high school transcript. If presented with a high school transcript how do you process it? Do you think using multiple measures is beneficial to students? Do you think Cochise is doing enough to implement multiple measures placement? Are you comfortable placing students using methods other than the traditional placement test results? Please explain what you would need to be more confident with other (placement) methods. Are there other types of multiple measures that you think Cochise should adopt? Please list other types of multiple measures that you would like for Cochise to adopt. Why do you think multiple measure placement is not being used for more students at Cochise College?

“[Multiple measure placement] lacks structure and is very subjective depending on the advisor/counselor seeing the student. With the exception of Math, I think high school grades are inflated and I don’t trust English grades on a high school transcript to tell me whether or not a student can read/write at a college level.” - MM Advisor Survey Respondent

Obstacles Survey Results Training Leadership Awareness Skepticism Why do you think Multiple Measure Placement is not being used for more students at Cochise College?

Training Obstacles Survey Results Leadership Awareness Skepticism “Most of my knowledge… comes from fellow advising co-workers.” “Lack of knowledge” “Lacks structure” “it is something we need more clarity on.” Why do you think Multiple Measure Placement is not being used for more students at Cochise College?

“lack of push by management” Obstacles Survey Results Training Leadership Awareness Skepticism “lack of push by management” “I would prefer to have official information/recommendations from the English & Math departments” “Training and policy need to be revisited to create a concise list of all [Multiple Measure] exceptions.” "First rule of leadership: Everything is your fault." -Hopper, A Bug’s Life

“High school transfers are not bringing their transcript” Obstacles Survey Results Training Leadership Awareness Skepticism Students need to be made aware to bring in all unofficial documentation” “High school transfers are not bringing their transcript” “It is unhelpful and confusing… having just one type of multiple measure for placements rather than all” “Most students do not come to registration with a copy of their high school transcripts or SAT/ACT scores for me to review.” “Students don’t want to go get a copy of their HS transcript then come back later to register”

“I don’t have any faith in HS GPA or Grades” Obstacles Survey Results Training Leadership Awareness Skepticism “I don’t have any faith in HS GPA or Grades” “[Multiple Measure placement is] very subjective depending on the advisor/counselor” “Lets see some results of pass rates on students who have been placed with MMPL”

Keep moving forward! You failed!

Working Multiple Measures into the fine print

Working MM into the Fine Print Transition to ACCUPLACER Next Generation Math Department Objectives Results

Working MM into the Fine Print Transition to ACCUPLACER Next Generation In April, 2017, Cochise transitioned from using ACCUPLACER Classic placement tests to ACCUPLACER Next Generation Transition to ACCUPLACER NG Classic ACCUPLACER tests Arithmetic Elementary Algebra College Level Math Next Generation ACCUPLACER tests Arithmetic Quantitative Reasoning, Algebra, and Statistics Advanced Algebra and Functions

Working MM into the Fine Print Transition to ACCUPLACER Next Generation Testing was invited to the February, 2017, math department meeting to discuss the transition. Discussion included: Transition to ACCUPLACER NG Insight statements Background questions Weighted measures Branching profiles Observations about the current math placement testing Departments notified in November 2016

Working MM into the Fine Print Transition to ACCUPLACER Next Generation Three math tests is a lot of math tests, so in addition to setting new cut scores the math department embraced an additional goal… Place students using the fewest number of math tests Without compromising placement validity Save time, reduce test fatigue, save money

Working MM into the Fine Print Math Department Objectives Use multiple measures to decrease the number of math placement tests students need to take. Ensure this method is placing students into the proper level math course.

Working MM into the Fine Print The Math Department developed Multiple Measure type pre-test questions that are used to start candidates in the appropriate test “Where/when/which was the last math class you completed?” “What grade did you earn?” “What degree are you pursuing?” “Do you think you are a good math student?”

Working MM into the Fine Print Transition to ACCUPLACER Next Generation Prior to April 2016 (ELA Branching Profile) Elementary Algebra < 40 40-70: PLACE >/= 71 Arithmetic College Level Math Any score: PLACE April 2016-April 2017 (CLM Branching Profile) College Level Math < 40 >/= 40: PLACE Elementary Algebra < 40 >/= 40: PLACE Arithmetic Any score: PLACE

Working MM into the Fine Print Transition to ACCUPLACER Next Generation Prior to April 2016 (ELA Branching Profile) Elementary Algebra < 40 40-70: PLACE >/= 71 Arithmetic College Level Math Any score: PLACE Current Branching Profile Answer a few questions….. Arithmetic QRAS Advanced Algebra PLACE April 2016-April 2017 (CLM Branching Profile) College Level Math < 40 >/= 40: PLACE Elementary Algebra < 40 >/= 40: PLACE Arithmetic Any score: PLACE

Working MM into the Fine Print Results

Working MM into the Fine Print Results Result: Number of students placing with only 1 math test increased from 16% to 73%

Working MM into the Fine Print Results

Working MM into the Fine Print Results

Working MM into the Fine Print Results Saved time Saved money Collecting MM type data Kept Multiple Measure concepts on the table

Working MM into the Fine Print Results The analysis found no significant difference in the grade distributions or success rate for students completing the mathematics class into which they were placed by Accuplacer and NextGen. Using only these criteria, it is concluded that the two placement tests are equally viable instruments. Method and Results The grades for students in 2016 and 2017 who enrolled in the math class recommended by their placement tests were analyzed. For this analysis, a grade of “A,” “B,” and “C” were considered a “Pass” of the developmental mathematics class while a grade of “D” or “F” was considered a failure. The grade of “W” indicated a student who dropped the class and was not included as either a “Pass” or “Fail.”

Working MM into the Fine Print Results The bar chart above indicates that the percentages of “passing” grades, in general, were higher for the 2017 than the 2016 cohorts but this investigation did not find that the difference was statistically significant. Thus, it seems reasonable to conclude that both tests correctly place students in developmental mathematics classes.

How are students reacting? Students were asked to rate if they believe their placement or previous math course prepared them for the course they were currently enrolled in.

Some comments made by students: “081 is like a refresher class for me, I knew things already but if I was placed higher I wouldn’t have been too optimistic.” “My placement test allows me to be able to find a comfortable spot for me to perfect and learn new math skills.”

Discussion: the future of this valuable commodity

Discussion What are your MM strategies? What are your successes? Failures? What are your obstacles? What can we do as a state to implement MM that are fair and balanced yet work within our unique academic cultures?

Up