1. MATH ASSESSMENT TEST OCMA May, 2008

2. HISTORY OF MAT Test originally developed in late 60’s

3. HISTORY OF MAT Test originally developed in late 60’s Original purpose was to verify foreign credentials of applicants

4. HISTORY OF MAT Test originally developed in late 60’s Original purpose was to verify foreign credentials of applicants Name changed by 1970 to Math Inventory test and given to all students taking first semester math

5. HISTORY OF MAT Test originally developed in late 60’s Original purpose was to verify foreign credentials of applicants Name changed by 1970 to Math Inventory test and given to all students taking first semester math Purpose at this time was to collect data on students to determine their math skill level

6. HISTORY OF MAT Test originally developed in late 60’s Original purpose was to verify foreign credentials of applicants Name changed by 1970 to Math Inventory test and given to all students taking first semester math Purpose at this time was to collect data on students to determine their math skill level Math Learning Centre developed in 1973 and MAT now had a remedial component

7. HISTORY OF MAT Test originally developed in late 60’s Original purpose was to verify foreign credentials of applicants Name changed by 1970 to Math Inventory test and given to all students taking first semester math Purpose at this time was to collect data on students to determine their math skill level Math Learning Centre developed in 1973 and MAT now had a remedial component Remedial modules were further developed during the mid 70’s

8. ORIGINAL TEST Modular multiple choice test Students given overall grade on the test + any modules that were weak Remedial work assigned only on those topics that were weak Work completed through the Math Learning Centre during the semester

9. Past & Present Diagnostic test with a remedial component Push to use test as an assessment tool for streaming

10. Past & Present Diagnostic test with a remedial component Students had higher level math at entry Push to use test as an assessment tool for streaming Student coming in with more varied math background

11. Past & Present Diagnostic test with a remedial component Students had higher level math at entry Students scored higher on average on the MAT Push to use test as an assessment tool for streaming Student coming in with more varied math background Students scoring lower on the MAT

12. Original Business and Technology Versions Business 1.Fractions 2.Decimals 3.Percents 4.Order of Operations and Law of Signs 5.Equations

13. Original Technology and Business Versions Business 1.Fractions 2.Decimals 3.Percents 4.Order of Operations and Law of Signs 5.Equations Technology 1.Fractions 2.Decimals 3.Percents 4.Order of Operations 5.Laws of Signs 6.Exponents and Radicals 7.Algebraic Fractions 8.Equations and Formulas 9.Mensuration 10.Trigonometry and Geometry 11.Logarithms

14. Current Computerized Version 1.Fractions 2.Decimals 3.Percents 4.Order of Operations 5.Laws of Signs 6.Exponents 7.Basic Algebra 8.Equations 9.Radicals 10.Algebraic Fractions 11.Mensuration 12.Trigonometry and Geometry 13.Logarithms It is possible to select different modules by the program of the students.

15. Current Computerized Version 1.Fractions 2.Decimals 3.Percents 4.Order of Operations 5.Laws of Signs 6.Exponents 7.Basic Algebra 8.Equations 9.Radicals 10.Algebraic Fractions 11.Mensuration 12.Trigonometry and Geometry 13.Logarithms It is possible to select different modules by the program of the students. FALL 2007 Business Students did only Topics 1-5 Technology students did Topics 1- 10, as research showed these were the critical modules.

16. Current Computerized Version 1.Fractions 2.Decimals 3.Percents 4.Order of Operations 5.Laws of Signs 6.Exponents 7.Basic Algebra 8.Equations 9.Radicals 10.Algebraic Fractions 11.Mensuration 12.Trigonometry and Geometry 13.Logarithms It is possible to select different modules by the program of the students. FALL 2007 Business Students did only Topics 1-5 Technology students did Topics 1- 10, as research showed these were the critical modules. (54 questions) FALL 2008 Business Students will add Topics 7 & 8 (40 questions)

17. PAST AND PRESENT PAST 1.Written on first day of class PRESENT 1.Students write prior to semester start – available from Feb.

18. PAST AND PRESENT PAST 1.Written on first day of class 2.Pencil and Paper PRESENT 1.Students write prior to semester start – available from Feb. 2.Computerized test completed in a proctored site. (Practice tests available online with built in remedial.)

19. PAST AND PRESENT PAST 1.Written on first day of class 2.Pencil and Paper 3.Results given in weeks 2 or 3 PRESENT 1.Students write prior to semester start – available from Feb. 2.Computerized test completed in a proctored site. (Practice tests available online with built in remedial.) Results given on exit from test 3.Results given on exit from test

20. PAST AND PRESENT PAST 1.Written on first day of class 2.Pencil and Paper 3.Results given weeks 2 – 3 4.Remedial done during semester through the MLC PRESENT 1.Students write prior to semester start – available from Feb. Practice tests available online with built in remedial 2.Computerized test completed in a proctored site. (Practice tests available online with built in remedial.) 3.Results given on exit from test 4.Remedial completed online prior to September 30.

21. Current Research The push to change the use of the assessment test for streaming meant that the test needed to be “tested”. We are currently doing –Item Analysis on each question in the test (new) –Validity of the test as a predictor of success in semester 1 college math (this is ongoing from previous years)

22. Item Analysis of Test (Creating good questions in a multiple choice test) 1.The incorrect options, distractors, must be a clue to the nature of the misunderstanding and thus prescriptive of appropriate remediation 2.Each question must discriminate correctly between the top 27% and the bottom 27% of students.

23. Validity Test validity refers to the degree to which the test actually measures what it claims to measure, in this case, success in first semester college math courses. It is also the extent to which inferences, conclusions, and decisions made on the basis of test scores are appropriate and meaningful.

24. ITEM ANALYSIS DISTRACATORS The good and the bad! Green is correct answer

25. Discrimination of Question 7 Green is correct answer.

26. Question 7 Each distractor for this question represented a common error in dealing with mixed fractions The question discriminates correctly between the top 27% of students and the bottom 27 % of students

27. Question 8 This question did not give much useful information since over 90% of students answered correctly (index of difficulty is low) The distractors were reviewed and changed to reflect more typical mistakes that students make when multiplying fractions

28. Item Analysis We are Reviewing each question in this manner to ensure that each question adds to our knowledge of any weaknesses students may have i.e. it has appropriate distractors

29. Item Analysis We are Reviewing each question in this manner to ensure that each question adds to our knowledge of any weaknesses students may have i.e. it has appropriate distractors Reviewing the discrimination of each question to ensure that the question is not misleading (discrimination – top 27% should have a higher percent correct than bottom 27%)

30. Item Analysis We are Reviewing each question in this manner to ensure that each question adds to our knowledge of any weaknesses students may have i.e. it has appropriate distractors Reviewing the discrimination of each question to ensure that the question is not misleading (discrimination – top 27% should have a higher percent correct than bottom 27%) Changing our remedial to give specific mini lessons based on the wrong answer the student selects. (Currently it is a more general remedial on a topic such as adding fractions.)

31. Validity of Test for Placement This test was designed as a diagnostic tool to assess areas of weakness and provide remedial assistance. Over time we have shown that there is a positive linear relationship between the test score on our assessment and success in first year college mathematics BUT is it strong enough i.e. does it predict well enough to use as a placement test?

32. 2004 Research Questions 1.Does attendance affect performance in first semester math? 2.Are basic math skills on entry to the college a factor in determining student success?

33. Passed Low Attendance Independent Learners Passed High Attendance Passers Failed Low Attendance Failures Failed High Attendance Heart Breakers Attendance Correlation between final grade and attendance was statistically significant. (r=0.467, p=0.000)

34. One-way ANOVA: ASSESSMENT versus Quadrant Source DF SS MS F P Quadrant 3 54369 18123 69.33 0.000 Error 1148 300084 261 Total 1151 354453

35. Correlation coefficient r=0.43 and p=0.000 MAT% AS A PREDICTOR OF SUCCESS IN MA101 MAT CONTINUES TO BE A SIGNIFICANT PREDICTOR OF SUCCESS IN 2007

36. Validity of Test for Placement What does that mean to students, if they are streamed into an extra math course using this test?

37. MAT AS A PLACEMENT TEST (extra math course) Able to Pass first Semester Math TrueFalse Streamed into an extra math course based on Assessment Test YesCorrect Decision 1 (helps students succeed) No Correct Decision 1 This is the group we are trying to help. They do not have the basic math skills required to pass the regular first semester math course.

38. AS AN ASSESSMENT TEST Able to Pass first Semester Math TrueFalse Streamed into an extra math course based on Assessment Test YesTYPE 1 ERRORCorrect Decision NoCorrect DecisionTYPE 2 ERROR (issues other than math)

39. Type 1 Error (able to pass, extra math course) Type 2 Error (can’t pass, no extra course) Student is required to do the extra math course even though they are able to pass the first semester course. Student is not streamed into the extra math course when the course is required. Cost: 1.time: student is an unnecessary semester behind in math 2.cost of additional course that they did not require Cost: 1.time: student is a semester behind BUT still does not have basic math skills required to pass course COST OF ERRORS TO STUDENTS OR COLLEGE

40. ? QUESTIONS