1. Two Levels of Course Assessment to Identify Student Learning Carol Lerch, Ph.D lerch@dwc.edu S 52 Friday 11:45 – 12:35

2. Background Assess new format – College Algebra – Introduced fall 2000 – Previous course failure rate +40% AMATYC 20072 Assess new format – College Algebra – Introduced fall 2000 – Previous course failure rate +40% Looking at students in one specific major Assess new format – College Algebra – Introduced fall 2000 – Previous course failure rate +40% Looking at students in one specific major Course description – Purpose of this course is to provide students with a proficiency in algebraic skills necessary for success in future coursework …

3. Collect Data Specifics Fall 2001 – Summer 2007 – Pre-2000 60% pass (anecdotal) Fall average 85% Spring average 64% Summer 80% Includes Withdraws AMATYC 20073

4. Analysis Improvement over previous College Algebra course AMATYC 20074 Improvement over previous College Algebra course Fall pass rates acceptable – Continuing to improve Improvement over previous College Algebra course Fall pass rates acceptable – Continuing to improve Spring pass rates much lower – Needs analysis & new plan Improvement over previous College Algebra course Fall pass rates acceptable – Continuing to improve Spring pass rates much lower – Needs analysis & new plan Summer needs more data – Online first time summer 07

5. Changes to Course Added text related course management system – online homework – Interactive textbook – Instant help – Video, animation AMATYC 20075 Added text related course management system – online homework – Interactive textbook – Instant help – Video, animation Added section JITTs – Just-in-time-tests

6. Purpose of AssessmentPlan AMATYC 20076 Pass rates – Pass rates for particular course Pass rates – Pass rates for particular course Course sequence – Semesters to complete math sequence Thanks to Jack Rotman Pass rates – Pass rates for particular course Course sequence – Semesters to complete math sequence Thanks to Jack Rotman Concept assessment – Learning goals/objectives

7. Semesters to Complete Semesters to pass through the math sequence, passing MA152 for each semester since fall 03: – 82% of those who started in MA098 passed MA152 in 4 semesters (4 courses) – 93% of those who started in MA122 passed MA152 in 3 semesters (3 courses) AMATYC 20077

8. Levels of Assessment Gathering information about students’ achievement (Marzano, 2001, p. 115) AMATYC 20078 Level I Assessment – General Course information Pass rates Success in subsequent math courses Semesters to complete sequence

9. Levels of Assessment AMATYC 20079 Level II Assessment – Content specific – Concepts Learning Map o Summative assessment  Overall course content  Final exam Item Analysis o Formative assessment o Individual concepts tested

10. Cycle of Assessment PlanCollect DataAnalyze AMATYC 200710

11. Online homework required as of spring 2006 – Introduced as option for fall 2005 Daily quiz – 2-3 questions on concepts in each section Read text Come to class prepared Carrot – replace test grade with quiz average AMATYC 200711 Plan

12. Collect Data Fall pass rate improved overall – 90% Spring stayed same – 63% AMATYC 200712

13. Analysis Fall pass rates acceptable – Online homework helped Anecdotal evidence – Students like all options available with online homework – Have more opportunities to learn Spring rates need improvement AMATYC 200713

14. Plan Assesses student learning – Concept assessment Learning goals/objectives Develop “Learning Map” – Identify major concepts Identify teaching objectives Identify student learning objectives Means of assessment Set success standard AMATYC 200714 Plan

15. Learning Map AMATYC 200715

16. Learning Map 1 st step – Major topics of course Linear equations, quadratic equations, inequalities, absolute value, exponential & log equations Modified as further developed AMATYC 200716 1 st step – Major topics of course Linear equations, quadratic equations, inequalities, absolute value, exponential & log equations Modified as further developed 2 nd step – What do I want them to learn Teaching objectives

17. Learning Map Knowledge level Cognitive Process – Action verbs AMATYC 200717

18. Levels of Knowledge AMATYC 200718 Factual – Basic elements of the discipline – Facts specific to topic Parts of linear or quadratic equation Methods to solve quadratic equations Factual – Basic elements of the discipline – Facts specific to topic Parts of linear or quadratic equation Methods to solve quadratic equations Procedural – Processes of mathematics – Algorithms, techniques, methods Complete the square Quadratic formula

19. Levels of Knowledge Conceptual – Interrelationships – Classifications – Principles, generalizations Pythagorean Theorem Properties AMATYC 200719 Conceptual – Interrelationships – Classifications – Principles, generalizations Pythagorean Theorem Properties Metacognitive – Self knowledge – Students need to reflect to learn

20. Teaching vs. Learning Objectives Teaching objective – What I will do – What I want the students to know AMATYC 200720 Student learning experience What they will do Student learning objective – “Students will be able to …”

21. Assessment on Map Each objective has generic question Indicates how student is assessed AMATYC 200721

22. Learning Map Gave to students as part of syllabus – Type of questions included – ‘Truth in advertizing’ Matched final exam to map – Redesigned map Concepts tested but not part of map – Redesigned final exam Some concepts not tested but important Some concepts tested multiple times AMATYC 200722

23. Level II Assessment Assess concepts learned Allow overall picture of student progress Shows strengths Identifies weaknesses – Areas for further development AMATYC 200723

24. Collect Data Collected chapter test data – Right/wrong only Collected final exam item results 4 th semester of data collection Stats: Percent correct AMATYC 200724

25. Collect Data Identified concept tested by semester – Shows what is not tested (over-tested) AMATYC 200725 Results of chapter tests for Fall 06, Spring 07, Summer 07

26. Analysis - Tests More Student Learning Objectives – Needed to add very specific objectives – Dropped some that I thought I wanted but don’t test for AMATYC 200726 More Student Learning Objectives – Needed to add very specific objectives – Dropped some that I thought I wanted but don’t test for Success measure – 65% or better correct More Student Learning Objectives – Needed to add very specific objectives – Dropped some that I thought I wanted but don’t test for Success measure – 65% or better correct Results below need attention More Student Learning Objectives – Needed to add very specific objectives – Dropped some that I thought I wanted but don’t test for Success measure – 65% or better correct Results below need attention 0 - 10% indicates problem with question as well as topic

27. Analysis – Final exam Final used by all sections – Data just my sections Shows problems with online course on specific concepts AMATYC 200727

28. Asides Only list what is being assessed – If can’t assess, don’t put as objective AMATYC 200728 Have a thick skin Data show changes made in course – Provides longitudinal look at course Allows comparison of student results by semester or year Data show where more emphasis needed in teaching

29. Before Beginning Assessment Purpose of assessment – What do you want to know? – Why do assessment? Action leading from assessment – What do you do with results? AMATYC 200729

30. 5 Basic Questions AMATYC 200730

31. References Angelo, T. A. (1991). Introduction and overview: From classroom assessment to classroom research. Classroom research: Early lessons from success. T. A. Angelo and K. P. Cross. San Francisco, Jossey-Bass: 7-15. Brockbank, A. and I. McGill (1998). Facilitating reflective learning in higher education. Philadelphia, Society for Research into Higher Education & Open University Press. Carson, T. (1990). "What kind of knowing is critical action research?" Theory Into Practice XXIX(3): 167-173. Cochran-Smith, M. and S. L. Lytle (1993). Inside outside: Teacher research and knowledge. New York, Teachers College. Elliott, J. (1991). Action research for educational change. Philadelphia, PA, Open University Press. Leibowitz, M. (1997). Instruction for process learning. Supporting the spirit of learning: When process is content. A. Costa and R. M. Liebmann. Thousand Oaks, CA, Corwin Press, Inc.: 47-54. Marzano, R. J. (2001). Designing a new taxonomy of educational objectives. Thousand Oaks, CA, Corwin Press. Mezirow, J. (1990). Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning. San Francisco, Jossey-Bass. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA, Harvard University. AMATYC 200731

32. Participation Gather into groups Pick one course you teach List the major concepts – Very few topics Pick one or two to use Identify Teaching Objective Identify Student Objective Identify knowledge levels – Can be more then one Identify possible means of assessment Identify success measures AMATYC 200732

33. SHARE AMATYC 200733

34. Two Levels of Course Assessment to Identify Student Learning Carol Lerch, Ph.D lerch@dwc.edu S 52 Friday 11:45 – 12:35