Designing a Course for Evaluation: Triangulation of assessment to improve teaching in a single course section Richard W. Stratton Economics Department The University of Akron November 7, 2003

The University of Akron Begin at the beginning How could I design a course that would allow me to evaluate its effectiveness objectively? 11/28/2018 The University of Akron

The University of Akron Begin at the beginning Set specific goals and objectives Identify course components Identify assessment tools Evaluate success 11/28/2018 The University of Akron

Set specific outcome goals Mine were based on Hansen’s proficiencies (2001 JEE 32:232-233) access existing knowledge demonstrate command of knowledge interpret knowledge find and interpret existing economic data apply existing knowledge 11/28/2018 The University of Akron

Set specific outcome goals Objectives (cont.) demonstrate their ability to apply economic knowledge effectively employ select analytical and critical thinking tools used by economists demonstrate their ability to apply these tools persist to the end of the semester 11/28/2018 The University of Akron

The University of Akron Two objectives Students will be able to write a well reasoned supporting a policy recommendation Students will demonstrate ability to work cooperatively with others (Teams) 11/28/2018 The University of Akron

Design course components to address goals and objectives Worksheets – prior to class Team work – in class Participation – in class CAT – in class Essays – after class Exams – after class 11/28/2018 The University of Akron

Criterion based Mastery influenced Worksheets – Pre-class encounter Basis of team exercises CAT – student assessment of class Essays – Rubric based grading Exams 11/28/2018 The University of Akron

The University of Akron How well did I do? Did Retention improve? no - ~50% Did the grade distribution improve? Yes - 40% of completers A or B 11/28/2018 The University of Akron

The University of Akron How well did I do? Analysis of Final Essay Used rubric to determine performance 11 of 21 acceptable on all 11 criteria 14 of 21 acceptable on 9 of 11 criteria 13 of 22 acceptable overall points 15 of 22 acceptable on 7 of 11 dimensions 11/28/2018 The University of Akron

The University of Akron How well did I do? Learning Gains Pre-test / Post-test Richard Hake’s “normalized gain” (Post – Pre)/(40 – Pre) Average 13% gain 8 students more than 1 s.d. above 0 12 students within 1 s.d. of 0 11/28/2018 The University of Akron

The University of Akron How well did I do? Student CATs Increase rating of all three components 5 point scale Worksheets 2.48 to 2.88 sig. 1% Essays 2.29 to 2.35 ns Class time 3.24 to 3.35 ns 11/28/2018 The University of Akron