1 Selecting Appropriate Assessment Methods Presented at the Teaching & Learning Innovations 17 th Annual Conference At the University of Guelph May 12, 2004 Peggy Maki, Ph.D. Material from Maki’s forthcoming book (May, 2004): Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus and AAHE

2 Alignment Methods that align with pedagogy, content, curricular and instructional design, educational tools, educational experiences

3 Assumptions Underlying Teaching Actual Practices Assumptions Underlying Assessment Tasks Actual Tasks

4 Inventories Position us to reflect on our expectations in terms of pedagogy Position us to think about the relationship between how and what students learn and how we choose to assess that learning

5 Methods: Emerge from multiple and varied opportunities to learn Emerge from assessment practices such as self-reflection, peer to peer feedback, instructor feedback Align with students’ learning histories and our values

6 Methods Reveal different dimensions of learning: Recall and recognition Higher order thinking abilities Students’ abilities to construct meaning

7 Standardized Instruments Psychometric approach—values quantitative methods of interpretation History of validity and reliability Quick and easy adoption and efficient scoring One possible source of evidence of learning

8 Do Not Provide: Typically, evidence about strategies, processes, ways of knowing, understanding, and behaving that students draw upon to represent learning Evidence of complex and diverse ways in which humans construct and generate meaning

9 Highly useful results that relate to pedagogy, curricular design, sets of educational practices

10 Locally-Designed Methods Align with outcome statements Relate to assessment methods students have experienced Invite collaboration in design (faculty, students, TAs, tutors)

11 Do Not Provide Usually, easy scoring unless closed- ended questions are used.

12 Authentic, Performance-based Methods Focus on integrated learning Directly align with students’ learning experiences Provide opportunity for students to generate responses as opposed to selecting responses

13 Provide opportunity for students to reflect on their performance Provide opportunity for quantitative and qualitative assessment

14 Some Methods That Provide Direct Evidence Student work samples Collections of student work (e.g. Portfolios) Capstone projects Course-embedded assessment

15 Performance on a case study/problem Performance on problem and analysis (Student explains how he or she solved a problem) Performances, productions, creations

16 Externally or externally juried review of student projects Progression from writing to speaking to visual presentation

17 Mapping, charting, graphing (mind mapping or conceptual mapping) Observations of student behavior Disciplinary or professional practices

18 Internships and Service Projects Oral Examinations Self-reflective journals or writing Learning logs

19 Team-based or Collaborative projects Performance on national licensure examinations

20 Locally developed tests Standardized tests Pre-and post-tests

21 Development of Criteria (Scoring Rubrics) Criteria descriptors (ways of thinking, knowing or behaving represented in work): Creativity Self-reflection Originality Integration Analysis Disciplinary logic

22 Criteria descriptors (identify traits of the performance, work, text) Coherence Accuracy or precision Clarity Structure

23 Performance descriptors (how well students execute each criterion or trait along a continuum of score levels): Exemplary—Commendable-- Satisfactory Unsatisfactory Excellent—Good—Needs Improvement— Unacceptable Expert—Practitioner—Apprentice--Novice

24 Development of Scoring Rubrics Emerging work in professional and disciplinary organizations Research on learning (from novice to expert) Student work

25 Interviews with students Experience observing students’ development

26 Scoring Sheet (rubric) Criterion 1 Criterion 2 Criterion 3 Criterion 4