Alternative Assessment GradSTEP 2011 John Morrell Blaine Smith

Which of these best describes the discipline in which you teach? 1. Natural Sciences / Math / Engineering 1. Natural Sciences / Math / Engineering 2. Humanities / Fine Arts 2. Humanities / Fine Arts 3. Social Sciences 3. Social Sciences

Think of a course in which you expect to be assessing students. How many students will be in this course? or fewer or fewer – – – – – – or more or more

Now think of an assignment in that course that will be challenging to grade. What kind of assignment is it? 1. Test / Exam 1. Test / Exam 2. Homework 2. Homework 3. Short Paper 3. Short Paper 4. Long Paper 4. Long Paper 5. Project 5. Project 6. Presentation 6. Presentation 7. Other 7. Other

Muddiest Point What is the most challenging or confusing aspect of assessment for you? What is the most challenging or confusing aspect of assessment for you?

Alternative Assessment The utilization of non-traditional approaches in judging student performance.

Alternative Assessment

Common Characteristics of Alternative Assessments Asks students to perform, create or produce something Evaluation criteria and standards are known to the student Involve interaction between assessor (instructor, peers, self) and person assessed Provides self-assessment opportunities for students Provides opportunities for both individual and group work Encourages students to continue the learning activity beyond the scope of the assignment

Key Features of Alternative Assessments Assesses a learning process and encourages student self-reflection Based on authentic task with real-world application Topics and means of expression are of interest to the students

Assess Process of Learning Flexible Flexible Show development Show development Increases communication Increases communication Promotes reflection Promotes reflection Provides feedback Provides feedback

Assess Process of Learning Process Journals Process Journals Interactive Journal Interactive Journal Drafts Drafts Portfolio Portfolio

Classroom Assessment Techniques Background Knowledge Probe: short, simple questionnaire given to students at the start of a course, or before the introduction of a new unit, lesson or topic. Background Knowledge Probe: short, simple questionnaire given to students at the start of a course, or before the introduction of a new unit, lesson or topic. Minute Paper: brief response to the following questions: “What was the most important thing you learned during this class?” and “What important question remains unanswered?” Minute Paper: brief response to the following questions: “What was the most important thing you learned during this class?” and “What important question remains unanswered?” Muddiest Point: jot down a quick response to one question: “What was the muddiest point in [the lecture, discussion, homework assignment, film, etc.]?” Muddiest Point: jot down a quick response to one question: “What was the muddiest point in [the lecture, discussion, homework assignment, film, etc.]?”

Authentic Assessment Close to actual practice Close to actual practice Real world scenarios Real world scenarios Relevance outside of class Relevance outside of class Larger audience Larger audience

Student Choice Choose topic or propose assessment Choose topic or propose assessment Help to develop evaluative criteria Help to develop evaluative criteria Mode of expression Mode of expression Visual Visual Written Written Multimodal Multimodal Genre Genre Media Media

Questions to Ask Does the assessment meet outcome goals? Does the assessment meet outcome goals? Does the assessment enable students to demonstrate their progress and capabilities? Does the assessment enable students to demonstrate their progress and capabilities? Do the assessments use authentic, real world tasks? Do the assessments use authentic, real world tasks? What options/choices are allowed? What options/choices are allowed? Will the assessment be meaningful and engaging to students so that they will be motivated? Will the assessment be meaningful and engaging to students so that they will be motivated? Does the assessment involve real problems, situations, and audiences? Does the assessment involve real problems, situations, and audiences?

Revamp an Assignment: 3-minute Paper On the top of your paper write down an assignment that could be revamped into an alternative assessment On the top of your paper write down an assignment that could be revamped into an alternative assessment Consider ways to make it: Consider ways to make it: Authentic Authentic Measure process Measure process Reflective Reflective Include student choice Include student choice Engaging and motivating Engaging and motivating

Grading

Denis Rancourt N6xXZ8 N6xXZ8 N6xXZ8 N6xXZ8 9/02/08/the-two-languages-of-academic- freedom/ 9/02/08/the-two-languages-of-academic- freedom/ 9/02/08/the-two-languages-of-academic- freedom/ 9/02/08/the-two-languages-of-academic- freedom/

What are the purposes of grading? Evaluation Evaluation Motivation Motivation Intrinsic vs. Extrinsic Intrinsic vs. Extrinsic Ranking relative to others Ranking relative to others More? More?

What are the potential problems with grading? Validity Validity Reliability and Accuracy Reliability and Accuracy Fairness Fairness Bias Bias Others? Others?

The Curve

Norm-Referenced Assessment Obtain information about a learner’s performance in relation to others’. Obtain information about a learner’s performance in relation to others’. ACT, GRE, MCAT, etc. ACT, GRE, MCAT, etc. Can take more than a year for a single multiple choice question to be approved. Can take more than a year for a single multiple choice question to be approved. Some instructors “curve” test scores for various reasons Some instructors “curve” test scores for various reasons Department requirements, personal preference, compensating for lower than average scores, etc. Department requirements, personal preference, compensating for lower than average scores, etc. Is the adjusted distribution fair to all of the students? Is the adjusted distribution fair to all of the students? Do you have other assessments that reflect similar score distributions for the same students? Do you have other assessments that reflect similar score distributions for the same students?

Criterion-Referenced Assessment Define the content for assessment in terms of what knowledge and skills the student must demonstrate Define the content for assessment in terms of what knowledge and skills the student must demonstrate Define a grading scale for judging student performance Define a grading scale for judging student performance Make sure that the assessment requires students to perform the same skills they practiced during their learning activities prior to the assessment Make sure that the assessment requires students to perform the same skills they practiced during their learning activities prior to the assessment Distribution can take any shape Distribution can take any shape

Teaching with Rubrics Evaluative Criteria Evaluative Criteria Quality Definitions Quality Definitions Scoring Strategy Scoring Strategy Holistic Holistic Analytic Analytic

Potential Problems with Rubrics Task specific evaluative criteria Task specific evaluative criteria Excessively general evaluative criteria Excessively general evaluative criteria Dysfunctional detail Dysfunctional detail Equating the test of the skill with the skill itself Equating the test of the skill with the skill itself From “What’s Wrong—and What’s Right—with Rubrics” in Educational Leadership (Oct. 1997) From “What’s Wrong—and What’s Right—with Rubrics” in Educational Leadership (Oct. 1997)

Class Participation 0.Absent.1. Present, not disruptive. Tries to respond when called on but does not offer much. Demonstrates very infrequent involvement in discussion. 2. Demonstrates adequate preparation: knows basic case or reading facts, but does not show evidence of trying to interpret or analyze them. Offers straightforward information (e.g., straight from the case or reading), without elaboration or very infrequently (perhaps once a class). Does not offer to contribute to discussion, but contributes to a moderate degree when called on. Demonstrates sporadic involvement. 3. Demonstrates good preparation: knows case or reading facts well, has thought through implications of them. Offers interpretations and analysis of case material (more than just facts) to class. Contributes well to discussion in an ongoing way: responds to other students' points, thinks through own points, questions others in a constructive way, offers and supports suggestions that may be counter to the majority opinion. Demonstrates consistent ongoing involvement. 4. Demonstrates excellent preparation: has analyzed case exceptionally well, relating it to readings and other material (e.g., readings, course material, discussions, experiences, etc.). Offers analysis, synthesis, and evaluation of case material, e.g., puts together pieces of the discussion to develop new approaches that take the class further. Contributes in a very significant way to ongoing discussion: keeps analysis focused, responds very thoughtfully to other students' comments, contributes to the cooperative argument-building, suggests alternative ways of approaching material and helps class analyze which approaches are appropriate, etc. Demonstrates ongoing very active involvement.

VALUE Rubrics (AAC&U) VALUE: Valid Assessment of Learning in Undergraduate Education VALUE: Valid Assessment of Learning in Undergraduate Education Intellectual and Practical Skills Intellectual and Practical Skills Inquiry and analysis Inquiry and analysis Critical thinking Critical thinking Creative thinking Creative thinking Written communication Written communication Oral communication Oral communication Reading Reading Quantitative literacy Quantitative literacy Information literacy Information literacy Teamwork Teamwork Problem solving Problem solving Personal and Social Responsibility Personal and Social Responsibility Civic knowledge and engagement Civic knowledge and engagement Intercultural knowledge and competence Intercultural knowledge and competence Ethical reasoning Ethical reasoning Foundations and skills for lifelong learning Foundations and skills for lifelong learning Integrative and Applied Learning Integrative and Applied Learning Integrative and applied learning Integrative and applied learning

Participatory / Consensus Rubrics Have students collaborate in the creation of a rubric. Have students collaborate in the creation of a rubric.

Self Evaluation Create opportunities for self reflection. Create opportunities for self reflection. Can you allow students to contribute a self-reflection as part of the assignment? Can you allow students to contribute a self-reflection as part of the assignment? Could you allow students to grade themselves? Could you allow students to grade themselves?

Revision and Test Correction Create structured opportunities for learning and improvement. Allow students to revise papers and correct test questions. Create structured opportunities for learning and improvement. Allow students to revise papers and correct test questions. Before or after assigning grades? Before or after assigning grades?

Other ideas? Non-binding grades. Non-binding grades. More? More?

Exercise Think of an assignment on which you might experiment with some form of alternative grading (3 mins) Think of an assignment on which you might experiment with some form of alternative grading (3 mins) Share with your neighbor (5 mins) Share with your neighbor (5 mins) Ideas? Ideas?

Return to Muddiest Point Other questions and thoughts? Other questions and thoughts?

Resources Vanderbilt Center for Teaching ( guides/assessment/) Vanderbilt Center for Teaching ( guides/assessment/) guides/assessment/ guides/assessment/ The CFT is hosting a workshop on January 26 entitled “Negotiating Student Expectations about Grades and Goals” from 4:10 – 5:30 The CFT is hosting a workshop on January 26 entitled “Negotiating Student Expectations about Grades and Goals” from 4:10 – 5:30 Furman Center for Teaching ( Furman Center for Teaching ( RubiStar – rubric templates ( RubiStar – rubric templates ( AAC&U VALUE Rubrics AAC&U VALUE Rubrics