Innovative Rubrics for Online Discussions Douglas Hemphill Instructional Designer SUNY Oswego Focus on week long, module specific discussions.

Deconstructing the Discussion Art from Todd Mclellan

What is the Purpose of a Discussion Developing a Student Sense of Community Develop Communication Skills Analyzing Student Learning

What is the purpose of a rubric? Meant to communicate expectations. Provides consistent criteria for grading. Provides feedback to student. A scoring guide used to evaluate elevate the quality of student posts via modeling desired behavior.

Quantitative versus Qualitative Quantitative – How Much Qualitative – How Good

Quantitative Measuring the level of participation. How often do students post? How many primary posts, how many responses? How should student activity be spread out? The problem of the last day post. The seven day model: Sunday Monday Tuesday Wednesday Thursday Friday Saturday Post Period 1 Post Period 2

Pure Quantitative Very simply Rubric. Binary Grading – Acceptable vs. Not Acceptable: Outline criteria outside Rubric. 5 – acceptable posts. 4 – Four acceptable posts. 3 – Three acceptable posts. 2 – Two acceptable posts. 1 – One acceptable post. 0 – No posts made.

Qualitative – Analyzing Student Learning Apply a pedagogical model aimed toward accomplishing the learning outcomes you want from the discussion. Image from: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

Reflect on Course Content (Learning Outcome) Information and knowledge used in the post are accurate. It is very clear that the student understood course readings were understood by their incorporation into postings. The student demonstrates high level comprehension of the concepts from the course readings by connecting the appropriate reading concepts with the discussion question. The student demonstrates mastery of significant ideas relevant to the issue under discussion. This is indicated by correct use of terminology, precise selection of the pieces of information required to make a point, correct and appropriate use of examples and counterexamples, demonstrations of which distinctions are important to make, and explanations that are concise and to the point. Example Criteria for Learning Outcomes or Behaviors The basics.

Construct New Knowledge (Learning Outcome) Response goes beyond simply answering the prompt and attempts to stimulate further thought and discussion. The post shows original thought that goes far beyond the obvious and explores the implications of the discussion question. Demonstration of Higher Level Learning My definition: Demonstration of knowledge that goes beyond what is directly provided by course materials or external sources.

Use Citations (Learning Outcome and Desired Behavior) Sources are appropriately cited. Sources are integrated in a way that support’s a students point. External source choices show student’s efforts to go beyond the supplied course materials. Using and Moving beyond the course materials

Ask Questions (Desired Behavior) The student actively stimulates and sustains inquiry by asking thoughtful questions. The student generates questions that explore concepts not answered within the course materials. The student generates questions which will help other student’s in understanding of course materials. Encouraging Interactive Behavior and Metacognition

Adversarial Concepts (Overall Strategy) Develop Alternative Perspectives – The student is able to identify and address weaknesses in their own claims/statements/arguments and/or opinions. Evaluate and Challenge Alternative Positions - The student is able to identify and expose weaknesses in the claims/statements/arguments and/or opinions of others. Defend Positions –The student is able to adjust their claims/statements/arguments and/or opinions in the face of scrutiny, either by reinforcement or refinement. Structuring a Discussion Strategy

Other Rubric Considerations Discrete Criteria versus Aggregate Criteria Not every criteria has to be worth the same. Negative Point Values (Penalties) Poor naming conventions Inappropriate Behavior (Breach of Netiquette) Late Submissions Other Considerations Negative Point Values exist outside the Rubric Proper

Customized Rubrics versus Generic Rubrics Is there a reason to customize? Minor customization can make management more complicated. Large scale changes of a rubric format can cause student confusion. Review the learning outcomes you desire from your discussions, then decide whether customization is appropriate. For example, consider the expectations and outcomes desired from an “Ice Breaker Discussion” versus an academically focused discussion within a module. Course specific discussions versus assignment level discussions. Customization can take place in the discussion question, rather than the rubric.

Pure Qualitative A detailed rubric that allows a discussion to be evaluated based on a student’s work throughout a module, rather on their individual posts. The focus is on the quality over the number of posts, though the number of posts could be part of the score calculation. Quality of Posts Grammar and Spelling Number of Posts Activity Conciseness Examples of types of Rubrics Pure Qualification

Simple Hybrid Provides for individual grading of posts within a single discussion. Any number of posts can be required. This rubric is based on a 4 point scale, which is easy to understand but can be somewhat harsh. 4 - The comment is accurate, original, and relevant; it teaches us something new, and is well written. Four point comments add substantial teaching presence to the course, and stimulate additional thought about the issue under discussion. 3 - The comment lacks at least one of the above qualities, but is above average in quality. A three point comment makes a significant contribution to our understanding of the issue being discussed. 2 - The comment lacks two or three of the required qualities. Comments which are based upon personal opinion or personal experience often fall within this category. 1 - The comment presents little or no new information. However, one point comments may provide important social presence and contribute to a collegial atmosphere. 0 - No post was made, or the comment adds no value to the discussion. Includes most “I agree” comments. Hybrid is a combination of Qualitative and Quantitative

Simple Hybrid – Friendlier Scoring Similar to the previous, but on a modified 10 point scale. 10 - The comment is accurate, original, and relevant; it teaches us something new, and is well written. Four point comments add substantial teaching presence to the course, and stimulate additional thought about the issue under discussion. 9 - The comment maintains all of the above qualities, but not as well as it potentially could have. This comment still adds substantial teaching presence to the course, and stimulates additional thought about the issue under discussion. 8 - The comment lacks at least one of the above qualities, but is above average in quality. A three point comment makes a significant contribution to our understanding of the issue being discussed. 7 - The comment lacks two or three of the required qualities. Comments which are based upon personal opinion or personal experience often fall within this category. 6 - The comment presents little or no new information. However, one point comments may provide important social presence and contribute to a collegial atmosphere. 0 - No post was made, or the comment adds no value to the discussion. Includes most “I agree” comments. Aligns with A+, A, B, C, D, E

First Post Worth More In this example, while a student is expected to make three posts, the student's first post is worth more than their response posts. 3 - The comment is accurate, original, and relevant; it teaches us something new, and is well written. Four point comments add substantial teaching presence to the course, and stimulate additional thought about the issue under discussion. 2 - The comment maintains all of the above qualities, but not as well as it potentially could have. This comment still adds substantial teaching presence to the course, and stimulates additional thought about the issue under discussion. 1 - The comment presents little or no new information. However, one point comments may provide important social presence and contribute to a collegial atmosphere. 0 - No post was made, or the comment adds no value to the discussion. Includes most “I agree” comments. Refer to not requiring all criteria to be worth the same amount.

Addative Rubric Take the required posts and use one of the other rubric models. Once the initial requirements have been met, each additional post will be worth additional posts, up to the maximum possible point value. Encourages posting beyond the minimal requirements.

https://tinyurl.com/RUBRICCIT2019 Rubric Examples https://tinyurl.com/RUBRICCIT2019 Plug for Crafting an Engaging Discussion at 8:45 in HUM 2043.