Bridget Arend University of Denver If You Grade it Will They Learn? A Study About Assessment in Online Courses Bridget Arend University of Denver Bridget Arend, University of Denver 2006 Sloan-C Conference

Why is assessment important? Behavioral to cognitive Admin/teacher-centered to learner-centered Quantitative to qualitative Bridget Arend, University of Denver 2006 Sloan-C Conference

“Will this be on the test?”

Importance of assessment Students typically concentrate their efforts on whatever content or cognitive skills they believe will be tested (Black & William, 1998; Bull & McKenna, 2004, Shepard, 2000) Different forms of assessment encourage different types of learning (Gipps, 1994; Hynd et al., 2000; McKeachie et al., 1986; Pintrich, 1989) The form of an exam or essay question can affect how students study (Entwistle, 1996) The nature of online discussion questions influence the types of responses (Meyer, 2004) Q to group – does this seem true based on your experiences? What examples can you share where students adapt to your assignments or tests? Bridget Arend, University of Denver 2006 Sloan-C Conference

Criticisms of HE Assessment Classroom assessment often encourages superficial and rote learning (Black & William, 1998) Grading is over-emphasized while learning is under-emphasized (Assessment Reform Group, 1999) Higher education instructors do not give priority to providing feedback to students on their learning (Cross, 1988)

The Big Lesson From the Literature: Assessment for learning rather than assessment of learning Bridget Arend, University of Denver 2006 Sloan-C Conference

Are online assessment practices different?

Dissertation Study Site = CCCOnline Two phase, mixed-method design Phase I/Quantitative = 60 courses across disciplines Surveys of 411 students and 51 instructors 60 course observations Phase II/Qualitative = follow up of 9 courses Email/phone interviews with 29 students and 8 instructors 9 course observations

Assessment Practices Summative and Formative Assessment

Summative Assessment Thomas Angelo’s framework of AAHE’s 9 Principles of Good Practice for Assessing Student Learning (1992) Principle #2 Use multiple methods Assess multiple dimensions Use multiple assessors Assess over time

Study Findings A “typical” online course: 29 assignments using 5 different methods Students use a variety of learning strategies 65% use non-graded assignments, No peer/group work Assignments due 10 out of 15 weeks

Formative Assessment Non-graded assignments What makes formative assessment successful? Frequent and timely feedback Precise feedback Students using feedback Changing course content or teaching methods when needed

Study Findings A “typical” course: 7 non-graded assignments Instructor interacts with each student at least 20 times, responds within 24-48 hours Instructors give precise feedback 96% of the time Instructors/students use feedback to improve course/learning only half of the time Even with rubrics, students wonder why they got a certain grade

Study Findings Critical Thinking Strategies Positively related Discussions Written assignments (Problem assignments, Journals) Negatively Related Finals/Midterms Non-graded assignments (Knowledge-based, multiple-choice questions)

Study Highlights Discussions Play informal, formative role Instructor role = neutral, participate infrequently, probe/challenge students

Study Highlights Knowledge-Based, Multiple-Choice Exams Least useful for understanding student knowledge Instructors use as discipline tool or because they are supposed to Has evolved from summative to formative role

Take-aways

Thank you! Bridget Arend University of Denver bridget.arend@du.edu http://portfolio.du.edu/barend