1. Desire2Learn for Quality Matters Haomin Wang, Manager of Instructional Technology Mingming Shao, Instructional Designer Dakota State University

2. Background Growing attention on assessment Measurable learning objectives Alignments between learning objectives and assessment measures

3. Dakota State University A small public baccalaureate university in South Dakota Using Course Management System to support online and hybrid course delivery since 1998 Started using Desire2Learn as the state- wide CMS in Spring 2008

4. Quality Matters Developed by MarylandOnline Standard-based rubric to improve quality of online courses through a faculty peer review process Assessment aligned with learning objectives Assessment be formative and progressive Grading procedures be open and transparent to students

5. Desire2Learn to Help A built-in structure for integrating competency-learning objective-learning activity Learning activities associated with learning objectives Learning activities assessed with grading rubrics

6. D2L COA Structure

7. COA Hierarchy

8. Activities Learning activities can be discussion topics, assignment submissions, quizzes, and surveys. Most activities, except for quiz or grade item, need to be assessed through rubrics. Rubrics must be created before activities.

9. Rubrics A bulleted list or a tabular form specifying levels of performance and grading criteria. Performance benchmarks Assignment guidelines Assessment contract between students and instructor.

10. A Sample Rubric A 90 or higher B 80-89 C 70-79 D 60-69 Accuracy & Clarity Covered all key points. Well informed & supported. Clearly and effectively presented. Covered most key points. Generally accurate & clear. Support not always appropriate. Covered about half of the points. Some minor misconceptions or misinterpretations. Little support Missed more than half of the points. Some major misconceptions or misinterpretations. No support. Coherence & Focus Consistently coherent and focused Mostly coherent, with occasional digression Noticeable lack of focus, with irrelevant points. Much of the content is irrelevant. Text Well organized. Text flows smoothly. Free of mechanical errors. Text is generally organized & easy to read. Lacks some transitions. Some mechanical errors. Text is readable, but does not flow. Lacks transitions. A number of mechanical errors. Text lacks organization & is hard to read. Some major mechanical errors.

11. Benefits of Rubrics (1) Rubrics help students and teachers define "quality." Rubrics reduce the time teachers spend grading student work and make it easier for teachers to explain to students why they got the grade they did and what they can do to improve.

12. Benefits of Rubrics (2) When students use rubrics regularly to judge their own work, they begin to accept more responsibility for the end product. It cuts down on the "am I done yet?" questions.

13. What We Have Learned Control levels of competencies Define Measurable learning objectives Write rubrics with good granularity Monitor learner progress Manage learning paths

14. Control Levels of Competency How many levels?

15. Measurable Learning Objectives Measurable Action-oriented Specific

16. Some Sample Learning Objectives Be able to describe the conceptual structures of arrays. Be able to describe how arrays are initialized and accessed. Be able to declare and initialize arrays, populate arrays with data, and retrieve data from arrays. Be able to integrate the applications of arrays with loops.

17. Building Granular Rubrics Measurable: Criteria should be in measurable terms and can be assessed in reference to learning outcomes. Granular: Levels of performance should be differentiated. Comprehensive: All important aspects of the task should be covered.

18. A Sample of Granularity A Level: Posts are consistently reflective, insightful, original, and thoughtful. B Level: Posts are generally reflective and thoughtful, with some original thoughts. C Level: Posts are sometimes reflective, and occasionally show original thoughts. D Level: Posts are not reflective nor original. Posts mostly echo other posts.

19. Rubric Resources http://rubistar.4teachers.org http://www.rubrics4teachers.com/ http://school.discoveryeducation.com/schr ockguide/assess.html http://school.discoveryeducation.com/schr ockguide/assess.html iRubric: http://www.blurtit.com/answers.php?quer y=rubrics&uid=adwords-TS10269-SJ- 20071211-050021- 37&search_type=content&ad=108398333 0 iRubric http://www.blurtit.com/answers.php?quer y=rubrics&uid=adwords-TS10269-SJ- 20071211-050021- 37&search_type=content&ad=108398333 0

20. Monitoring Learning Progress Competency Results ClassList > Learner Progress

22. Using Release Conditions to Manage Learning Paths Linear paths Restrictive preview

23. Questions? Mingming.Shao@dsu.edu Haomin.Wang@dsu.edu