1. Reflections, Discussion Threads and Peer Review for Assessment in Online Learning Kristine Rabberman, Ph.D. Carol A. Muller, Ph.D.

2. © 2012 SAS Presentation Outline 1.Penn Arts & Sciences Online Learning 2.Credit Course: History of Sexuality Challenges & Goals Threaded discussions and blogs: reflective writing 3.Coursera: Listening to World Music Challenges & Goals Videos, quizzes, peer assessment 4.Questions

3. © 2012 SAS Online Credit Course: GSWS 422 Hist of Sexuality Asynchronous course site Read, discuss, reflect, collaborate, write Focus on student engagement & critical reflection

4. © 2012 SAS  Student engagement  Live class discussions  Student presentations  Alignment with Penn culture Live Synchronous Sessions Online Credit Course: GSWS 422

5. © 2012 SAS GSWS 422 History of Sexuality 1.Advanced seminar, mixed population of students 2.Covers wide span of time; no prerequisites 3.Goals: Enhance student engagement; Explore rich themes between and across class sessions; Develop critical and analytical skills; Provide experience with some primary sources

6. © 2012 SAS Discussion Forums: extension of synchronous class discussions

7. © 2012 SAS Discussion Forums: encourage reflection & critical analysis

8. © 2012 SAS Discussion Forums: replace synchronous discussion

9. © 2012 SAS Discussion Forums: deepen learning

10. © 2012 SAS Student blogs: critical reflection & personal interest

11. © 2012 SAS GSWS 422: Rubric for threaded discussions 1.Students awarded 0-5 points every week 2.Categories Critical analysis of course readings: thoughtful, rigorous Critical, engaged, appropriate responses to classmates’ posts Demonstrated understanding of course content Demonstrated ability to make connections in class themes across readings, sessions, topics Effective use of supporting evidence and citations Quality of writing: clarity, style, grammar, spelling Timeliness and number of posts

12. © 2012 SAS GSWS 422: Rubric for Blogs 1.Students awarded 0-5 points every week 2.Categories Critical analysis of course readings and themes Creative reflection on specific theme or question Demonstrated ability to make connections across readings, sessions, topics, experiences Demonstrated understanding of course content Effective use of supporting evidence and citations Quality of writing: clarity, style, grammar, spelling Timeliness

13. © 2012 SAS GSWS 422: Assessment and Learning Outcomes 1.Critical understanding of key concepts 2.Knowledge of specific historical periods 3.Engagement in critical reflection individually and as a class 4.Ability to make thematic connections across historical periods and regions 5.Student involvement in shaping class according to their interests 6.Extra Credit: deeper understanding of historical analysis through primary source discussion forums

14. © 2012 SAS Listening to World Music

15. © 2012 SAS Free & Open Courses 62 Universities: Stanford, Princeton, Brown, Columbia + 329 courses 3,083,461 enrolled & counting Course Length: 4 weeks to 14 weeks 196 countries participating Courses taught in English, French, Spanish, Chinese and Italian

16. © 2012 SAS Taught by Carol Muller, Professor of Music Over 32,800 students registered in 2012 7 weeks long Course Components & Assignments: 10-15 minute videos with short reflection/question/quiz Discussion forums Almost live videos by graduate assistants Peer-graded assignments Longer quiz and final exam Listening to World Music: Course Overview

17. © 2012 SAS Listening to World Music: Challenges & Questions How to manage large class size? How to introduce expectations to international audience? How to understand low course completion rate? What are intended outcomes according to students versus professor?

18. © 2012 SAS Video Lectures & Embedded Quizzes

19. © 2012 SAS Listening to World Music: Feedback

20. © 2012 SAS Listening to World Music: Scaling the course Challenge: large course size Discussion forums on specific topics Focused Separate threads for technical/process questions Not graded Peer-graded assignment with rubric for student evaluation

21. © 2012 SAS Open Discussion Forums

22. © 2012 SAS Peer Assessments Process

23. © 2012 SAS Peer Graded Assignments: Rubric

24. © 2012 SAS Peer Graded Assignments: Rubric Five requirements graded on scale of 0-2, mechanics & content evaluated Writing style Address the question posed Strength of argument Integrates personal response to music with critical thought Relevance to larger course themes Rubric has maximum of 10 points Students provide written justification for score

25. © 2012 SAS Assessment Feedback

26. © 2012 SAS Rubrics should support learning outcomes & be transparent to audiences Value of models & feedback Determine how students learn & engage in course material when designing assessments Online Assessments: Lessons Learned

27. © 2012 SAS Challenge: Discover other creative ways to develop & integrate asynchronous assessments in credit courses. Questions: Will MOOCs’ designs change to reflect variety of students’ intended outcomes? How will this influence assessment? Online Assessments: Future Challenges/Questions

28. © 2012 SAS Questions?