1. Lenny Shedletsky & Alice Goodwin
Can we improve the quality & quantity of online discussion with rubrics and peer rating?
Lenny Shedletsky & Alice Goodwin
Second New England Regional Sloan-C Conference on Online Learning, 2011

2. SETTING THE STAGE: STUDY 1

3. RESULTS OF STUDY 1 FACE-to-FACE ONLINE AMOUNT OF TALK GREATER
LESS (than f2f)
LEVEL OF CRITICAL THINKING
LOW LEVEL
LOWER (than f2f)
OTHER (chit chat)

4. RESULTS STUDY 1 OTHER TYPE OF REPORT (INDIV. Vs. CONSENSUS)
TRIGGERING
EXPLORATION
INTEGRATION
RESOLUTION
OTHER
TYPE OF REPORT (INDIV. Vs. CONSENSUS)
> TRIG IN CONSENSUS
> EXPLOR. FOR CONSENSUS
NO DIFFERENCE
> OTHER FOR CONSENSUS
TYPE OF ANALYSIS (EX. Vs. ABSTRACT)
> TRIG IN ABSTRACT
> OTHER FOR ABSTRACT

5. RESULTS OF STUDY 1
1. GREATER AMOUNT OF TALK F-2-F BUT MOSTLY CHIT CHAT (LOW LEVEL OF CRITICAL THINKING);
2. ONLINE PRODUCED AN EVEN LOWER LEVEL OF CRITICAL THINKING & LESS TALK THAN F-2-F;

6. STUDY 2
STUDY 2

7. RESULTS OF STUDY 2
Summaries from online students received higher grades from 2 professors blind to the study than summaries for F2F students.

8. STUDY 3
STUDY 3

9. RESULTS of STUDY 3
The TA’s involvement in discussion had little to no effect on student-to-student interaction and only affected the level of EXPLORATION responses for critical thinking.

10. Study 4
Study 4

11. Results of Study 4
Personal Relevance of topics had no effect on student postings or critical thinking. Students’ self reported prediction of how personal relevance would affect their postings was not found in actual behavior observed.

12. Study 5
STUDY 5

13. RESULTS OF STUDY 5 WITH RUBRICS AND GRADING POSTS
WITHOUT RUBRICS AND GRADING POSTS
CRITICAL THINKING
INCREASED
LOWER
INITIAL POSTS
MORE
LESS
EARLIER POSTS
EARLIER
LATER

14. STUDY 6:
Can we improve the quality & quantity of online discussion with rubrics and peer rating?
Since study 5 showed us that rubrics and grading posts increased critical thinking, produced earlier posts and more interaction, we wanted to find out if we could have the students rate one another.

15. Study 6
Moreover, since grading student posts is labor intensive, it seeks to discover if peer ratings will do a good job of improving the discussion board posts for quality and quantity.

16. Can Rubrics & Peer Rating Produce
Earlier Posts
Quantity of Posts
Quality of Posts
Higher Levels of Critical Thinking

17. STUDY 6: PROCEDURES & RESULTS
SPRING 2008 [NO RUBRICS, NO PEER RATINGS]
FALL 2009 [RUBRICS & PEER RATING]
CRITICAL THINKING
no difference
DAY OF INITIAL RESPONSE
earlier
later
FREQUENCY OF RESPONDING

18. Recap The students rated their peers
They were not graded for level of critical thinking or day of the week for initial posts and responses
They were not asked to their ratings and rationale to the teacher

19. Looking Ahead to Study 7
What if students were required to their ratings and rationale to the teacher?
Would the quantity and quality of discussion board posts improve?
Would class scores improve as a whole for levels of critical thinking?

20. REFERENCES
Cases on Online Discussion and Interaction