Lenny Shedletsky & Alice Goodwin Can we improve the quality & quantity of online discussion with rubrics and peer rating? Lenny Shedletsky & Alice Goodwin Second New England Regional Sloan-C Conference on Online Learning, 2011

SETTING THE STAGE: STUDY 1

RESULTS OF STUDY 1 FACE-to-FACE ONLINE AMOUNT OF TALK GREATER LESS (than f2f) LEVEL OF CRITICAL THINKING LOW LEVEL LOWER (than f2f) OTHER (chit chat)

RESULTS STUDY 1 OTHER TYPE OF REPORT (INDIV. Vs. CONSENSUS) TRIGGERING EXPLORATION INTEGRATION RESOLUTION OTHER TYPE OF REPORT (INDIV. Vs. CONSENSUS) > TRIG IN CONSENSUS > EXPLOR. FOR CONSENSUS NO DIFFERENCE --------- > OTHER FOR CONSENSUS TYPE OF ANALYSIS (EX. Vs. ABSTRACT) > TRIG IN ABSTRACT ---------- > OTHER FOR ABSTRACT

RESULTS OF STUDY 1 1. GREATER AMOUNT OF TALK F-2-F BUT MOSTLY CHIT CHAT (LOW LEVEL OF CRITICAL THINKING); 2. ONLINE PRODUCED AN EVEN LOWER LEVEL OF CRITICAL THINKING & LESS TALK THAN F-2-F;

STUDY 2 STUDY 2

RESULTS OF STUDY 2 Summaries from online students received higher grades from 2 professors blind to the study than summaries for F2F students.

STUDY 3 STUDY 3

RESULTS of STUDY 3 The TA’s involvement in discussion had little to no effect on student-to-student interaction and only affected the level of EXPLORATION responses for critical thinking.

Study 4 Study 4

Results of Study 4 Personal Relevance of topics had no effect on student postings or critical thinking. Students’ self reported prediction of how personal relevance would affect their postings was not found in actual behavior observed.

Study 5 STUDY 5

RESULTS OF STUDY 5 WITH RUBRICS AND GRADING POSTS WITHOUT RUBRICS AND GRADING POSTS CRITICAL THINKING INCREASED LOWER INITIAL POSTS MORE LESS EARLIER POSTS EARLIER LATER

STUDY 6: Can we improve the quality & quantity of online discussion with rubrics and peer rating? Since study 5 showed us that rubrics and grading posts increased critical thinking, produced earlier posts and more interaction, we wanted to find out if we could have the students rate one another.

STUDY 6: PROCEDURES & RESULTS SPRING 2008 [NO RUBRICS, NO PEER RATINGS] FALL 2009 [RUBRICS & PEER RATING] CRITICAL THINKING no difference DAY OF INITIAL RESPONSE earlier later FREQUENCY OF RESPONDING

REFERENCES Cases on Online Discussion and Interaction