Presented at the MCRI Workshop May 30, 2008 Redesigning General Frostburg State University MCRI Workshop May 30, 2008 Primary FSU NCAT Team: Megan E. Bradley, Ph.D. Bill Southerly, Ph.D. Cindy D. Hay, MDE & MS Joseph Hoffman, Ph.D. John Bowman, Ph.D.

Presented at the MCRI Workshop May 30, 2008 Successes Assessment #1: Comparison of comprehensive final exam All instructors blind to exam content

Presented at the MCRI Workshop May 30, 2008 Successes Redesign sections significantly better than traditional sections* Redesign sections the difference between a student earning a C vs. D *A one-way ANOVA of section (6 total: 2 traditional and 4 redesign) on final exam percentage grades was significant, F = 4.710, p =.000, η2 =.090. Comparison of Common Comprehensive Final Exam:

Presented at the MCRI Workshop May 30, 2008 Successes Redesign sections again significantly better than traditional sections by nearly 10%* *A one-way ANOVA of section (2 total: traditional and redesign) on final exam percentage for conceptual questions grades was significant, F = , p =.000, η2 =.102. Comparison of Factually-Based Questions from Common Comprehensive Final Exam:

Presented at the MCRI Workshop May 30, 2008 Successes Redesign sections yet again significantly better than traditional sections* *A one-way ANOVA of section (2 total: traditional and redesign) on final exam percentage for conceptual questions grades was significant, F = 7.452, p =.007, η2 =.030. Comparison of Conceptually-Based Questions from Common Comprehensive Final Exam:

Presented at the MCRI Workshop May 30, 2008 Successes Reason for success on multiple choice part of final exam by redesign students? Final exam scores positively correlated with average scores on Mastery Quizzes r =.523, p =.000

Presented at the MCRI Workshop May 30, 2008 Successes Assessment #2: Comparison of extra credit essay question on final exam All instructors blind to exam content Question: Discuss the psychological phenomenon of prejudice. In your answer, feel free to reflect on causes/explanations, consequences of being a victim of prejudice, and/or ideas to counter prejudice.

Presented at the MCRI Workshop May 30, 2008 Successes Assessment #2: Comparison of extra credit essay question on final exam Scoring rubric emphasized use of psychological concepts No upper limit for scores 1 point per correct use of psychological concepts 1 point per accurate definition of terms 1 point per additional psychological concept

Presented at the MCRI Workshop May 30, 2008 Successes Redesign sections significantly better than traditional sections* Mode Traditional = 0 or 1 Redesign = 1, 2, 2, 3 *A one-way ANOVA of section (6 total: 2 traditional and 4 redesign) on final exam percentage grades was significant, F = 6.787, p =.000, η2 =.153. Comparison of Extra Credit Essay Question:

Presented at the MCRI Workshop May 30, 2008 Successes Reason for success on essay question of final exam by redesign students? Essay score positively correlated with Grade on semester long prejudice project r =.328, p =.000 Grade across all online discussions r =.244, p =.005

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 1)Identify struggling students? Yes No 2)% students meet for help? % 6% 3) communication?Average 60 0 Instructor Questionnaire Weeks 1-2

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 4a) In class behavior Lecture notes 7% 40% Asking questions 10% 14% Large group discussions 13% 4% Small group discussions 35% 6% Self-assessments 4% 7% Psychology-related activities 28% 13% Videos 5% 8% Instructor Questionnaire Weeks 1-2

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 4b) In class behavior Taking quizzes 0% 0% Critical thinking skills 0% 9% Writing about Psychology 0% 0% Oral presentations 0% 0% Taking tests 0% 0% Instructor Questionnaire Weeks 1-2

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 5a) Out of class/online behavior Group discussions 15% 0% Self-assessments 5% 0% Psychology-related activities 15% 40% Videos 0% 0% Instructor Questionnaire Weeks 1-2

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 5b) Out of class/online behavior Visiting web pages 5% 0% Taking quizzes 15% 0% Retaking quizzes (master) 17% 0% Writing about Psychology 20% 0% Critical thinking skills 10% 18% Instructor Questionnaire Weeks 1-2

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 4a) In class behavior Lecture notes 18% 30% Asking questions 13% 8% Large group discussions 10% 8% Small group discussions 23% 8% Self-assessments 8% 5% Psychology-related activities 13% 5% Videos 8% 7% Instructor Questionnaire Full Semester

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 4b) In class behavior Taking quizzes 0% 3% Critical thinking skills 10% 13% Writing about Psychology 0% 4% Oral presentations 0% 3% Taking tests 0% 10% Instructor Questionnaire Full Semester

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 5a) Out of class/online behavior Group discussions 19% 3% Self-assessments 8% 10% Psychology-related activities 8% 3% Videos 8% 0% Instructor Questionnaire Full Semester

Presented at the MCRI Workshop May 30, 2008 Successes RedesignTraditional 5b) Out of class/online behavior Visiting web pages 8% 0% Taking quizzes 15% 0% Retaking quizzes (master) 23% 0% Writing about Psychology 9% 15% Critical thinking skills 10% 35% Instructor Questionnaire Full Semester

Presented at the MCRI Workshop May 30, 2008 Successes ULAs (Undergraduate Learning Assistants) New course and Leadership Certificate Program Accepted by MHEC May 19, 2008 Nominated for campus-wide leadership award 2 students earned prestigious summer opportunities 1 student earned graduate assistantship related to teaching

Presented at the MCRI Workshop May 30, 2008 Successes Source Adjuncts 5 positions Full-time faculty Need 1 less position TOTAL SAVINGS Savings $11,000 $62,699 $73,699 Financial savings

Presented at the MCRI Workshop May 30, 2008 Implementation Issues DWF rate / retention issue Previous average: 15% Fall 2007 rate: 33% Trend: increasing since 2002 Spring 2008 Traditional sections: 4% Redesign sections: 22%

Presented at the MCRI Workshop May 30, 2008 Implementation Issues DWF rate / retention issue Possible reasons for lower than normal DWF rate for traditional sections Final exam, believed by students to be required and count in grade while taking test, was treated as extra credit upon completion Only traditional sections had students earning extra credit for participating in research studies

Presented at the MCRI Workshop May 30, 2008 Implementation Issues DWF rate / retention issue Possible reasons for higher than normal DWF rate for redesign sections Difficult beginning for 2 reasons Access issues to online (graded) content related to financial aid reimbursement Technological issues Students not ready for self-motivation needed for blended part Too many required activities in first half of semester

Presented at the MCRI Workshop May 30, 2008 Implementation Issues DWF rate / retention issue Adjustments Access to online portion course/e-text Publisher Future: Lab fee Computer labs with ULAs Supplemental Instruction Meetings Student Support Group Extension of assignments from 1 week to 2 weeks “Deadline Disorder”

Presented at the MCRI Workshop May 30, 2008 Implementation Issues Financial reimbursement to Department Work study money to cover labs Limited lab space requires 2 ULAs per lab time Recouping gains to make redesign effort worthwhile University or system-wide policy

Presented at the MCRI Workshop May 30, 2008 Implementation Issues Student Motivation/Preparation Readiness for active learning Attitude towards online/blended learning Willingness to check Need for university change to student system (too many s) Preventing/treating Deadline Disorder Other ideas?

Presented at the MCRI Workshop May 30, 2008 Implementation Issues Reactions from colleagues Department: Very supportive Pre-approval before submitting proposal Fall 2007 retreat

Presented at the MCRI Workshop May 30, 2008 Implementation Issues Reactions from colleagues University-wide Hesitant or resistant to Online/blended courses Large sections “Inexperienced” undergraduates helping courses Changing the perceived FSU identity

Presented at the MCRI Workshop May 30, 2008 Implementation Issues Solutions for colleagues Department Continue yearly retreat Seek financial reimbursement for departmental initiatives University-wide Held spring workshop for university Carolyn Jarmon guest speaker Sponsored by Houghton-Mifflin Large positive response Plan for more in fall

Presented at the MCRI Workshop May 30, 2008 Implementation Issues Technology issues Met with Worth representatives Issue for labs Divide each section into 2 “shifts” in 2 classrooms N = 75 per section for fall (replacement model) Largest lab size = 30 Ways to motivate students to bypass lab?? Reward system that is easy to implement

Presented at the MCRI Workshop May 30, 2008 Implementation Issues Retention and student support Removed “blended” label Working with student support services to implement university-wide policies Tie-in Learning communities Orientations (redesign) General Psychology

Presented at the MCRI Workshop May 30, 2008 Implementation Issues What would we have done differently? Reduce number of activities/deadlines Get financial policy finalized before implementation System-wide policy Deadlines needed for both the redesign team and the administration for financial reimbursement