Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Full Implementation Results for General Frostburg State University MCRI Workshop May 29, 2009 Primary FSU NCAT Team: Megan E. Bradley, Ph.D. Bill Southerly, Ph.D. Cindy D. Hay, MDE & MS Joseph Hoffman, Ph.D. John Bowman, Ph.D.

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Psyc150: General Psychology Frostburg State University in Frostburg, Maryland Annual enrollment: About 900 Mostly traditional students and 1 st year students Required course for Psychology Majors and 5 other majors Replacement Model

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Impact on Student Learning General Psychology Students

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Comparisons N compared to prior # ADMIT GPA*FSU GPA* Traditional16% smaller Redesign100% larger *Significantly different; p =.005 (eta 2 =.027), p =.000 (eta 2 =.075)

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Results: 50 Question Common Comprehensive Final Exam* Mean Test Scores *A one-way ANOVA of section on final exam percentage grades was significant, F = , p =.000, eta 2 =.090. Also significant with GPA as covariate: F = , p =.000, eta 2 =.11. *Instructors blind to exam content

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Results: 50 Question Common Comprehensive Final Exam SectionAdmit GPAFSU GPAMean % on Final Exam Section 1 Traditional % Section 2 Traditional % Section 3 Redesign % Section 4 Redesign %

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Results: 50 Question Common Comprehensive Final Exam Final exam scores positively correlated with average scores on MQs r =.523, p =.000

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Results: Extra Credit Final Exam Essay Question on Prejudice Assessment #2: Comparison of extra credit essay question on final exam All instructors blind to exam content Question: Discuss the psychological phenomenon of prejudice. In your answer, feel free to reflect on causes/explanations, consequences of being a victim of prejudice, and/or ideas to counter prejudice.

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Assessment #2: Comparison of extra credit essay question on final exam Scoring rubric emphasized use of psychological concepts No upper limit for scores 1 point per correct use of psychological concepts 1 point per accurate definition of terms 1 point per additional psychological concept Pilot Results: Extra Credit Final Exam Essay Question on Prejudice

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Mode Traditional = 0 or 1 Redesign = 1, 2, 2, 3 *A one-way ANOVA of section on essay grades was significant, F = 6.787, p =.000, eta2 =.153. Pilot Results: Extra Credit Final Exam Essay Question on Prejudice Mean Test Scores

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Essay score positively correlated with Grade on semester long prejudice project r =.328, p =.000 Grade across all online discussions r =.244, p =.005 Pilot Results: Extra Credit Final Exam Essay Question on Prejudice

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Redesign Issues & Implementation Solutions Increase use of more learning principles Spacing effect: 3 unit exams Deeper learning: Reduced overall coverage “Deadline Disorder” Reduced to: MQs, Discussions, Prejudice Activity 2 weeks to complete

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Redesign Issues & Implementation Solutions Students not ready for blended design “Freshmen don’t do optional” + “Students need structure” Required computer lab 1x/wk Students needed more in-class assistance Updated our online instructor’s manual to include brief direct instruction

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Pilot Redesign Issues & Implementation Solutions Need for more campus-wide support Held 3 workshops on redesign Implemented student support services programs Tutoring, Supplemental Instruction ORIE + Learning Communities Wellness initiative

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Full Implementation Results: 43 Common Questions (3 exams)* *A one-way ANOVA of section (3 total) on common question percentage was significant, F = , p =.000, eta 2 =.825. Mean Test Scores *Instructors blind to exam content

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Factually-Based versus Conceptually-Based Questions *Factual: F = , p =.000, eta 2 =.771 Conceptual: F = , p =.000, eta 2 =.813 Mean Test Scores

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Mode Pre-Essay = 0 or 1 Post-Essay = 0,1, 2, 3, 4 *A one-way repeated measures ANOVA on essay grades was significant, F = , p =.000, eta2 =.420. Full Implementation Results: Pre vs. Post Prejudice Essay Mean Test Scores

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Essay score positively correlated with Grade on semester long prejudice project r =.119, p =.030 Full Implementation Results: Pre vs. Post Prejudice Essay

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley DWF Rate Previous average: 12.5% 18% prior to pilot Pilot Semester Traditional sections: 4% Redesign sections: 22% Full Implementation - Fall 12.8%

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Impact on Student Learning ULAs

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley ULA Benefits “Field Experience” course for top students Leadership in Psychology Certificate Program Interning as a ULA Research experience included

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley ULA Benefits Future opportunities Graduate School Teaching or Research Assistantship Prestigious Internships Most recent: Leadership Institute

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Cost Savings Plan: $89 to $32 Reality: $89 to $26 Reason: Only 50% of ULAs getting paid Biggest cost effectiveness: Tripling capacity in class Staffing

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Cost Savings Use of savings Realized: Communication Response System Implementation (“clickers”) Laptop lab Future sustainability Support to Chair?

Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley Special Thanks Our administrative team…for the support Joseph Hoffman & John Bowman USM…for seizing an opportunity Don Spicer & Nancy Shapiro Board of Regents NCAT…for the vision, knowledge & skills Carol Twigg & Carolyn Jarmon