EXPLORING CONNECTIONS BETWEEN ASSESSMENT, MOTIVATION, AND METACOGNITION Matthew A. Ludwig, NSF Doctoral Fellow Heather L. Petcovic, Assistant Professor Herb Fynewever, Associate Professor* This work is supported by the National Science Foundation under Grant No. 0733590. The Michigan Space Grant Consortium, and Western Michigan University. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Introduction So What? Assessment, Motivation, and Metacognition AfL Operationally? A Research Project Conclusions http://coe.sdsu.edu/eet/Articles/metacognition/student17.jpg

So What? 0) Students are always … uhh… ?metacognificating? Assessment  Student Motivation  Metacognition  Learning AfL  Motivation  Learning Be patient grasshoppa! Find a Giant 0) The key question is can we induce positive metacognition in our students An extensive review and synthesis of the literature on achievement motivation and assessment learning yields two generalizations Theoretical and empirical connections A special type of assessment (AfL) has shown dramatic and positive impacts on both motivation and learning in laboratory settings However, experiments in real classrooms present a significant gap in the resch literature. Changing student motivation, particularly with students who have learned to thrive in existing classroom structures takes time. BE PATIENT! Finally, my experiences here at GSA and beyond indicate a significant interest in the field of practical student motivation. However, it also seems that many are developing proprietary instruments to measure student motivation in their classrooms. http://www.usask.ca/education/coursework/802papers/Adkins/TWOFROGS.GIF

So What? 0) Students are always … uhh… ?metacognificating? Assessment  Student Motivation  Metacognition  Learning AfL  Motivation  Learning Be patient grasshoppa! Find a Giant

Assessment for Learning and Self-Regulated Learning Your students are metacognating! You can help to induce adaptive metacognition as opposed to maladaptive metacognition. Setting the stage may induce performance vs Mastery orientation (from Butler & Winne 1995 used with permission from SAGE publications)

Assessment for Learning and Self-Regulated Learning Start Your students are metacognating! You can held to induce adaptive metacognition as opposed to maladaptive metacognition. Task Presentation results ing perceived cues that moderate Student motivation orientation Setting the stage may induce performance vs Mastery orientation (from Butler & Winne 1995 used with permission from SAGE publications)

Assessment for Learning and Self-Regulated Learning Once a student receives an task they immediately disassemble it into familiar cues. Some of the more powerful cues are related to evaluation (i.e. how can I get an acceptable grade) (from Butler & Winne 1995 used with permission from SAGE publications)

Assessment for Learning and Self-Regulated Learning These cues are a significant moderator for how the student will eventual behave to accomplish (or not) the task. (from Butler & Winne 1995 used with permission from SAGE publications)

Assessment for Learning and Self-Regulated Learning Metacognition is an ever present process embedded in the task (from Butler & Winne 1995 used with permission from SAGE publications)

Assessment for Learning and Self-Regulated Learning External Feedback is both the beginning and the end. IT provides the basis for: future task cues Or Second attempts at the existing task. (from Butler & Winne 1995 used with permission from SAGE publications)

Assessment for Learning and Self-Regulated Learning And the process starts again. (from Butler & Winne 1995 used with permission from SAGE publications)

What is Assessment for Learning? Three tenets of AfL Three players of AfL (Black and Wiliam, 2009)

Can student motivation be improved via assessment reform? A Research Study Purpose: To explore the connections between assessment and engagement in pre-service elementary education students and their instructor. Given my background in geophysics… Driving Question: Can student motivation be improved via assessment reform?

Methodology Research Subjects (n=~75/semester) The students and the instructor in an intro-level earth systems science course for pre-service elementary teachers at a regional, mid-western, public, research university Design Non-equivalent pretest posttest design Instrumentation Student Surveys (SCEQ, KS) Semi-structured Interviews (students and instructor) Classroom Observations (MATO, VFO) Method details Image used with permission from artist Sara Beise & Time for Kids

Knowledge Survey Results What students should know Section dividers a.m.  Students  p.m. eve. KS Items (1 – 148)  I have no clue This is familiar I know the answer Oops! (Nuhfer, 1996) (Wirth & Perkins, 2006)

Knowledge Survey Results What students should know Section dividers a.m.  Students  p.m. eve. KS Items (1 – 148)  I have no clue GREEN IS GOOD! This is familiar I know the answer Oops!

Knowledge Survey Results Day 1

Knowledge Survey Results Day 1 Half-way through Unit 1

Knowledge Survey Results Day 1 Half-way through Unit 1 Pre-exam #1 Three different sections have unique issues

Knowledge Survey Results Day 1 Half-way through Unit 1 Pre-exam #1 ✓End of Semester✓

Student Classroom Engagement Questionnaire 23 Likert-type items 1 = not at all characteristic of me  5 = very characteristic of me Maladaptive behaviors/beliefes  Adaptive behaviors/beliefs 4 sub-scales (from exploratory factor analysis) Performance (3 items, α=0.8421) Emotional (5 items, α=0.8781) Participation/Interaction (6 items, α=0.8174) Skills (9 items, α=0.9168) Handelsman’s Alpha values were between 0.76 and 0.82 (Handelsman et al, 2005)

Example SCEQ subscale Items Skills engagement subscale 9. ____ Taking good notes in class 23. ____ Listening carefully in class Emotional engagement subscale 7. _____ Thinking about the course between class meetings 21. ____ Finding ways to make the course material relevant to my life Participation/Interaction engagement subscale 2. _____ Participating actively in small group discussions 3. _____ Asking questions when I don’t understand the instructor Performance engagement subscale 15. _____ Getting a good grade 16. _____ Doing well on the tests Skills represents student engagement through practicing skills Emotional represents student engagement through emotional involvement with the class material Participation/Interaction involved classroom participation and interactions with the instructor and other students Performance represents student engagement through levels of performance in the class.

SCEQ Results First, the Good News n=147 Describe the plots Y-participation interaction X-emotion Color- Emotion skills Size - performance No significant changes within semesters or across semesters was noted.

SCEQ Results The Other news Pre Post Skills Emotional P & I Perf. Sadly, no significant pre-post was found Motivation appears more state-like that trait-like in the short term Skills Emotional P & I Perf.

SCEQ Results Describe the plots Y-participation interaction X-emotion Color- Emotion skills Size – performance

SCEQ Results Control Groups Treatment Group Spring 08 (n=20) Fall 08 (n=63) Spring 09 (n=56) PRE POST Describe the plots Y-participation interaction X-emotion Color- Emotion skills Size - performance No significant changes within semesters or across semesters was noted. Control Groups Treatment Group

Conclusions Generally Students’ value KS’s The instructor was responsive to (and surprised by) gaps in the KS’s Students report moderately high engagement Over the short term Student Engagement appears to be more trait-like than state-like Future Work Expand the timeframe Explore other correlations (achievement…) Compare with other surveys (SMQ, MSLQ, PALS, etc…?) Explore connections between SCEQ and Overt Engagement Perhaps there would be a more significant change if the post-survey was completed after the first exam.

So What? 0) Students are always … uhh… ?metacognificating? Assessment  Student Motivation  Metacognition  Learning AfL  Motivation  Learning Be patient grasshoppa! Find a Giant