In search of engagement: Describing the drivers of secondary student science engagement in the United States and Finland Justin Bruner Michigan State University

Outline  Background and Research Questions  Defining Engagement Using Flow Theory  Methods  Experience Sampling Methods  Data and analysis  Between Country Differences  Correlations  T-Test of between country difference  Conclusion

Background  Much of the international comparative policy discussions have focused on two main themes:  Access MDG and EFA CCT programs But what happens when kids get to school?  Achievement PISA, PIRLS, TIMSS But there is more to learning than just a test score…  This work will focus on engagement with an emphasis on science classes

Research Questions  What is driving student engagement in United States and Finnish classrooms?  What are the differences in engagement between United States and Finnish students?  Out of School?  In school?  In Science?

Defining Engagement Using Flow Theory  Flow Theory (Csikszentmihalyi, 1990)  Applies to many domains of life  “Time flies by”  Try to capture the elements that lead to a flow state or the “optimal experience”  Increased feelings of challenge and skills in the task  “Challenge of activity”  “Your skills in the activity” Low to High (4 point) Source: Csikszentmihalyi, 1990

Defining Engagement  Also add Interest  Adapt Flow for school settings (Shernoff et al., 2003) Increased feelings of interest in the task –Concentration and Affect also a part of their model »Did not ask these “Is this activity interesting?” –Not at all to very much (4 point)  Therefore a student is considered engaged when they show increased levels of challenge, skill, and interest  Lab example of our student  How this scales up from student to teacher/classroom to school to country

Experience Sampling Method (ESM)  Capture daily routines and changes on an individual level  Random signaling to avoid “stacking the deck”  Scheduled science response  Make sure at least one science observation per day  Creates a nested data structure of responses within students  Need more power for classrooms, schools, subjects

Table 1 - Descriptive Statistics United States Within country percentFinland Within Country PercentTotal Number Students Number Beeps Out School Beeps178636%195957%3745 In School Beeps316864%148443%4652 Science Beeps56511%57617%1141 Total Engaged Beep66515%51015%1175 Total Engage Out-Sch Beep22513%23612%461 Total Engage In-Sch Beep44014%27318%713 Total Engage Sci Beep6211%12422%186

Table 2 – Correlations with Engagement PartialSemipartPartial2Semipart2SigN US All Challenge ***4422 Skill ***4422 Interest ***4422 Finland All Challenge ***3325 Skill ***3325 Interest ***3325 US Out School Challenge ***1588 Skill ***1588 Interest ***1588 Finland Out School Challenge ***1882 Skill ***1882 Interest ***1882 US In School Challenge ***2834 Skill ***2834 Interest ***2834 Finland In School Challenge ***1437 Skill ***1437 Interest ***1437 US Science Challenge ***498 Skill ***498 Interest ***498 Finland Science Challenge ***558 Skill ***558 Interest ***558

Table 3 – t-test of between country differences US MeanFinland MeanUS SEFinland SET-valueP-value All Engagement Challenge Skill Interest Out School Engagement Challenge *** Skill * Interest In School Engagement *** Challenge <0.001*** Skill Interest In Science Engagement <0.001*** Challenge <0.001*** Skill * Interest <0.001*** Notes: Means and standard errors are based on standardized person means. *=.10 significance, **=.05 significance, and ***=.01 significance.

Findings of analysis  Overall and out of school challenge is the biggest driver of engagement in both countries  In the United States it is same for in school and in science  In Finland it is interest in school and equal in science  In both countries students feel lower levels of challenge and skill out of school and lower levels of interest in school  US students show very low interest in science  Finnish students show higher levels of engagement in school, in science, and for each emotion in science  US students show higher levels outside of school but the difference is only significant for the emotions, not engagement

Future Steps for Analysis  More in depth exploration of other predictors of engagement  Explore and test other possible definitions  More exploration of other correlates of engagement  See what are the other related emotions when students are engaged (or not)  Look at student background and teacher characteristics  Gender, SES, teacher experience  Need to explore the nested structure of the dataset  Requires methodology that is not well known

Future Steps for Project  Collecting more detailed science teacher lesson data  Link it directly with the ESM data  Better understand what is happening in science classroom when students are engaged or not  Create a more explicit partnership with teachers  Show them their students data  Create some PD workshops utilizing that information

Thank you!  I appreciate your time and interest  Please feel free to contact me Justin Bruner Michigan State University

Literature Review  Why science?  Declining number of students are studying science beyond requirements globally (Osborne et al., 2003).  United States fills about 90% of H-1B Visas in STEM professions (Rothwell and Ruiz, 2013)  STEM jobs offer many employment premiums over other professions (Langdon et al., 2011)  What is happening?  Gender differences  Teacher differences  Quality of teaching  Curriculum  Cultural Diffences

Literature Review  Students see science as too difficult and have difficulty seeing it outside the classroom (Lavonen et al., 2008, Lavonen and Laaksonen, 2009, Osborne et al., 2003)  Girls generally experience science in more negative way than boys (Griffith, 2010, Osborne et al., 2003, Uitto et al., 2006, Reigel-Crumb and Moore, 2013)  Science teachers vary in quality  Connected instruction, academic rigor, lively teaching (Cooper 2013)  Cultural Differences (Tsai and Yang, 2011)  Asian students show higher performance but low interest in science. Cultural mediating factors.