LEARNERS & LEARNING ENVIRONMENTS: Comparison of cognition level between lecture-based classroom and experiential study abroad assessment Abigail J. Lynch and William W. Taylor Michigan State University, Department of Fisheries and Wildlife, Center for Systems Integration and Sustainability 30 April 2013 FAST Fellowship Symposium

Research need Research approach Preliminary results Implications Fostering learning environments which result in learning. What is our objective as instructors? 2 Evaluating effectiveness of learning environments to ensure learning. What is our objective as scholars? Research needResearch approachPreliminary resultsImplications

Research need Research approach Preliminary results Implications Study abroad: not available on campus  “My study abroad gave me tools and experiences that studying on campus could never provide. In fact, without studying abroad, I would never have known how valuable the experiences were.” 3 Research need

Research need Research approach Preliminary results Implications Study abroad: experiential learning  “Most of the time they not only told us about their work and knowledge, but took us out into the field to show us too. This alone made this program a unique educational experience, because rather than just talking about what we were learning, we actually went to see where policy making or science and environmental restoration work takes place.” 4 Research need

Research need Research approach Preliminary results Implications Study abroad: experience with impact  “This learning did not happen in a classroom and there was barely any lecture time…These people had devoted their lives to fish and water preservation, so at no point were they teaching us. They were actually physically showing us the things they were most passionate about. This is a type of learning that makes an impact on students.” 5 Research need

Research need Research approach Preliminary results Implications Learning Environments  Lecture-based  Experiential Traditional ClassroomStudy Abroad zaidlearn.blogspot.com How can you compare? Research need

Research need Research approach Preliminary results Implications Comparing learning environments  Foreign language competency  Segalowitz et al. (2004): U.S. classroom vs. study abroad in Spain Study abroad students: significantly greater gains in oral proficiency, oral fluency but not grammar or pronunciation. Cognitive thresholds for second language learning readiness (regardless of location) 7 Research need  Foreign language competency  Segalowitz et al. (2004): U.S. classroom vs. study abroad in Spain  Freed et al. (2004): U.S. classroom vs. summer immersion vs. study abroad in France Immersion students: significant gains in oral proficiency Study abroad students: significant gains in speech fluidity Classroom students: no statistically significant gains Gains related more to hrs/week speaking and writing French than anything else  Foreign language competency  Segalowitz et al. (2004): U.S. classroom vs. study abroad in Spain  Freed et al. (2004): U.S. classroom vs. summer immersion vs. study abroad  Other subject areas?  Limited opportunities for paired design  Foreign language competency

Research need Research approach Preliminary results Implications Paired comparison: FW 481 Global issues in Fisheries and Wildlife  Classroom: Spring 2010, Spring 2012  Study abroad: Summer 2011, Summer 2013  Same learning objectives  Same essay assignments  Similar number of students  Similar demographics 8 Research approach  Classroom: Spring 2010, Spring 2012  Study abroad: Summer 2011, Summer 2013

Research need Research approach Preliminary results Implications To compare cognition level between lecture-based students and experiential study-abroad students Teaching-as-Research Objective 9 Research approach

Research need Research approach Preliminary results Implications Cognition level  Bloom’s Taxonomy (1956): hierarchical classification of thinking skills  Most assessment is at lower levels of the taxonomy  Students remember more when they handle a topic at the higher levels of the taxonomy Garavalia, Hummel, Wiley, & Huitt (1999) 10 Research approach edweb.sdsu.edu

Research need Research approach Preliminary results Implications “Blooming”  Scored essay assignments using Bloom’s Taxonomy (1956)  Coded for highest competence at the sentence, paragraph, and paper level 1: knowledge 2: comprehension 3: application 4: analysis 5: synthesis and evaluation  Standardized by number of sentences, paragraphs  Weighted by each level 0.5: sentence 0.3: paragraph 0.2: paper 11 Research approach

Research need Research approach Preliminary results Implications “Blooming” example 12 Research approach cognitive levelsentencesparagraphspaper knowledge 72 ⨯ 1 = 72 comprehension 38 ⨯ 2 = 76 application 22 ⨯ 3 = 66 analysis 15 ⨯ 4 = 604 ⨯ 4 = 16 synthesis and evaluation 8 ⨯ 5 = 403 ⨯ 5 = 151 ⨯ 5 = 5 sum  Scored essay assignments using Bloom’s Taxonomy (1956)  Coded for highest competence at the sentence, paragraph, and paper level  Standardized by number of sentences, paragraphs  Weighted by each level full score

Research need Research approach Preliminary results Implications Is there a difference between cognition level between: Classroom and study abroad students? Students the beginning and end of the course? Students with stated preference for active over lecture-based pedagogy? Research Questions 13 Research approach

Research need Research approach Preliminary results Implications Learning environment effects  Could learning environment affect demonstrated cognition level?  T-test for learning environment effects: Is there a difference between demonstrated cognition level in the final essay between study abroad and classroom students? 14 Preliminary results t = df = p-value = study abroadclassroom study abroad essay mean significantly higher cognition level than classroom essay mean

Research need Research approach Preliminary results Implications Student effects  Could the course influence demonstrated cognition level? Self-selection?  T-test for prior effects: Is there a difference in the cognition level between the beginning and end of the study abroad course? 15 Preliminary results t = df = 7 p-value = initialfinal final essay mean significantly higher cognition level than initial essay mean

Research need Research approach Preliminary results Implications Student effects  Could the course influence demonstrated cognition level? Self-selection?  Could pedagogical preference affect demonstrated cognition level?  T-test for pedagogical effects: Is there a difference in demonstrated cognition level between students who prefer active vs. lecture-based pedagogy? 16 Preliminary results

Research need Research approach Preliminary results Implications Lecture vs. active learning preferences  Conducted a unit of the 2012 classroom course (three class periods) using different pedagogical styles  Lecture  Active learning jig-saw and problem-based learning  Distributed survey on pedagogical preferences  Likert scale  Open ended questions 17 Preliminary results

Research need Research approach Preliminary results Implications Lecture vs. active learning preferences  “Greater trust in [lecture] information because it comes from the TA/professor instead of classmates.”  “I like small group discussions because you hear multiple viewpoints and interpretations to better comprehend the material” LectureActive learning 18 Preliminary results

Research need Research approach Preliminary results Implications Student effects  Could the course influence demonstrated cognition level? Self-selection?  Could pedagogical preference affect demonstrated cognition level?  Test for pedagogical effects: Is there a difference in demonstrated cognition level between students who prefer active vs. lecture-based pedagogy? 19 Preliminary results t = df = p-value = activelecture preference for active learning mean NOT significantly higher cognition level than preference for lecture mean

Research need Research approach Preliminary results Implications Implications  Evidence that learner preference does not impact demonstrated cognition level  Small sample size  Evidence that experiential study abroad learning environments can promote higher order thinking  Final assignment mean cognition score: Significantly higher than initial assignment Significantly higher than classroom final assignment  Need to compare initial assignments from both learning environments to isolate effects 20 Implications

Research need Research approach Preliminary results Implications Next steps  Add 2010 and 2013 students, 2012 initial essay  Add second reader to ensure reproducibility  Consider regression analyses?  Consider publication?  Any recommendations?  Journal of Natural Resources and Life Sciences Education? 21 Implications

THANK YOU! Abigail J. Lynch and William W. Taylor Michigan State University, Department of Fisheries and Wildlife, Center for Systems Integration and Sustainability 30 April 2013 FAST Fellowship Symposium ACKNOWLEDGEMENTS FW 481 students FAST Fellowship Steering Committee FAST Fellows

Research need Research approach Preliminary results Implications Pearson Correlation Coefficient (r)  measures the strength of the linear relationship between the paired x and y values in a sample.  COV xy = covariance between x and y  σ x, σ y = standard deviation of x and y  r = +/-.50 are considered strong linear relationship  r = +/-.30 are considered moderate linear relationship  r = +/-.10 are considered weak linear relationship Cohen (1988) 23

Research need Research approach Preliminary results Implications Methods: “Blooming”  “Bloom” course assignments  Two independent readers  Two readings each  Each sentence coded 1: knowledge and comprehension 2: application and analysis 3: synthesis and evaluation  Assignment “scored” by edweb.sdsu.edu sum of sentences codes total number of sentences 24 Research approach

Research need Research approach Preliminary results Implications  Compares the mean values between independent groups with rank-ordered data  U = Mann-Whitney statistic  N 1 = number of samples in group 1  N 2 = number of samples in group 2  R 1 = rank sum for samples in group 1 Mann-Whitney U Test (U) 25 Research approach