1. Effect of Implementation of JTF Engagement and Feedback Pedagogy on Change of Faculty Beliefs and Student Performance
NSF Grant #
Steve Krause (ASU), Terry Alford (ASU), Shariar Anwar (ASU), Adam Carberry (ASU), Candace Chan (ASU), Casey Ankeny (ASU), Bethany Smith (ASU), Brady Gibbons (OSU), Milo Koretsky (OSU),
Bill Brooks (OSU), Deb Gilbuena (OSU), Cindy Waters (NCAT), Joe Stuart (OIT), Peggie Weeks (Ext. Evaluator), Eugene Judson (Internal Evaluator), Karl Smith (Advisory Bd.) and John Baglin (Advis Bd.)
Just-in-Time-Teaching with Interactive
Frequent Formative Feedback (JTF)
Objective
Results
Student Attitude, Achievement, and Persistence with JTF
JTF Student Course Persistence - 2nd week to final
To test the effectiveness of JTF pedagogy on changing faculty beliefs and classroom practice and on changing associated student performance across diverse settings
97% Persistence (n = 227/235) Fa Four Institutions, five courses
95% Persistence (n = 311/328) Sp Four Institutions, five courses
Roger’s Model – JTF Innovation Change
Knowledge, Persuasion, and Evaluation of JTF -
9 out 9 faculty agreed that, “Being involved in the JTF project has
motivated me to change my classroom practice.”
“Increased awareness of confusing or poorly explained concepts.”
“Students are made part of the learning team.”
“I heard from several students directly that using muddiest points
helped them engage in the class further than they normally would.”
Implementation and Adoption of JTF -
7 out of 8 faculty agreed that “their views on teaching had changed
somewhat or changed significantly”
8 out of 8 faculty agreed that “their classroom practice had changed
Background
Class Persistence Student Impact Value Survey
JTF is a web-enabled, engagement and feedback pedagogy that uses muddiest points for two-way formative feedback and classroom engagement activities with online resources
Shift to Student-Centered Classroom Practice
Student Achievement with JTF Across 3 Universities and 4 Classes
Cindy Waters; North Carolina A&T – Intro Materials Shariar Anwar; Arizona State – Math. Methods Materials
Coburn’s Model – JTF Scaling of Innovation
Depth: Change of Belief toward Student-Centered Learning-
What is your role in the classroom today compared to before JTF?
“More of a coach than a ‘lecturer’.”
“More of a guide now.”
“More as a coach and encourage and guide…”
“…I am a guide and they must take on the learning.”
“...Now I feel like, in addition to those, I am asking questions of EACH student during lecture. That’s not possible without what we are doing.”
Sustainability: Adaptability of JTF in Many Classroom Settings -
Six faculty rated, on a scale from 1-10:
Frequency of contextualization for class content rated 9.3
Effectiveness of engagement strategies for learning rated 8.3
Adopted by multiple faculty in diverse settings at 4 institutions
Spread: Creation of Community of Practice (CoP) -
All faculty collected Muddiest Points and adjusted instruction
5 to 8 faculty participated in monthly web meetings over 2 years
6 faculty participated in two ASEE JTF workshops
2 faculty created 2 new survey tools used in the CoP
Shift of Ownership: External Facilitators to Faculty Practitioners -
All participants use web-enabled, two-way formative feedback
All use web-based approaches to creating classroom materials
Feedback pedagogy use increased from 3.9 pre-JTF to 8.3 post-JTF
Pre-JTF: Mean = 68.0%; SD = 15.5%
Post-JTF: Mean = 76.6%; SD = 19.5%
Pre-JTF: Mean = 71.8%; SD = 17.9%
Post-JTF: Mean = 81.2%; SD = 7.5%
Traditional
Shifted Classroom
Roger’s Innovation Change Model[1]
Characteristics of Change within Individuals:
Knowledge or Awareness - Individual exposed to innovation
Persuasion or Interest - Individual seeks more information
Evaluation and Decision - Individual adopts or rejects innovation
Implementation or Trial - Individual puts innovation to use
Confirmation or Adoption - Innovation is continued/sustained
Joe Stuart; Oregon Institute of Technology – Intro Materials Stephen Krause; Arizona State – Intro Materials
Pre-JTF: Mean = 86.8%; SD = 7.8%
Post-JTF: Mean = 92.0%; SD = 6.8%
'09: Mn = 66.3%; SD = 13.8%
'11: Mn = 71.1%; SD = 17.9%
'13: Mn = 77.0%; SD = 13.0%
Coburn’s Scaling of Innovation Model[2]
Characteristics of Effective and Sustainable Scaling of Innovation:
Depth – Faculty change beliefs at a deep level
Sustainability – Innovation is adaptable to many settings
Spread – Beliefs and norms change across an organization
Shift of Ownership – Shifts from facilitator to faculty practitioner
Conclusions
Faculty shifted beliefs and adopted JTF pedagogy in classroom practice
Faculty assumed ownership of JTF innovations in a community of practice
These factors enhanced student attitude, achievement, and persistence
[1] Rogers, E.M. (1962). Diffusion of innovations. Glencoe: Free Press.
[2] Coburn, C.E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.
Project Outcomes
Collaborating Institutions
Implement web-based JTF instructional resources
Build Community of Practice to implement JTF pedagogy.
Investigate how instructors use feedback to adjust instruction
Assess effect of JTF on student attitude, learning & retention
Find potential of JTF strategies for other engineering domains
Disseminate JTF strategies, findings, and products
Dissemination of JTF Methods and Publications
Granta, FIE (3 papers), and ASEE (3 posters, 2 workshops, 10 papers)
Callister 9th ed. Materials Text – MP tutorial videos; e Vocab.; MPs