1. Investigation of Inverted and Active Pedagogies in STEM Disciplines: A Preliminary Report
Reza O. Abbasian
Texas Lutheran University, Department of Mathematics, Computer Science, and Information Systems
Michael Czuchry
Texas Lutheran University, Department of Psychology
John T. Sieben
Texas Lutheran University, Department of Mathematics, Computer Science, and Information Systems

2. Overview Description of the project and grant Goals of the project
Methodology and data gathering
Preliminary results
Future work
Acknowledgements
References

3. Description of the project
NSF, IUSE grant
Three years, $290K
Study the effectiveness of Inverted classrooms and active learning in STEM disciplines
Inverted versus Traditional courses in Statistics, Biology, Physics and Chemistry
Faculty involvement:
Six (plus a few unpaid volunteers) faculty managing the grant and teaching the inverted classes, one data analyst and one archivist/web site manager
Courses: STAT , BIOL , CHEM , PHYS

4. Goals and Objectives of the project The overall goal of the project is to develop, implement and evaluate the impact of the Inverted and Active Learning Pedagogies ( IALP) for Student Success and Retention in STEM disciplines
Specific goals:
1. Develop IALP courses which optimize content coverage and maximize higher- order learning
Summer workshop(s) to create videos
Summer workshop/presentations for best practices on effective inverted classroom teaching
Summer 2017 : Develop data collection instruments and the management tools to analyze data
Summer 2017: develop pre- and post-tests for each course
Fall 2017 and 2018: Discussion sessions each semester on sharing experiences
December 2017 and 2018, May 2018 and 2019 : Collect data on the results of pre- /post-tests for Inverted and traditional courses and conduct faculty and student surveys.

5. 2. Determine the impact of IALP on the success and retention of STEM majors
Analyze the data at the end of each semester and the cumulative results
Analyze registrar’s data by academic level and major
Determine the impact of IALP courses on the retention rate
Analyze the student surveys to determine the impact of ILAP course satisfaction

6. 3. Disseminate to the national STEM community the projects findings and course material
Disseminate faculty work through presentations and publications
Create an archive of IALP material ( videos, projects, etc.)
Serve as mentors to other universities to replicate and implement the strategies and materials

7. Methodology and Data Gathering (Fall 2017 and Spring 2018)
We examined faculty teaching traditional or inverted/hybrid classes in Statistics, Biology (Fall only), and Chemistry
We used Bloom’s Taxonomy to create questions at multiple cognitive levels
Students in all classes were administered the exams at the beginning and end of each semester
Grades and withdrawals over the course of the semester were also examined

8. Approach for Question Generation
Cognitive Dimensions for Multiple-Choice Questions
18 Q
Knowing
6 Q
Doing
6 Q
Interpreting
= 30 Questions total
Remember
Understand
Apply
Analyze
Evaluate
Cognitive Dimension
Knowledge
Dimension
1. Remember
2. Understand
Apply
Analyze
Evaluate
Create
A. Factual
B. Conceptual
C. Procedural
D. Metacognitive 3 3 2 1 1 3 2 1 1 3 3 3 2 1 1

9. Preliminary Results (Pre- Post-Tests)
2X3X2 Mixed MANOVAs were conducted examining pre- post- tests
Condition (Inverted/Hybrid vs. Traditional)
Discipline (Statistics vs. Biology* vs. Chemistry)
Time (Beginning of Semester vs. End of Semester)
*Fall 2017 only
Dependent variables (in separate MANOVAs or ANOVAs) included:
Total score
Factual, Conceptual, and Procedural Knowledge
Remembering and Understanding
Application
Analyzing and Evaluating

10. Sample Sizes (Fall 2017) Discipline Statistics Biology Chemistry
Totals
Inverted/Hybrid
N = 20
N = 28
N = 24
N = 72
Class Type
Traditional
N = 25
N = 19
N = 28
N = 72
Totals
N = 45
N = 47
N = 52
N = 144

11. Representative Findings (Fall 2017)

12. Figure 1. Means and Standard Errors for the Percent Correct at the Beginning and End of the Semester. * * *
% Correct
Notes. *p < .05; Fall 2017

13. Figure 2. Means and Standard Errors for the Percent Correct for Procedural Knowledge at the
Beginning and End of the Semester in Statistics, Biology, and Chemistry Classes. * * *
% Correct Procedural Knowledge
Notes. *p < .05; Fall 2017; Statistically significant interaction

14. Figure 3. Means and Standard Errors for the Percent Correct for Procedural Knowledge at the
Beginning and End of the Semester in Inverted/Hybrid and Traditional Classes. * * *
% Correct Procedural Knowledge
Notes. *p < .05; Fall 2017; Statistically significant interaction

15. Summary of Findings (Fall 2017)
Students did better over time demonstrating important learning
Students did better in Statistics classes over time compared to Biology and Chemistry classes
Students in traditional classes did better than inverted/hybrid classes on procedural knowledge over time as well as understanding*
* Results not depicted in prior Figures

16. Representative Findings (Spring 2018)

17. Sample Sizes (Spring 2018) Discipline Statistics Chemistry Totals
Inverted/Hybrid
N = 35
N = 28
N = 63
Class Type
Traditional
N = 26
N = 13
N = 39
Totals
N = 61
N = 41
N = 102

18. Figure 4. Means and Standard Errors for the Percent Correct at the Beginning and End of the Semester. * * *
% Correct
Notes. *p < .05; Spring 2018

19. Figure 5. Means and Standard Errors for the Percent Correct for Procedural Knowledge at the
Beginning and End of the Semester in Statistics, Biology, and Chemistry Classes. * *
% Correct Procedural Knowledge
Notes. *p < .05; Spring 2018; Statistically significant interaction

20. Figure 6. Means and Standard Errors for the Percent Correct for Procedural Knowledge at the
Beginning and End of the Semester in Inverted/Hybrid and Traditional Classes. * *
% Correct Procedural Knowledge
Notes. *p < .05; Spring 2018; Interaction p = .158

21. Summary of Findings (Spring 2018)
Again, students did better over time demonstrating important learning
Again, students did better in Statistics classes over time compared to Chemistry classes
Students in traditional classes did better than inverted/hybrid classes on evaluating information over time*
* Results not depicted in prior Figures

22. Preliminary Results (Final Grades and Withdrawals)
2X3 ANOVAs were conducted examining final grades and withdrawals
Condition (Inverted/Hybrid vs. Traditional)
Discipline (Statistics vs. Biology* vs. Chemistry)
*Fall 2017 only
Dependent variables (in separate ANOVAs) included:
Final grade in courses
Withdrawals from course (Fall 2017 only)

23. Sample Sizes for Final Grade (Fall 2017)
Discipline
Statistics
Biology
Chemistry
Totals
Inverted/Hybrid
N = 24
N = 29
N = 31
N = 84
Class Type
Traditional
N = 26
N = 19
N = 33
N = 78
Totals
N = 50
N = 48
N = 64
N = 162

24. Figure 7. Means and Standard Errors for Final Grades in Statistics, Biology, and Chemistry.* *
Average Grade Received
Notes. *p < .05; Fall 2017

25. Figure 8. Means and Standard Errors for Final Grades in Inverted/Hybrid and Traditional Classes.
Average Grade Received
Note. Fall 2017

26. Sample Sizes Final Grades (Spring 2018)
Discipline
Statistics
Chemistry
Totals
Inverted/Hybrid
N = 45
N = 30
N = 75
Class Type
Traditional
N = 28
N = 14
N = 42
Totals
N = 73
N = 44
N = 117

27. Figure 9. Means and Standard Errors for Final Grades in Statistics and Chemistry.*
Average Grade Received
Notes. *p < .05; Spring 2018

28. Figure 10. Means and Standard Errors for Final Grades in Inverted/Hybrid and Traditional Classes.*
Average Grade Received
Notes. * p < .05; Spring 2018; Effect holds with Statistics N = 44 and Chemistry N = 44

29. Sample Sizes for Withdrawals (Fall 2017)
Discipline
Statistics
Biology
Chemistry
Totals
Inverted/Hybrid
N = 25
N = 31
N = 31
N = 87
Class Type
Traditional
N = 27
N = 25
N = 37
N = 89
Totals
N = 52
N = 56
N = 68
N = 176

30. Figure 11. Means and Standard Errors for Withdrawals in Statistics, Biology, and Chemistry.* *
Proportion of Withdrawals
Note. *p < .05; Fall 2017

31. Figure 12. Means and Standard Errors for Withdrawals in Inverted/Hybrid and Traditional Classes.*
Proportion of Withdrawals
Note. *p < .05

32. Summary of Primary Findings (Final Grades and Withdrawals)
Students had higher grades in Statistics than in Biology or Chemistry
Students withdrew more from Biology than Statistics or Chemistry in Fall 2017
Students withdrew more from traditional classes than inverted/hybrid classes in Fall 2017
Students had higher grades in inverted/hybrid classes than traditional classes in Spring 2018

33. Key Points Students demonstrated important learning over time
Traditional lectures may benefit learning to a degree (at least as evaluated here)
Decreased withdrawals from inverted/hybrid classes warrants further attention
Higher grades in Spring 2018 for inverted/hybrid classes warrants further attention

34. Limitations
Students were not randomly assigned to classes and thus many factors influence which class they end up in
Cannot rule out potential impact of professor (although some of the findings collapse across professors, and across disciplines)
The degree to which the class was hybrid or inverted could influence what was observed

35. Future Work
Examine transfer of learning in sequential classes (e.g., STAT 374 – STAT 375)
Examine faculty surveys on number, type, and length of instructional videos
Examine content that is inverted vs. traditional in the same classes (to control for potential effects of professor)
Examine the role of technology

36. Acknowledgements
We would like to thank NSF for the generous funding, all the participating faculty in this grant ( Drs. Sauncy, Sieben, Ruane, Davis, Jonas, Hijazi, Owenby ) and our colleagues at Texas Lutheran University for their support

37. References
1. Maureen Lage, Glenn Platt, Michael Treglia (2000), Inverting the Classroom: A gateway to Creating an Inclusive Learning Environment, Journal of Economic Education
2. A. Kaw, M. Hess (2007). Comparing Effectiveness of Instructional Delivery Modalities in an Engineering Course Autar Kaw and Melinda Hess, International Journal of Engineering Education, Vol. 23, No. 3, pp
3. Bergmann, J., & Sams, A. (2012). Flip your classroom: reach every student in every class every day. Washington, DC: International Society for Technology in Education
4. Clive Thompson (15 Jul 2011), "How Khan Academy is Changing the Rules of Education", Wired
5. Reza Abbasian & John Sieben“Creating an Inverted Classroom”, ICTCM , Boston, MA, March 2013.
6. Reza Abbasian & John Sieben“Inverted Classrooms: the What, the Why, the How”, Annual MAA-Texas Section, Lubbock, TX, April 2013.
7. Reza Abbasian and John Sieben” Inverted classroms: Videos, Sofware, Hardware, Class Activities and Best Practices”