1. Digital Assessments in Flipped Classrooms Workshop George Szanto Docent Marketing Management g.szanto@fontys.nl @hippity7 Fontys Toetsconferencie 2014 g.szanto@fontys.nl

2. Voettekst2 14:55 – 15:05 Introduction of workshop participants 15:05 – 15:20 Introduction to my flipped classroom; case & dilemmas 15:20 – 15:30 work on dilemmas in pairs 15:30 – 15:50 discuss answers to dilemmas as group 15:50 – 16:10 Review of George’s efforts & dilemmas: quantitative & qualitative testing 16:10 – 16:15 Wrap-up, Q & A. Agenda

3. Voettekst3 Name Institute or Fontys Service Classes/Subjects that you teach Yes/No Digital testing experience Yes/No Flipped class teaching experience BRIEF Participant Introductions

4. Why Flip My Class Motivations of George

5. International Exchange Students

6. Student Complaints

7. Poor Quality Group Work

8. Fighting, Disputes in Groups

9. Limited Resources

10. History of IBMS Sem. 6 eMarketing Strategy class 2011 - 2014 Classical TeachingFlipped Classroom Teaching Fall 2011 Spring 2011 Fall 2012 Spring 2013 Fall 2013 Spring 2014 Fall 2014 Live Lectures 4442 Recorded Lectures 626 27 Live Workshops 1225566 Group Presentation in Class 1 Individual Presentation in Class or Video Recording Indiv 20 % Indiv %30% Indiv In pairs 50% In pairs 60% Extra Group Assignment (20%) 20% Main Group Assignment Report (80%) 80% Sit down individual final exam (50 - 80%) 60% 50% 80% 5/6 Individual Open Book/Notes In Class Quizzes 40 % Online discussion board / class participation 20% 10% Peer Grading

11. IBMSEM.ORG Flipped Classroom 11

12. Assessments

13. Digital On Demand SaaS Quizzes € 45 per year € 350 per year / 200 students / 1 Lecturer ~ € 2000 per year

14. 10 – 14 Minute Executive Level Emarketing Stratety Pitch 14

15. Introducing the digital testing dilemmas for this workshop 1.BYOD or Fontys Computers? 2.Wifi/Computer Network down – now what? 3.Asynchronous testing & cheating 4.Participation in formative quizzes < 20% 5.Archiving test results for accreditation 6.Technical issues for a few students/quiz 7.Online quizzes – full use of Internet/open resources or not? 8.How to handle students with learning disabilities ? 9.Summative quizzes do not help raise final score of project – now what? 10.Can student take summative quiz outside of workshop? 15

16. Digital Testing Dilemmas For the next 5 minutes, please work in pairs discussing solution(s) to your assigned dilemma 16

17. Your Solutions to Dilemmas 1-5 17

18. Your Solutions to Dilemmas 6-10 18

19. Voettekst19 Do digital quizzes improve the performance and quality of the final eMarketing Strategy Plan submitted by students in a flipped classroom? What kind of positive and negative feedback is given about the digital quizzes by students who have taken the class? Key Questions of my Research

20. Voettekst20 Dependent & Independent Variables Final Project Grade Quiz Scores Quantity Quality Videos In-class participation Student Motivation Past student knowledge # of hrs. studied outside class Assigned Case

21. Results Final Exam Spring 2013 (no formative quizzes) N=48

22. 22 Results Fall 2013 (where 70% of final grade is from the exam; +formative quizzes)

23. Fall 2013 Poor correlation between formative quiz results and final exam score (but N is small!)

24. Fall 2013 How about students who take few formative quizzes?

25. Student Expectations – baseline survey before class starts (N= 24)

26. Fall 2013 Other Independent Variables – Videos Watched (no correlation, N is small)

27. Independent Variable – class participation (No correlation, N is small)

28. Fall 2013 Independent Variable – past GPA full time students (poor correlation, N is small)

29. Spring & Fall 2014 Summative Quizzes

30. Voettekst30 Digital summative quizzes don’t seem to help produce quality video pitches Student feedback: –Positive, force students to keep up in class –Test deeper understanding of theoretical concepts Considering reducing time limits; and introducing harder questions. Conclusions

31. Voettekst31 SURF website – see: https://www.surf.nl/kennis-en- innovatie/kennisbank/onderwerp/leren-en- toetsen/onderwerp/digitaal-toetsenhttps://www.surf.nl/kennis-en- innovatie/kennisbank/onderwerp/leren-en- toetsen/onderwerp/digitaal-toetsen Additional Resources

32. Thank You – Q & A

33. 33

34. Some Descriptive Statistics Fall 2013 Voettekst34 Grading based on 65% of highest grade (no partial grading) Highest Grade = Grade of 9.0 (because < 100 pts scored)81 65% of Highest Grade52.65 # of students passing11 # of students failing17 Partial Grading Results (3/4 std) Average Score Exam (110 pts)68.26 Standard Deviation (Pts)11.03 Pass Score (Ave - 0.75*STD)59.98 # of students failing8.00 # of students passing20.00 Max Score:87.00 Min Score:47.73 Average Student Grade7.00 Average Total # Pts (100)70.38 Average Exam Pts (70)49.23 Ave. Class Parcip. Pts (30)21.15 Ave. # of workshops attended (6)4.77 No Partial Grading (3/4 Std) Average Score Exam (110 pts)48.93 Standard Deviation (Pts)14.48 Pass Score (Ave - 0.75*STD)38.06 # of students failing9.00 # of students passing19.00 Max Score81.00 Min Score24.03