Mixed-Format Undergraduate Pedagogic Delivery Steve Buchheit| Feb. 23rd, 2011

Overview: What’s New Here? Institution-specific experience at Texas Tech RCB Mediasite experience and exposure Pilot-study data in a required undergraduate course Extensive-pilot-study in the same course Implications for current cost/quality debates College should do better argument College should be less expensive argument College is just fine argument

teaching & Technology at ttu History of self-selection into teaching technology (TLTC: awesome but voluntary) Mediasite enabled room for Executive-style MBA Largely unknown, unused capacity during the week Logistical and political concerns about using Mediasite in introductory managerial accounting

introductory managerial accounting (ACCT 2301) at ttu Required course for all Business majors Coordinated exams Multiple “sections” per semester (4 – 8) Avg. class size ≈ 55 to 70 students Taught by a wide range of instructors Doctoral student through full professor Lecture style On-line homework (Blackboard)

Spring 2010 Logistical & Political concerns Logistical: “Fitting” a large class into a 35 person Mediasite-enabled room Political: Not upsetting ANY students (thus, self-selection)

Spring 2010 Logistics Day 1 Mediasite visual demo Surveyed potential student interest Exam 1 Remind students about room change Remind students about virtual class option Exam 3 Conclude Qualtrics surveys, return to original format (final week of class).

Spring 2010 Unexpected Political concerns Upsetting faculty Accounting faculty not directly affiliated with introductory managerial accounting Concerns “Turning off” potential accounting majors Jeopardizing success (Texas Tech is currently #12 in CPA pass rate; 100% MSA placement)

Spring 2010 Results, Part 1 25% of students physically attended class (target section only) Virtually no participation from non-target sections (4 students) Significantly positive student attitudes Both testimonials and Likert Scale items. Only reported drawback, ‘insufficient internal discipline’

Spring 2010 Results, Part 2 Student evaluations improved! 4.85 out of 5 (all other instructors 3.48 out of 5) 4.85 vs. same-instructor, prior semester 4.68 (p value = .06) Target class > Non-target class (p value = .02) ‘Virtual’ participants ≥ ‘In seat’ participants (+ 2% improvement was not significant) Only 1 student interacted ‘live’ with Mediasite Note: the Mediasite class was at 8:00 am Discontinued use of TurningPoint’s ResponseWare

Fall 2010 Extensive Pilot Study – Same course Two different instructors on Mediasite Both instructor’s lectures available to all students Student Awareness not an issue Logistical “fit” into Mediasite room not an issue Extensive Day 1 explanation & Surveys

Fall 2010 Logistics Day 1 Mediasite visual demo Surveyed student interest / ZTPI scale Directed students to Mediasite room Exam 3 Return to original format (final week of class). Surveyed students

Fall 2010 Results, Part 1 25% of students physically attended class (≈ 20% at 8:00 am; ≈ 30% during later classes) Self-reporting consistent with instructor observation 26% of students claimed to mix physical attendance with ‘virtual attendance’ Self-reporting not consistent with instructor observation (other than the single class just prior to exams)

Fall 2010 Results, Part 2 84% of students successfully finished the course Slightly better than prior semesters Multiple caveats (increasing BA gpa, etc.) Within-semester performance slightly better Multiple caveats (attempt to control difficulty, etc.) ZPTI scale not effective at predicting success.

Fall 2010 Results, Part 3 Students overwhelmingly found ‘virtual participation’ a good thing. Zero-scaled 5-pt Likert average = 1.48 (p < .000001) We didn’t increase the score for ‘write-ins’ Stated rationale supported the numeric results next slide Only 4% rated the option worse than neutral Surprisingly all but one took responsibility (“lack of discipline” explanations)

Fall 2010 Results, Part 4 Why was the Mediasite option useful? It wasn’t (4%), no response (4%), indifferent (5%) Work at own pace (4%) Multiple teacher’s perspectives (9%) Safety (15%) Flexibility, work on own time (23%) ‘Rewind’ option (36%)

Fall 2010 Results, Part 5 Student evaluations improved! ‘New experience’ 4.09 vs. same-instructor, prior semester 3.73 (p value = .03) ‘Repeat experience’ remained exceptional (4.78 FA10) Where to go from here?

What’s Next? Prior-semester lectures & current ‘live’ lecture SP 11, ≈ 30% attendance ‘preview’ lecture anecdotes Implications for current cost/quality debates College should do better argument College should be less expensive argument College is just fine argument

College should do better argument “Academically Adrift” view Take lecture ‘off line’ & enable more in-class critical thinking, writing, etc. Personal view This would be very difficult at the sophomore level. We are ‘screening’ a large number of students with limited resources Lots of potential at the upper-division levels.

College should be less expensive argument Bill Gates, Rick Perry argument KhanAcademy.org = a great concept, but short & medium-term practical implementation seems to require a great deal of judgment at the university level. Short-term savings don’t simply ‘materialize’ by leveraging current resources. Some serious deployment issues

College is just fine argument It’s not 61% of our sample work, and 39% work more than 20 hours per week. Faculty and students feel overwhelmed From a pseudo-novice perspective, a clear explanation of exactly how technology can help very beneficial.

THANK YOU