Understanding and Promoting Interaction in the Classroom Ph.D. Defense Steven A. Wolfman Computer Science & Engineering University of Washington

 t Steve Wolfman Understanding and Promoting Interaction2 The Blackboard “…in the winter of 1813 & '14 … I attended a mathematical school kept in Boston…On entering [the] room, we were struck at the appearance of an ample Black Board suspended on the wall… I had never heard of such a thing before.” [Samuel J. May, 1855]

 t Steve Wolfman Understanding and Promoting Interaction3 Mediating Artifact [Vygotsky] An external object or structure that participates in cognition by supporting or shaping thought.

 t Steve Wolfman Understanding and Promoting Interaction4 Slides as Mediating Artifact In the classroom: –facilitate communication –structure discussion Outside the classroom: –used as memory aid –used as study guide Across terms: –reify course knowledge

 t Steve Wolfman Understanding and Promoting Interaction5 Thesis Question How can computer technology exploit the mediating nature of presentation slides to support and shape interactive learning and teaching?

 t Steve Wolfman Understanding and Promoting Interaction6 Distance & Large Class Studies Classroom Presenter Gestural Model of Ink Classroom Feedback System Feedback Patterns Structured Interaction Presentation system Research History

 t Steve Wolfman Understanding and Promoting Interaction7 Design Experiment Methodology [Brown] Class studies Theoretical framework System design Evaluation & user-centered design

 t Steve Wolfman Understanding and Promoting Interaction8 Outline Introduction Classroom Presenter Classroom Feedback System (CFS) and feedback patterns Structured Interaction Presentation System (SIP) Conclusions

 t Steve Wolfman Understanding and Promoting Interaction9 Classroom Presenter Goals Maintain strengths of slideware (organization, preparation, sharing, execution) Mitigate weaknesses of slideware (inflexibility, immobility, passivity) Secure classroom adoption Prepare for more ambitious systems

 t Steve Wolfman Understanding and Promoting Interaction10

Instructor view with notes Displayed view without notes

 t Steve Wolfman Understanding and Promoting Interaction12 Innovations from User-Centered Design Instructor notes Filmstrip and slide previews “Whiteout” “TV Talk Show” Tablet Collective brainstorming

 t Steve Wolfman Understanding and Promoting Interaction13 Classroom Deployments Surveyed since Spring ’02: –37 courses –21 instructors –2,000+ students –CSE courses: introductory to Master’s level –UW, U. of Virginia, & U. of San Diego

 t Steve Wolfman Understanding and Promoting Interaction14 Survey Results Positive comments and repeat use by instructors Instructor survey: N = 9 Student surveys: N = 479 Attention to lecture 10% less35% no change55% more Encourage future use 8% disc.22% neutral69% enc. Students engaged in lecture 0% less44% no change56% more Use in future 0% no33% maybe67% yes

 t Steve Wolfman Understanding and Promoting Interaction15 Contributions (Presenter) Combined strengths of slides w/increased flexibility, mobility, potential for interaction Developed features that exploit slides as mediating artifact: –Ink in context –Separation of views Secured broad adoption Established basis for student extensions

 t Steve Wolfman Understanding and Promoting Interaction16 Outline Introduction Classroom Presenter Classroom Feedback System (CFS) and feedback patterns Structured Interaction Presentation System (SIP) Conclusions

 t Steve Wolfman Understanding and Promoting Interaction17 CFS Goals Understand challenges to interaction Develop system exploiting slides as mediating artifact to respond to challenges Evaluate impact of feedback system Understand how feedback system changes interaction

 t Steve Wolfman Understanding and Promoting Interaction18 Challenge to Interaction: Feedback Lag A student hesitates to pose a question until the instructor finishes a point. When the instructor moves on, the question seems out of place and is left unasked.

 t Steve Wolfman Understanding and Promoting Interaction19 Leaving Feedback on Current Slide

 t Steve Wolfman Understanding and Promoting Interaction20 Leaving Feedback on Last Slide

 t Steve Wolfman Understanding and Promoting Interaction21 Instructor View

 t Steve Wolfman Understanding and Promoting Interaction22 “Retrospective” Feedback Students’ response (n=12; 7 sessions; 150 students total): –29 episodes of retrospective feedback –CFS helped all who reported feedback lag Instructor’s response: –Retro. feedback is important; often responded –Retro. feedback upset pacing

 t Steve Wolfman Understanding and Promoting Interaction23 “Prospective” Feedback What if a student leaves feedback ahead of the discussion?

 t Steve Wolfman Understanding and Promoting Interaction24 Prospective Feedback Episode

 t Steve Wolfman Understanding and Promoting Interaction25 Contributions (CFS) Identified interaction challenges Proposed slide context as interaction medium Developed contextual feedback system Established potential for student feedback Discovered novel interaction patterns –retrospective feedback: addressing feedback lag –prospective feedback: enabled by computer-mediated communication

 t Steve Wolfman Understanding and Promoting Interaction26 Outline Introduction Classroom Presenter Classroom Feedback System (CFS) and feedback patterns Structured Interaction Presentation System (SIP) Conclusions

 t Steve Wolfman Understanding and Promoting Interaction27 “Conductor-of-Performances” Model Computer-Supported Collaborative Learning (CSCL) has been “a move from ‘sage-on- the-stage’ … to ‘guide-by-the-side’”. New CSCL systems will be “much more like the ‘conductor-of-performances’ for an orchestra: students … [will contribute] to an overall performance.” [Roschelle & Pea]

 t Steve Wolfman Understanding and Promoting Interaction28 Goals of Structured Interaction Presentation System (SIP) Mitigate slides’ passivity, oversimplification Maintain intuitive, flexible design Explore enabled interactions Understand how integrated exercises affect the classroom Support the design, use, sharing, and reflection on the “orchestra’s” score.

 t Steve Wolfman Understanding and Promoting Interaction29

 t Steve Wolfman Understanding and Promoting Interaction30 SIP Architecture Presentation design environment Presentation/ Widget database Instructor view Viewer scrnsht Viewer scrnsht Viewer scrnsht Student views Interactive widget design environment

 t Steve Wolfman Understanding and Promoting Interaction31

 t Steve Wolfman Understanding and Promoting Interaction32

 t Steve Wolfman Understanding and Promoting Interaction33

 t Steve Wolfman Understanding and Promoting Interaction34 Distributed Human Computation (DHC) Are these on the same or distinct topics? Which would you rather discuss? Of those who died from receiving the vaccine, what percentage had compro- mised immune systems? What are the death rates for specific groups who received this vaccine?

 t Steve Wolfman Understanding and Promoting Interaction35

 t Steve Wolfman Understanding and Promoting Interaction36 DHC Results: Instructor’s View Group members Group “winners”

 t Steve Wolfman Understanding and Promoting Interaction37 Experimental Class Results Interaction analysis (video/audio/logs): substantial engagement by students Student survey results –Factors supporting interaction: highlights particular strengths of integration –Factors hindering interaction: highlights important design lessons

 t Steve Wolfman Understanding and Promoting Interaction38 Interaction Analysis teacher talk student talk student discussion student thinking other (62%) (15%) (13%) (5%)

 t Steve Wolfman Understanding and Promoting Interaction39 Factors Supporting Interaction (n=18) Category% students Sharing responses w/ whole class39 Participatory feel33 Novelty22 Anonymity17 Forced to participate17 Helps follow instructor11 Neighbor discussion6 None6 Did not respond6

 t Steve Wolfman Understanding and Promoting Interaction40 Factors Hindering Interaction (n=18) Category% students None56 Distracting applications22 Distracted looking at my slides17 Did not respond11 Reduced coverage11 Lack of student control11 Pace too fast6 Pair discussions6 Anti-climactic exercise6

 t Steve Wolfman Understanding and Promoting Interaction41 Contributions (SIP) Proposed “score” as role of slides as mediating artifact in “orchestra” CSCL model Developed prototype SIP system Designed novel interactive exercises (e.g., DHC, “sampled quiz”) Identified advantages and pitfalls of integrated interactivity

 t Steve Wolfman Understanding and Promoting Interaction42 Conclusions Developed and evaluated widely-adopted Classroom Presenter system Developed and evaluated Classroom Feedback System Developed and evaluated Structured Interaction Presentation System Demonstrated how to exploit slides as mediating artifact across all three systems (e.g., separated views, contextual feedback, “forced” participation)

 t Steve Wolfman Understanding and Promoting Interaction43 Related Systems Ubiquitous Computing: –eClass/Classroom 2000 [Abowd & Brotherton] –ActiveClass [Griswold, Ratto, Truong, et al.] –Cell-phone feedback [Brittain] Education/Educational Technology: –ClassTalk [Dufresne] –Debbie/DyKnow [Berque] –WILD [Roschelle & Pea] HCI: Pebbles [Myers]

 t Steve Wolfman Understanding and Promoting Interaction44 Related Pedagogy Active learning [Bonwell & Eison] Active learning in CS [McConnell] Classroom Assessment Techniques “CATs” [Angelo & Cross] CATs in CS [Schwarm & VanDeGrift] Collaborative Learning [Johnson & Johnson]

 t Steve Wolfman Understanding and Promoting Interaction45 Acknowledgments Richard Anderson and the Committee Rachel Pottinger Education and Educational Technology Group Microsoft Research LST Group Experiment participants Faculty, staff, and students of UW CSE Intel, MERL, Microsoft, and NSF for funding Everyone

 t Steve Wolfman Understanding and Promoting Interaction47 Modern Pedagogy vs. Modern Practice Opportunity for audience participation? active learning participatory interactive student-directed lecture instructor-dominated passive disconnected

 t Steve Wolfman Understanding and Promoting Interaction48 Pedagogy of Active Learning Encourage “connected” learning –Constructivism [Bruner] –Social learning [Lave] Recapture flagging attention –Attention studies [Stuart & Rutherford] –Heart rate/memory [Bligh] –Skin conductivity [Picard] Address varied learning styles –Index of Learning Styles [Felder & Silverman] –Bloom’s taxonomy

 t Steve Wolfman Understanding and Promoting Interaction49 Large class challenges Maintaining attention Communication/Feedback Spontaneous discussion Management of class activities

 t Steve Wolfman Understanding and Promoting Interaction50 Slides as Externalization/ Mediating Artifact

 t Steve Wolfman Understanding and Promoting Interaction51

 t Steve Wolfman Understanding and Promoting Interaction52 Slideware Strengths/Weaknesses Strengths Organization [PETTT] Preparation [PETTT] Sharing [Bligh] Easy execution [Bligh] Weaknesses Passivity [PETTT] Simplified ideas [Tufte] Inflexibility [VanDeGrift] Immobility

 t Steve Wolfman Understanding and Promoting Interaction53 Slide previews with navigation

 t Steve Wolfman Understanding and Promoting Interaction54

 t Steve Wolfman Understanding and Promoting Interaction55 Design Experiment [Brown] Discover what inhibits interaction Understand what makes a good design Design intervention Evaluate

 t Steve Wolfman Understanding and Promoting Interaction56 Feedback on Student View

 t Steve Wolfman Understanding and Promoting Interaction57 Instructor View (2/3)

 t Steve Wolfman Understanding and Promoting Interaction58 Instructor View (3/3)

 t Steve Wolfman Understanding and Promoting Interaction59 CFS Evaluation Intro. programming course, summer 2002: –150 students, 12 participants w/laptops –9 week course, 3 weeks with CFS Data: observations, surveys, focus groups, interview w/instructor, logs

 t Steve Wolfman Understanding and Promoting Interaction60 CFS increased interaction Voicings pre-CFS Voicings with CFS All inter- actions All but “Got it” # per class p-value.91.04*.14

 t Steve Wolfman Understanding and Promoting Interaction61

 t Steve Wolfman Understanding and Promoting Interaction62

 t Steve Wolfman Understanding and Promoting Interaction63

 t Steve Wolfman Understanding and Promoting Interaction64

 t Steve Wolfman Understanding and Promoting Interaction65

 t Steve Wolfman Understanding and Promoting Interaction66

 t Steve Wolfman Understanding and Promoting Interaction67

 t Steve Wolfman Understanding and Promoting Interaction68 Experimental SIP Class Topic: Risk Assessment Duration: 50 minutes Students: 24, CS stud- ents, faculty, staff, each w/a Tablet PC Presentation included six SIP exercises. Results: System successful 98% participation in SIP exercises 4-7 interactions per student (median) 62% “teacher-talk”

 t Steve Wolfman Understanding and Promoting Interaction69 Design Lessons SIP’s integrated exercises create “participatory feel” Students’ social expectations support participation SIP anonymity policies should be easy to specify and understand Student interface should present few distractions Student interface should provide clear value (independent navigation and notetaking) Instructors must still motivate interactive pedagogy

 t Steve Wolfman Understanding and Promoting Interaction70