1. Research In Educational Technology: Expanding Possibilities Richard Anderson Department of Computer Science and Engineering University of Washington July 1, 20081IUCEE: Research in Educational Technology

2. Research in Educational Technology How can computing technology enhance education? – Focus on classroom instruction Challenges: – Extending reach of education – Increasing interaction – Addressing problems of scale – Facilitating expression of ideas July 1, 20082IUCEE: Research in Educational Technology

3. Past and Current Research Projects Video conferenced distance education UW PMP DISC ConferenceXP Center for Collaborative Technologies Presentation systems Classroom Presenter 2.0 Classroom Presenter 3.0 Classroom interaction systems Classroom Feedback System CATs for CS1 Structured Interaction Presentations (SIP) Student submissions with CP Tutored Video Instruction UW CC TVI Project Beihang TVI project Digital StudyHall July 1, 20083IUCEE: Research in Educational Technology

4. Research Approach Deployment driven – Classroom use – Technology development and promotion Goals and success criteria – Adoption of technology and methodology – Influence educational practice This is a model that has been working for us – Target specific deployments that are innovative in some dimensions July 1, 20084IUCEE: Research in Educational Technology

5. Today’s Talk Significant point of time for the project – Substantial number of completed projects – Formation of Center for Collaborative Technologies – Deployment of Classroom Presenter 3.0 – Opportunity to develop classroom technologies that will have a broad impact Summary of educational technology projects – Lessons learned and remaining challenges Future projects July 1, 20085IUCEE: Research in Educational Technology

6. Video Conferenced Teaching Multi-site internet based audio-video conferencing UW PMP Program – Site-to-site courses between UW and Microsoft since Winter 1997 – www.cs.washington.edu/education/dl/course_index.html www.cs.washington.edu/education/dl/course_index.html – Master’s level courses – Goal: interaction across sites Approximate single classroom – Various technologies have been used since the program was introduced July 1, 20086IUCEE: Research in Educational Technology

7. July 1, 2008IUCEE: Research in Educational Technology7

8. Video conferencing in the PMP Winter 1997 – Winter 2002 – Polycom + Netmeeting for PPT and SmartBoard MSR DISC Project – Target: UW, CMU, UCB, Brown graduate class – Spring 2002 MSR ConferenceXP – Since Spring 2003 – Four way courses, Autumn 2004, Autumn 2005, Autumn 2006 UW, MSR, UCB, UCSD Ed Lazowska, Steve Mauer July 1, 20088IUCEE: Research in Educational Technology

9. DISC (PMP spring 2002) What went wrong – Technology and systems failures – High cost of interruptions – Loss of trust – Room configuration issues – Lack of control of lecture room – Production quality Meta lesson – Learn more from failures than from successes How to Fail at VideoConferenced Teaching – Microsoft Faculty Summit 2002 – Anderson & Beavers July 1, 20089IUCEE: Research in Educational Technology

10. ConferenceXP Target: High bandwidth internet video conferencing Technology: Multicast networking, Internet2 Vision: Single machine deployment, ease of use Designed as extensible platform – Integration of other information channels Slides and Ink – Source released by MSR as shared source Production use in UW PMP since Spring 2003 July 1, 200810IUCEE: Research in Educational Technology

11. Center for Collaborative Technologies at University of Washington UW center funded for continued work on ConferenceXP Platform – http://cct.cs.washington.edu http://cct.cs.washington.edu Extend functionality of ConferenceXP – Diagnostics, Security, Remote management, HDTV integration,... Build community of users and developers Deploy ConferenceXP in new scenarios – International education – Developing world July 1, 200811IUCEE: Research in Educational Technology

12. Success in distance classes Goals – Real time interaction between sites – High quality video Challenges – High bandwidth connections – Classroom Audio – Establishing a pattern of interaction July 1, 200812IUCEE: Research in Educational Technology

13. Hardware Multicast Technology bet (2001) – Multicast networking to support multisite courses – Substantial bandwidth savings – Multicast not uniformly supported July 1, 200813IUCEE: Research in Educational Technology

14. Dealing with multicast problems Reflector service – Plug in unicast to replace multicast Used as backup in our courses Solution when connecting to networks without multicast July 1, 200814IUCEE: Research in Educational Technology

15. Going International March 29, 2008, LACCIR Meeting – Latin American and Carribbean Collaboration for ICT Research Seattle and University of Chile, Santiago, Chile Seminar presentation CXP Unicast reflector July 1, 200815IUCEE: Research in Educational Technology

16. Masters class, UW - Pakistan Masters class – University of Washington – Lahore University of Management Science – Microsoft Computing for the Developing world July 1, 200816IUCEE: Research in Educational Technology

17. Technical Challenges Ensuring adequate bandwidth – Limited bandwidth to Pakistan – Reliability – Multicast – Ensuring this did not compromise UW-MS class – Limited time to prepare July 1, 200817IUCEE: Research in Educational Technology

18. Fred’s whiteboard July 1, 200818IUCEE: Research in Educational Technology

19. Basic PMP setup (2 sites) PMP VENUE Archiver Video cameras Audio Video Displays Speakers Video cameras Audio Video Displays Speakers CP3 Instructor CP3 Display CP3 Display Student Tablets UWMicrosoft CXP CP3 19

20. 3-way setup for UW, MS, LUMS PMP Venue 1 PMP Venue 2 Archiver MicrosoftLUMS UW CP3 Server CP3 July 1, 200820IUCEE: Research in Educational Technology

21. Use of Classroom Presenter Tablet PC based presentation and classroom interaction system Ink based presentation Classroom Activites July 1, 200821IUCEE: Research in Educational Technology

22. Classroom PresenterStudent Student Instructor Public Display July 1, 200822IUCEE: Research in Educational Technology

23. Classroom Activities July 1, 200823IUCEE: Research in Educational Technology

24. Projects related to distance learning Working with archived lectures Large library of recorded lectures available – Autumn 2006 Algorithms class recorded with close talking microphone Lecture indexing – support text search of speech (and slides and ink) – Language modeling necessary (train on algorithms or CS content) Lecture summarization – Classify lecture episodes Support for lecture browsing Feedback to the instructor Lightweight lecture capture July 1, 200824IUCEE: Research in Educational Technology

25. Classroom Presenter Support electronic slides and digital ink Initially developed for whiteboard integration of DISC – “PowerPoint sucks the life out of a lecture”, EDL Tablet PC application – Digital ink overlay on slide images – Feature set aimed at lecture presentation July 1, 200825IUCEE: Research in Educational Technology

26. Ink based presentation Tablet PC Inking on images Simple pen based controls Whiteboard, slide extension Multiple views – instructor/display – (dual monitor) Multiple slides decks with filmstrip navigation Instructor notes July 1, 200826IUCEE: Research in Educational Technology

27. Ink usage July 1, 2008IUCEE: Research in Educational Technology 27

28. Classroom Presenter Deployments Adoption in wide range of subjects and institutions Many of the key ideas have been generated by users Emphasis on simplicity of UI and application July 1, 200828IUCEE: Research in Educational Technology

29. Ink Based Presentation Challenge in developing UI to support presentation – Low attention UI – Introduce a richer set of operations without compromising usability Inking behavior very complicated – Post processing instructor ink Lecture summaries and visualization July 1, 200829IUCEE: Research in Educational Technology

30. Tablet PC Project: Analysis of Handwritten Notes Note taking – Many applications exist for taking notes, but the real value of TPC notes (over paper) is being able to work with them digitally – Notes vary greatly in structure and are often messy – Search: Find “dynamic programming” – Type search: Find all phone numbers – Classification: Find all pseudocode July 1, 200830IUCEE: Research in Educational Technology

31. Classroom Interaction Systems Integration of electronic devices into the classroom to support instruction General motivation is to involve students in ways that achieve specific pedagogical goals – E.g., Classroom networks have been demonstrated be very effective for science instruction July 1, 200831IUCEE: Research in Educational Technology

32. UW CSE Work on Classroom Interaction Systems Tutored Video Instruction – Activities to support the facilitator – Classroom Assessment Techniques (Angelo and Cross) Classroom Feedback System – Student response system associated with lecture slides Structured Interaction Systems – Steve Wolfman’s thesis – Rich activity model built into slides July 1, 2008IUCEE: Research in Educational Technology32

33. Student Submissions Simple model for activity taking advantage of digital ink Students write answers on slides, send them to the instructor Instructor previews results and selects slides to display to the class July 1, 2008IUCEE: Research in Educational Technology33

34. IUCEE: Research in Educational Technology Classroom Presenter Student Student Instructor Public Display July 1, 200834

35. Activity Examples July 1, 200835IUCEE: Research in Educational Technology

36. Activity Examples July 1, 2008IUCEE: Research in Educational Technology36

37. Deployments Algorithms, Digital Design, Software Engineering, Data Structures, Environmental Science at UW Outside UW: Physics, Calculus, Ethics, Biology, Electrical Engineering, Introductory Programming,... Used at all levels – High School, Community College, University July 1, 200837IUCEE: Research in Educational Technology

38. Classroom goals Active Learning Encourage students to contribute in multiple ways Promote engagement in the class – Interest – Alertness Demonstrate that all students have important opinions Peer interaction Feedback – classroom assessment Collection of ideas – Collective brainstorm Student generation of examples Discovery of a pedagogical point Gain understanding of an example Show misconceptions July 1, 200838IUCEE: Research in Educational Technology

39. Impact Instructors successful at achieving classroom goals Significant participation by students Change in classroom dynamics Negative: deployment overhead July 1, 200839IUCEE: Research in Educational Technology

40. Tutored Video Instruction Video recorded lectures shown with facilitator – Original model: lectures stopped by students for discussion – Peer tutors Developed by Jim Gibbons at Stanford University Positive results reported in Science [1977] July 1, 200840IUCEE: Research in Educational Technology

41. UW TVI Projects Introductory programming – Address community college articulation – Experiment with alternate approaches to introductory computing instruction UW – Beihang Algorithms course – Offering of CSE 421 in China Digital StudyHall – Primary education in rural india July 1, 200841IUCEE: Research in Educational Technology

42. UW – Community College Lectures recorded from UW Intro Class Shown at CCs with local instructors as facilitators Project lasted 3 years, involving 9 CCs Phase I – Materials from live lecture, centralized grading, management from UW Phase II – Studio created materials, CC grading July 1, 200842IUCEE: Research in Educational Technology

43. Lessons Learned Results were mixed Complicated institutional relationships – CC students concerned about competition with UW students Facilitation model – Did not achieve peer facilitation Co-teaching a more accurate description – Facilitators wanted external support (e.g., classroom activities) Program helped with instructor development July 1, 200843IUCEE: Research in Educational Technology

44. July 1, 2008IUCEE: Research in Educational Technology44 UW-Beihang CSE 421 Materials captured from live classes – Slides, talking head, digital ink Classroom Technology – Students used Tablet PCs to participate in classroom activities – Tablets PCs used both at Beihang and UW

45. Results Offering successful – Technology, institutional relationship Cross-cultural issues – English language materials were comprehensible – Classroom discussion primarily in Chinese Facilitation model – Significant support for facilitators – Classroom activities successful (and popular) – Facilitators innovative and reproduced some of the instruction – Interactive and informal classroom atmosphere July 1, 200845IUCEE: Research in Educational Technology

46. Digital StudyHall Affiliated Project Randy Wang, Paul Javid (MSRI, Bangalore) Richard Anderson, Tom Anderson (UW) Tutored Video Instruction for primary education in rural india YouTube + Netflix July 1, 200846IUCEE: Research in Educational Technology

47. What we’ve learned from all of this Value of electronic materials in the process of classroom instruction Tools for teaching – Teacher and students drive the process – Flexible and unpredictable use Structured Interaction model Broader context – interplay of technology and other issues July 1, 200847IUCEE: Research in Educational Technology

48. Deployment Driven Research Development and deployment of educational technology Internal – Working with our own classes – Opportunity to innovate – Pressure to make things work External – Broad range of ideas – User suggestions – Feedback on ideas July 1, 200848IUCEE: Research in Educational Technology

49. Fan mail To: Richard Anderson Subject: UW CSE Web: Classroom Presenter FAQs Dear Mr Anderson, i am edy from jakarta, indonesia. What a great software i found, made by UW CSE. To: Richard Anderson Subject: UW CSE Web: UW Classroom Presenter May I take a moment to say, once again, THANKS for creating CP! I've used it during a conference presentation and in all but one of my classes this year. To: Richard Anderson Subject: CSE Home Page: Classroom Presenter FAQs Dear Dr. Anderson, So, I think you can say I'm trying out CP for the first time. I really thank you for your enormous effort to provide such an excellent tool. To: Richard Anderson Subject: Re: TP Mode Richard, Thanks again for your support of this great product. Seriously, I would not be lecturing with my tablet pc without it. Powerpoint was way too restrictive and made me REALLY nervous. July 1, 200849IUCEE: Research in Educational Technology

50. Classroom Technology Challenges Make it universal Deepen level of interaction with materials Expand the reach July 1, 200850IUCEE: Research in Educational Technology

51. Broader Access Critique of Classroom Presenter –... but students don’t have Tablet PCs – High overhead in deployment – Many different costs Sustainable deployment – Student owned devices – Heterogeneous deployment of devices – Value to all participants July 1, 200851IUCEE: Research in Educational Technology

52. The next steps Electronic, slide based lecture supporting flexible instructor control Extend device and interaction models Wide range of interaction models available – Polling, Group Scribbles, Multipoint, shared whiteboard, student submissions Challenge – Maintain focus and simplicity July 1, 200852IUCEE: Research in Educational Technology

53. Richer content support for slide based lectures Slide model: static content or build slide animations Challenge: provide a richer model of content for dynamic presentations – Particular domain of interest: mathematical content Starting points – Instructor notes – Structured Interaction Presentations (SIP) [Wolfman] – Geometrical structure for slides July 1, 200853IUCEE: Research in Educational Technology

54. Facilitation for Tutored Video Instruction Teaching with recorded materials – Peer discussion vs. co-teaching Regular interruptions for active learning Beihang class – Facilitators made substantial use of Classroom Presenter – Activity structure was successful Projects – Develop integrated TVI replay, presentation and classroom interaction tools – Refine methodology for combining active learning with TVI – Replay tools for DSH scenarios July 1, 200854IUCEE: Research in Educational Technology

55. Classroom Accessibility Opportunities in electronic classroom for greater accessibility Classroom capture and archiving Real time interpretation – Captioning/Screen reading Input – Instant messaging, shared whiteboard, custom input facilities Collaborative work with Richard Ladner July 1, 200855IUCEE: Research in Educational Technology

56. 56 Enabling Access to STEM Education Slide courtesy of Richard Ladner/Anna Cavender July 1, 2008IUCEE: Research in Educational Technology

57. Classroom Presenter 3.1 Richer Feature Set – Display Control – Classroom Interaction Quick Poll – Expanded interaction models New classroom activities – Additional source content Performance – Scalability in wireless classroom July 1, 200857IUCEE: Research in Educational Technology

58. Center for Collaborative Technologies Development of ConferenceXP Platform Establish as a shared source project System enhancements – Multicast diagnostics – Security Deployments – Collaboration with Microsoft sponsored Latin America Virtual Institute – UW Professional Master’s Program July 1, 200858IUCEE: Research in Educational Technology

59. Domains of Special Interest Higher Education International Courses Developing World Global Health July 1, 200859IUCEE: Research in Educational Technology

60. International Education Multi-site classes with ConferenceXP Challenges – Networking issues (firewall, multicast) – Identifying cases where interactivity is needed – Time zones West Coast US (6:00 pm) & China (9:00 am) Short term – Pilot tests with Chinese Universities – Latin America Virtual Institute – International guest lectures for UW CSE PMP Class (spring) July 1, 200860IUCEE: Research in Educational Technology

61. Developing World Tremendous challenges faced in education in the developing world Technology supported instruction that is cost-realistic and sustainable Digital StudyHall – India, Bangladesh, Eritrea,... Interactive, Facilitated Video Instruction Low cost multi-person interaction – E.g., Multimouse Deployment issues – Lack of power, network connectivity July 1, 200861IUCEE: Research in Educational Technology

62. Global Health Strong regional opportunity Distance education to support medical education Alternate models of video based instruction July 1, 200862IUCEE: Research in Educational Technology

63. For more information Richard Anderson – anderson@cs.washington.edu anderson@cs.washington.edu Classroom Presenter – http://www.cs.washington.edu/education/dl/presenter/ http://www.cs.washington.edu/education/dl/presenter/ Center for Collaborative Technologies at UW – http://cct.cs.washington.edu/ http://cct.cs.washington.edu/ Digital StudyHall – http://dsh.cs.washington.edu/ http://dsh.cs.washington.edu/ Other contacts – CCT: Fred Videon (fred@cs.washington.edu)fred@cs.washington.edu – Digital StudyHall: Paul Javid (pjavid@cs.washington.edu), Tom Anderson (tom@cs.washington.edu)pjavid@cs.washington.edutom@cs.washington.edu – Classroom Accessibility: Richard Ladner (ladner@cs.washington.edu)ladner@cs.washington.edu July 1, 200863IUCEE: Research in Educational Technology

64. Acknowledgements Support from Microsoft Research, National Science Foundation, HP, Ford, UW CSE Jay Beavers, Jane Prey, Randy Hinrichs, Chris Moffatt, Jason Van Eaton, Paul Oka, Steve Wolfman, Ken Yasuhara, Kate Diebel, Ruth Anderson, Craig Prince, Valentin Razmov, Natalie Linnell, Krista Davis, Jonathon Su, Sara Su, Peter Davis, Tammy VanDeGrift, Joe Tront, Alon Halevy, Gaetano Borriello, Ed Lazowska, Hal Perkins, Susan Eggers, Fred Videon, Rod Prieto, Oliver Chung, Crystal Hoyer, Beth Simon, Eitan Feinberg, Julia Schwarz, Jim Fridley, Tom Hinkley, Ning Li, Jing Li, Luo Jie, Jiangfeng Chen July 1, 200864IUCEE: Research in Educational Technology