Craig Prince General Examination Talk January 17, 2008

Motivation Goal: Improve education by integrating computing devices into classroom. Do in low-cost, sustainable way. One way, leverage devices students carry! 2

Before I Begin… Who has a laptop or Tablet PC with them? Who has a cell phone or PDA with them? College Students: 86.1% Cell Phone 12.0% Smart Phone 11.9% PDA 56.3% Digital Gaming Device 73.7% Laptop Source: The ECAR Study of Undergraduate Students and Information Technology, 2007

My Talk – In Three Parts Part I: Background Part II: Systems Active Learning Classroom Responses Systems (CRSs) Different CRS activities Part II: Systems Tablet PC-based Handhelds Part III: Future Work Improve type of exercises on handhelds Integrate diverse handhelds into class 4

Part I: Background Active Learning Classroom Responses Systems (CRSs) Different CRS activities

Active Learning Active Learning = teaching pedagogy were best way to teach students is to have them be active in the learning process Abrahamson’s “Socratic teaching” method “Constructivism” vs. “Behaviorism” Student needs to: Build on existing mental framework Incorporate existing knowledge and ideas Abrahamson A. L. “Teaching with Classroom Communication System – What it Involves and Why it Works”. Mini-Course presented at the VII Taller Internacional "Nuevas Tendencias en la Ensenanza de la Fisica". Puebla, Mexico, May 27-30, 1999. 6

Quantitative Evidence Hake’s study 62 physics courses >6000 students Set taught interactively Set taught traditionally (lecture focus) Students in interactive courses had learning gains 2 std. dev. greater than traditional courses University, Community College, and High School Hake R. “Interactive-engagement Versus Traditional Methods: A Six-thousand-student Survey of Mechanics Test Data for Introductory Physics Courses”. American Journal of Physics, 66 (1), Jan. 1998.

Active Learning in the Classroom Mazur’s “Peer Instruction” Mechanism for Active Learning Basis for many classroom technologies Promotes: Discussing/collaborating between students Integrating student work into lectures Mazur, E. Peer Instruction: A User's Manual. Prentice-Hall, New Jersey, 1997.

Classroom Network C B D A Classroom Network Public Display Instructor Students A B C D Classroom Network 9

Classroom Technology Classroom Network Cooperative Note-taking e.g. LiveNotes PowerPoint Slides Digital Whiteboards Public Display Instructor Students Classroom Network What is the benefit of technology in the classroom? Digital Ink Presentation Tools Digital Whiteboards Classroom Response Systems Classroom Feedback Systems

Classroom Response System (CRS): Learning Cycle Activity Public Display Instructor Students Classroom Network Activity Activity Activity Activity B Now we want to talk a bit about classroom response system. Here is how they work. Activity

Multiple-Choice Example And these are the types of exercises that are enabled by these systems, start with simple multiple-choice questions, and more to more complex digital ink questions Clickers

Multiple-Choice Example 2 Classtalk

Textual Example 14

Diagrammatic Example

Diagrammatic Example

Image Example

Brainstorming Example

What Activity Types are Best? No quantitative studies showing digital ink makes for better activities in CRSs Advantages of Digital Ink Strictly more expressive Some activities clearly easier with Digital Ink Instructors like using these types of activities Disadvantages of Digital Ink Harder to aggregate/understand Need to make it clear why you are showing this slide

Part II: Systems Tablet PC-based Handhelds

Tablet PC CRSs General Properties: Examples: Support many types of exercises Examples: Classroom Presenter Ubiquitous Presenter DyKnow GroupScribbles 21

Classroom Presenter Devices: Activities: Features: Limits: Tablet PC Laptop Activities: Whole Slide, Ink/Text Features: Anonymous Submissions Push Model Limits: Cost Anderson R. J., Anderson R., Simon B., Wolfman S. A., VanDeGrift T., Yasuhara K. “Experiences with a Tablet PC Based Lecture Presentation System in Computer Science Courses”. SIGCSE '04, pp. 56-60, 2004. Site: http://classroompresenter.cs.washington.edu

Ubiquitous Presenter Devices: Activities: Features: Limits: Web-browser Activities: Whole Slide, Text Multiple-choice Features: Same as Classroom Presenter Limits: Requires Web Server Wilkerson M, Griswold W. G., Simon B. “Ubiquitous Presenter: Increasing Student Access and Control in a Digital Learning Environment”. SIGCSE ’05. 2005. Site: http://up.ucsd.edu/

DyKnow Devices: Activities: Features: Limits: Tablet PC Laptop Whole Slide, Ink/Text Multiple-choice Features: Integrated System Instructor Control Pull Model Limits: Cost DyKnow Website: http://www.dyknow.com

GroupScribbles Devices: Activities: Features: Limits: Tablet PC Laptop Post-It Note, Ink/Text Features: Multi-part Activities Multiple Subs. Editing/Moving Sub. Limits: Cost Brecht J., DiGiano C., Patton C., Tatar D., Chaudhury R., Roschelle J., & Davis K. “Coordinating networked learning activities with a general-purpose Interface”. To appear in the International Review of Research in Open and Distance Learning.

Handheld CRSs General Properties: Examples: Many types of small devices Activities more limited Examples: Clickers Classtalk Pebbles ActiveClass PLS TXT UR Thoughts Ubiquitous Presenter (Mobile) 26

Clickers Devices: Activities: Features: Limits: Custom Multiple-choice Small and Affordable Scales to 100s Limits: Limited Utility HW Used for Wrong Reasons? Pedagogy relies on aggregated results and discussion with neighbor Used for attendance and keeping people awake (these are wrong reasons). Duncan D. “Clickers: A New Teaching Aid with Exceptional Promise”. Astronomy Education Review, 5(1), 2006.

Classtalk Devices: Activities: Features: Limits: PDA Calculator (TI-83) Activities: Multiple-choice Short Answer Numeric Features: Binning (All Responses) Limits: PDA Cost/Popularity Dufresne R. J., Gerace W. J., Leonard W. J., Mestre J. P., Wenk L. “Classtalk: A Classroom Communication System for Active Learning”. Journal of Computing in Higher Education, 7, 3-47, (1996).

Pebbles Devices: Activities: Features: Limits: PDA Multiple-choice Question Asking Features: General Framework PebblesDraw Shared Whiteboard Limits: PDA Cost/Popularity Showed that their technology was preferred to raising hands or cards in class Myers B. A. “Using Hand-Held Devices and PCs Together”. Communications of the ACM. Volume 44, Issue 11. November, 2001. pp. 34 – 41.

ActiveClass Devices: Activities: Features: Limits: PDA Multiple-choice Question Asking Features: Voting on Questions Results Always Avail. Limits: PDA Cost/Popularity Ratto M., Shapiro R. B., Truong T. M., Griswold W. G. “The ActiveClass Project: Experiments in Encouraging Classroom Participation”. Computer Support for Collaborative Learning, 2003.

PLS TXT UR Thoughts Devices: Activities: Features: Limits: Cell Phone Question Asking Features: SMS Communication Anonymous Limits: Recurring SMS Cost SMS Length (160) Markett C., Sanchez I. A., Weber S., Tangney B. “Using short message service to encourage interactivity in the classroom”. Computers & Education, 46 280-293, 2006.

Ubiquitous Presenter (mobile) Devices: Cell Phone Activities: Multiple-choice Short Answer Camera-based Features: SMS  Abbreviations Anonymous Limits: Recurring SMS/MMS Cost Lindquist D., Denning T., Kelly M., Malani R., Griswold W. G., and Simon B. “Exploring the Potential of Mobile Phones for Active Learning in the Classroom”. SIGCSE ’07, March 2007.

Summary of All Systems Tablet PC Laptop PDA Calculator Phone Custom Digital Ink Textual Multiple-choice Whole Slide Post-It ? Asking Short Answer Numeric Tablet PC CP Grp. Scrb. DyKnow Laptop Ubiq. Pres. PDA ActiveClass Classtalk Pebbles Calculator Phone Ubiq. Mob. PLS TXT Custom Clickers Whole Slide and Post-It can be used for ? Asking, Short Answer, Numeric, and Multiple-choice. This just shows the primary types of exercises that they are used for.

Summary of All Systems Tablet PC Laptop PDA Calculator Phone Custom Digital Ink Textual Multiple-choice Whole Slide Post-It ? Asking Short Answer Numeric Tablet PC CP Grp. Scrb. DyKnow Laptop Ubiq. Pres. Grp. Scrb PDA ActiveClass Classtalk Pebbles Calculator Phone Ubiq. Mob. PLS TXT Custom Clickers Whole Slide and Post-It can be used for ? Asking, Short Answer, Numeric, and Multiple-choice. This just shows the primary types of exercises that they are used for.

Part III: Future Work Improve type of exercises on handhelds Integrate diverse handhelds into class

Enabling Adoption? Expressiveness Cost Integrate Existing Devices Digital Ink Expressiveness Increase Mobile Interaction Multiple Choice $ $$$ Cost 36

Challenges “Increasing Mobile Interaction” “Using Existing Devices” Understand types of activities How to do them on mobile devices “Using Existing Devices” Dealing with heterogeneous mix of devices 37

Steps to Tackle Challenges Look at styles of activities supported on different devices esp. what interactions are needed Look at how well each device feature performs on these activity styles Explore techniques for making activity styles feasible on mobile devices and test Develop application that supports multiple device types Develop tool for designing activities to suit multiple devices – design activity once, translate to several devices Develop tool for taking responses of different forms and interpreting/aggregating them 38

Step 1: Activity Styles Taxonomy What: Activity Patterns: Brecht et al., DiGiano et al. Classroom Presenter observations Other Education Work Focus on CRS activities Hierarchical Taxonomy Goal: Benchmark How: Large Corpus of Activities Compile Taxonomy Manually Expand as needed Brecht  41 Activity Patterns (Question Posing, Multiple Representations, Simultaneous Annotation, Image Mapping) DiGiano  Moon Phases Pattern (partial knowledge), mostly deals with communication in small groups I am interested in patterns of activities styles based on the information that instructor wants to gain from the activity.

Step 1: Potential Activity Styles Locate Item On Image Single/Multiple items? Location? Scale? Boundaries? Create a Diagram Elements? Labels? Connectivity? Modify/Complete a Diagram Structure? Highlight a Diagram

Step 2: Existing Techniques What: Given device capabilities, quantize performance Test feasibility of activity for each device Use existing research where avail. Text input speed on different devices Pointing task research Focus on diagrammatic/image responses How: Literature review Implement techniques Lab studies to test performance on benchmarks Transitions to next slide: Let’s look at some of the dimensions that we need to consider for our device First input, Also, screen size

Device Input Capabilities Input Modality (Hardware) Example Mobile Devices Keyboard Laptop, Tablet PC (hybrid) Keypad Mobile Phone Directional Pad (2-way, 4-way, or 8-way) Mobile Phone, PDA, Gaming Device Mouse Laptop (external) Trackball Touchpad (relative motion only) Laptop Joystick Gaming Device Accelerometer Mobile Phone, PDA Stylus/Pen Tablet PC, PDA Touch-screen iPhone, Tablet PC (some) Camera Mobile Phone, iPhone

Device Screen Sizes 43

Step 3: New Techniques What: How: No adequate techniques? Try to invent new ones How: Identify gaps from prev. step Examples: Stylus drawing on small screen w/ large virtual canvas Drawing arbitrary regions without stylus Test performance using lab tests

Step 3: Example Arbitrary Regions Rubber-band?

Step 4: Develop Application What: Base on Classroom Presenter Requirements: Identify device type/capabilities Screen size/resolution Open platforms? Android? How: Use existing standards: Ubiquitous Presenter uses HTML and JavaScript Wifi or SMS for networking Support for: Stylus Devices (Tablet PC, PDA, Smart Phone, Nintendo DS) Touch Screen Keypad Success = deployment & support for devices

Step 5: Instructor Design Tools What: Consider classroom issues with different devices Envy Equality Given benefits/drawbacks of different modalities, Recommend alternative activity types? Balance activity types given? How: Tool Input Instructor activity (PowerPoint?) Activity metadata Tool Internals Determine activity type Optimizes interaction for each device in class Tool Output Recommendations for activity types Evaluate with user-study of tool

Step 6: Instructor Class Usage What: Use activity metadata Use salient regions of responses Understand and aggregate results Finding common misconceptions How: Unsupervised/semi-supervised clustering Evaluation Test data from real classes Measure success on data User-study to observe utility Multi-device data to test on

Step 6: Example Tic-Tac-Toe ? X, X, O O, O, X ==

Conclusions Two Problems Solution Goal Increasing Mobile Interaction Using Existing Devices Solution Activities and devices Tools: Enable multiple devices Help instructor Goal Make deploying CRSs a reality Improve Education

Questions? 51