clickED Field Usability Study February 16, 2016

Experiments Mobile Goals: Test new back-and-forth communication Record exact time and errors numbers

Experiment 1 - Teacher Results Teacher initially tapped the question several times to “address” Preferred the multiple tabs to different colors The refresh button threw the teacher off

Experiments Mobile Goals: Test new back-and-forth communication Record exact time and errors numbers Web Goal: First test of implementation of second platform

Outline Experiment 1 Method and Results Experiment 2 Discussion Summary

Experiment 1 - Participants Ms. W - 8th Grade science teacher at St. Matthew’s in San Mateo, CA Classroom of 20 students

Tasks Student resolves his/her question Teacher addresses a question in class Teacher reviews questions after class

Experiment 1 - Apparatus Students and teacher downloaded app through Testflight Students used their iPads Test took place in classroom during study hall

Experiment 1 - Procedure Consent Forms Q & A with the class Each student asked questions on their own iPad Teacher acted as moderator while I presented Teacher reviewed the questions asked after class

Test Measures Number of errors Time to complete task Distraction level for both students and teachers

Experiment 1 - Student Results 95% students successfully resolved their question when instructed 100% students responded when teacher addressed their question

Experiment 1 - Teacher Results Teacher initially tapped the question several times to “address” Preferred the multiple tabs to different colors

Experiment 1 - Student Feedback 100% class wanted to see other student’s questions 90% of class wanted to have the option to send private questions 100% of class said they wanted the app for their class

Experiment 1 - Teacher Feedback Ability to export all questions asked before her next section Ability to shut down public questions in response to bad behavior

Experiment 2 - Participants Mr. G - 11th Grade English teacher at Washington High School in Fremont, CA Classroom of 15 students

Experiment 2 - Apparatus Teacher used web version of the app Tracked teacher and student screens using Quicktime Player Test took place in classroom during lunch break

Experiment 2 - Procedure Arshin and I did Q&A Teacher used web version to moderate Two stations running Quicktime player were used to track students and the teacher in the app

Experiment 2 - Student Results Time # of Errors Student 1 N/A 4 Student 2 3 Student 3 8 seconds 2

Tale of Two Classrooms Time # of Errors Student 1 N/A 4 Student 2 3 8 seconds 2

Experiment 1 - Teacher Results Teacher initially tapped the question several times to “address” Preferred the multiple tabs to different colors The refresh button threw the teacher off

Experiment 2 - Student Feedback 100% participants wanted to add notes to questions 100% participants thought they could tap into the question

Experiment 2 - Teacher Feedback Class snapshot over list of questions Visualization of class confusion per question Filter by “Confused” students

Discussion - Mobile Changes Add ability for students to see other students’ questions and upvote them Switch from text bubbles to card layout Add affordance for swipe to resolve Update color scheme

Discussion - Web Changes Add “Snapshot” class visualization Filter by “confused” students Data export Optimize for mobile web

Discussion - Study Improvements Film the classroom Have students ask questions during a real class Platform agnostic

Summary Ran a mobile app test in San Mateo and a web app test in Fremont Add public questions and upvoting on mobile Swipe is not clear Make web more visual