1. Predicting Task Execution Time on Handheld Devices Using the Keystroke Level Model Annie Lu Luo and Bonnie E. John School of Computer Science Carnegie Mellon University CHI’05 – April 6, 2005

2. © Annie Luo, Carnegie Mellon University, 2005 slide 2 Motivation and goals Keystroke Level Model (KLM) A priori prediction of expert user task time Intensively used on desktop computers Not yet been adapted to handheld devices Limited display size Input device: stylus, touch-screen, hardware buttons Interaction methods: tap, Graffiti, etc. Investigate KLM on handheld UIs Applicability of model to novel interface modalities Accuracy of model predictions

3. © Annie Luo, Carnegie Mellon University, 2005 slide 3 KLM in brief Describe a task by placing operators in a sequence K – keystroke (tap) P – point with mouse (stylus) H – homing (move hand from mouse to keyboard) (N/A) D (takes parameters) – drawing (N/A) R (takes parameters) – system response time M – mental preparation G – Graffiti stroke (580 ms – Fleetwood, et al 2002) Five heuristic rules to insert candidate Ms into the sequence Task execution time = Σ all operators involved

4. © Annie Luo, Carnegie Mellon University, 2005 slide 4 Start Handheld task: Find information about the MET 1 City map Museums list 2 Soft keyboard 4 Scroll list 3 Graffiti Region map Street map Query result

5. © Annie Luo, Carnegie Mellon University, 2005 slide 5 Create KLMs One KLM for each of the four methods Used CogTool (John, et al 2004) MacroMedia DreamWeaver Behavior Recorder Netscape HTML event handler ACT-R Environment Modeler mocks up interfaces as HTML storyboard Modeler demonstrates tasks on the HTML storyboard HTML mockups Interface event messages via LiveConnect ACT-Simple code based on KLM KLM Trace ACT-Simple complies code into ACT-R production rules

6. © Annie Luo, Carnegie Mellon University, 2005 slide 6 Mozilla Firefox Behavior Recorder ACT-R

7. © Annie Luo, Carnegie Mellon University, 2005 slide 7 User study 10 expert PDA users (Female:Male = 3:7) At least one year experience using: Palm series, pocket PC, or smart cell phone Instructed to perform the task on a PalmVx Using four different methods (within subject design) Training session before real session Repeating each method for 10 times Data collection EventLogger: records system events to a log file Videotaped modeler’s behavior for verification

8. © Annie Luo, Carnegie Mellon University, 2005 slide 8 New results since paper published -9.3% 8.9% 5.8% 7.7% Latest version of CogTool Better estimation of system response time Detailed analysis of model and user traces (140/400 removed) 2.3% -1.4% -6.9% -3.7%

9. © Annie Luo, Carnegie Mellon University, 2005 slide 9 Conclusion & Future work KLMs produced with CogTool are effective for handheld user interfaces: Produces accurate execution time prediction Supports new input modalities: Graffiti Future work: Detailed analysis of the user pauses (mental time) Use predictions of pauses to assist energy management

10. © Annie Luo, Carnegie Mellon University, 2005 slide 10 Thank you! Authors’ contact info: Bonnie John – bej@cs.cmu.edu Annie Luo – luluo@cs.cmu.edu The CogTool project: http://www.cs.cmu.edu/~bej/cogtool/

11. © Annie Luo, Carnegie Mellon University, 2005 slide 11 Participants information (backup) User (gender)Device ownedHow long 1 (M)Palm Vx5+ years 2 (M)Compaq iPAQ3 years 3 (M)Palm IIIe4 years 4 (M)Handspring Visor3 years 5 (F)Handspring visor Pro2 years 6 (F)Dell PDA1 year 7 (M)iPAQ 36304 years 8 (M)Kyocera 71354+ years 9 (M)Handspring Visor Prism3 years 10 (F)Palm VA3 years

12. © Annie Luo, Carnegie Mellon University, 2005 slide 12 Results in paper (backup) (Average)

13. © Annie Luo, Carnegie Mellon University, 2005 slide 13 New results since paper published 9.3% -8.9%-5.8% -7.7% Better measurements of system response time Removed error trials (140 out of 400) Latest version of CogTool

14. © Annie Luo, Carnegie Mellon University, 2005 slide 14 Interface Widgets: - Buttons - Check boxes - Text fields - Pull-down lists - Links - Menus - Audio input - Audio output

15. © Annie Luo, Carnegie Mellon University, 2005 slide 15 Netscape Behavior Recorder ACT-Simple