Demonstrating the Viability of Automatically Generated User Interfaces Jeffrey Nichols, Duen Horng Chau, Brad A. Myers IBM Almaden Research Center and Human Computer Interaction Institute, Carnegie Mellon University 25 th Conference on Human Factors in Computing Systems (CHI) May 3, 2007

1 History of Automated Design ITS, Wiecha 1990 UIDE and Cartoonist, Sukaviriya 1994 Jade, Vander Zanden 1990 Humanoid, Szekely 1992

2 New Automated Design Systems Supple, Gajos 2004 PUC and Uniform, Nichols 2006 Xweb, Olsen 2000 iCrafter, Ponnekanti 2001 INCITS/V2 Standard

3 Viability… Automatically generated interfaces must improve on those available today (subject to the same constraints) Development and manufacturing costs for supporting the automatically generated interfaces must not exceed costs of todays technology

4 Personal Universal Controller Personal Mobile Device Abstract Specification Control State Feedback Automatically Generated Interface PUC [Nichols, UIST 02]

5 Uniform Copier A Copier B Original (PUC) Interfaces [Nichols, CHI 06]

6 Consistency enabled – Copier A used firstOriginal (PUC) Interfaces Uniform Copier A Copier B Consistent Copier B Original Copier B [Nichols, CHI 06]

7 In This Talk I will present a study of the PUC system with two comparisons: Existing hand-designed interfaces with automatically generated interfaces Automatically generated interfaces with and without consistency Argue that cost of generating these interfaces is competitive with existing design costs

8 Outline Introduction Interfaces Study Procedure Results Discussion and Future Work

9 Appliances for Study Canon PIXMA MP780 HP Photosmart 2610

10 Specifications Written by different people to simulate differences due to manufacturer Faithful to design of the appliance Took advantage of specification language features, e.g. multiple labels Initial drafts produced in 2-3 days Debugging took another 2-3 days Similarity information for generation of consistent interfaces was created in several hours Quite complex: HP Printer 85 variables and commands 1924 lines of XML Canon Printer 134 variables and commands 2949 lines of XML HP printerCanon printer

11 Interfaces HP printer without consistency Canon printer without consistency Canon printer consistent with HP

12 Interfaces – Full View HP printerCanon printerConsistent Canon printer

13 Outline Introduction Interfaces Study Procedure Results Discussion and Future Work

14 Study Evaluation of Generated Interfaces Users perform 8 tasks with two all-in-one printer interfaces (HP and Canon) Two comparisons Physical interface to PUC interface Without consistency and with consistency 48 participants (28 male, 20 female) Divided into 6 groups, 8 per group Recruited through CMU service Metrics Completion time Failures HP printerCanon printer

15 Tasks 1. Send a fax to the number stored in the third speed dial 2. Configure the fax function so that it will always redial a number that is busy 3. Configure the fax function so that any document received that is larger than the default paper size will be resized to fit the default 4. Configure the fax function so that it will only print out an error report when it has a problem receiving a fax 5. Make two black-and-white copies of the document that has already been placed on the scanner of the printer 6. Imagine you find the copies too dark. Improve this by changing one setting of the device 7. Given a page with a picture, determine how to produce one page with several instances of the same picture repeated (demonstrated with actual paper copies) 8. The device remembers the current date and time. Determine where in the interface these values can be changed (do not change them)

16 Procedure 1. Subject performs all 8 tasks on appliance #1 (HP or Canon) Five minutes maximum allowed for each task 2. Subject is instructed on the optimal way to perform each task Subject performs the task again until completed correctly. Additional instruction given as necessary 3. Subject performs all 8 tasks on appliance #2 (Canon or HP)

17 Three Interface Conditions Built-In User sees only physical appliance interfaces Physical HP Physical Canon Physical Canon Physical HP AutoGen User sees only PUC interfaces without consistency PUC HP PUC Canon PUC Canon PUC HP Consistent User sees a PUC interface without consistency followed by a Uniform interface generated to be consistent with the previous PUC interface PUC HP Uniform Canon generated to be consistent with HP PUC Canon Uniform HP generated to be consistent with Canon

18 Conditions 3 interface conditions * 2 appliance orderings = 6 groups #1 - HP#2 - Canon Built-In Physical AutoGen PUC Consistent PUCUniform #1 - Canon#2 - HP Built-In Physical AutoGen PUC Consistent PUCUniform

19 Comparing Usability Focus primarily on the first set of tasks First set not affected by earlier tasks or instruction #1 - HP#2 - Canon Built-In Physical AutoGen PUC Consistent PUCUniform #1 - Canon#2 - HP Built-In Physical AutoGen PUC Consistent PUCUniform

20 Usability Results #1 - HP#2 - Canon Built-In Physical AutoGen PUC Consistent PUCUniform #1 - Canon#2 - HP Built-In Physical AutoGen PUC Consistent PUCUniform * = p < 0.05 = p < 0.1 ** Failures per subject also significantly less for PUC than Physical (Fishers Exact Test, p < 0.05) HP: Built-In vs PUC Canon: 2.0 Built-In vs PUC *********

21 Comparing Consistency Focus only on the second set of tasks Previous experience during study should influence results #1 - HP#2 - Canon Built-In Physical AutoGen PUC Consistent PUCUniform #1 - Canon#2 - HP Built-In Physical AutoGen PUC Consistent PUCUniform

22 Consistency Results #1 - HP#2 - Canon Built-In Physical AutoGen PUC Consistent PUCUniform #1 - Canon#2 - HP Built-In Physical AutoGen PUC Consistent PUCUniform * = p < 0.05 = p < 0.1 * * * *** Very few failures in these conditions: 4 total for all tasks performed by 32 subjects (256 tasks, 1.5% failure rate)

23 Outline Introduction Interfaces Study Procedure Results Discussion and Future Work

24 Discussion Fairness of Comparison Why not compare with hand-designed interfaces on PDA? Cost Many users already have a device capable of acting as remote control, so cost of device should not be counted. Main per-appliance development cost comes from writing specification, which is likely less than developing a full interface. Manufacturing cost may increase slightly due to need for wireless communication capability, though it may be possible to compensate by removing unneeded interface hardware. Limitations of Study We studied only two variants of one type of appliance

25 Conclusion Results: PUC interfaces were more usable than existing appliance interfaces The PUCs consistency algorithms created interfaces that were even better when learning to use a new appliance with familiar functionality This suggests that automated design can provide better user interfaces in situations where interfaces are constrained by external factors or individual user customization can provide substantial benefits Future Work What other types of auto-gen features would be useful? Can we automatically modify existing hand-designed interfaces to add features like consistency?

26 Acknowledgements Thesis Committee Brad A. Myers (chair) Scott Hudson John Zimmerman Dan Olsen Jr. Funding National Science Foundation Microsoft General Motors Intel Pittsburgh Digital Greenhouse Equipment Grants Mitsubishi (MERL) VividLogic Lucent Lutron Lantronix Nokia PUC Project Members Brandon Rothrock Duen Horng Chau Kevin Litwack Thomas K. Harris Michael Higgins Joseph Hughes Roni Rosenfeld Rajesh Seenichamy Pegeen Shen Htet Htet Aung Mathilde Pignol Suporn Pongnumkul Stefanie Shriver Jeffrey Stylos Peter Lucas Collaborators & Friends Naomi Ramos Desney Tan Daniel Avrahami Gaetano Borriello Laura Dabbish Andrew Faulring James Fogarty Krzysztof Gajos Darren Gergle Andy Ko Amy Nichols Mick Nichols Sally Nichols Trevor Pering Fleming Seay Irina Shklovski Roy Want Jake Wobbrock and many others…

Thanks for listening! For more information…

29 Questions Were users at all affected by a lack of spit/polish in the interface designs, or will users put up with poorer looking interfaces that are otherwise more usable? –Stina Nylander Have you done any studies to evaluate how easily people can learn to use the PUC specification language? –Unknown How do you deal with issue of interface branding? Without branding support, will manufacturers support your technology. – Nathan Freier