An Empirical Evaluation of Undo Mechanisms Aaron G. Cass Chris S.T. Fernandes Andrew Polidore Computer Science Dept. Union College Schenectady, NY, USA

2 of 18 Talk Outline Types of undo Problem statement Empirical evaluation design –Pilot study –Broad evaluation Results Future Work Conclusion

3 of 18 Linear Undo

4 of 18 Selective Undo Undo arbitrary action without requiring the undoing of subsequent steps [Berlage ’94]

5 of 18 Selective Undo Script model [Archer, et al ’84] –Removing action A i results in state: –A 1, A 2, …, A i-1, A i+1, …, A n Cascading model [Cass, Fernandes ’05] –Accounts for dependencies between actions –If A j dependent on existence or result of A i (j > i), A j also undone Differing semantics:

Problem Statement Selective undo is believed to be superior to linear undo But…  Will it be easier for users to use?  Is this a natural model for undo?  Does it match users’ mental models?

7 of 18 Goal Compare three different models of undo –Linear –Script selective –Cascade selective Conduct a user study to find out!

8 of 18 The Study Want to determine users’ mental model of undo Design a task that elicits their mental model in the performance of that task

9 of 18 The Instrument Paper-based instrument –Not associated with a computer application –User freedom –Selective undo not widely implemented

10 of 18 The Instrument Steps taken to create this picture: 1. Draw Circle 2. Draw Square 3. Draw Triangle 4. Color Circle Assume you have already done the steps above. Draw what you think the outcome should be if you were to undo step 4 (step 1).

11 of 18 Two Tasks 1. Draw Circle 2. Draw Square 3. Draw Triangle 4. Color Circle Undo Step 1 LinearScriptCascade Undo Step 4 All models

12 of 18 Pilot Study Concern that word “undo” may trigger response based on past experience Compare with “reverse the effects of” 4 subjects Result: kept “undo” in instrument wording

13 of 18 Evaluation Methods 29 subjects –Undergraduate population –Experience with widespread applications –Little experience with specialized applications Within-subjects design Post-task questionnaire Single dependent variable  2 analysis

14 of 18 Results (N=28) 1 (4%) 7 (25%) 18 (64%) 2 (7%)

15 of 18  2 Analysis Expected one model to be preferred p<0.05 Expected cascade to be preferred over linear p<0.05

16 of 18  2 Analysis Expected script to be preferred over linear   2 not applicable Expected either script or cascade to be preferred over the other p<0.05

17 of 18 Conclusions In familiar applications –Cascading selective is more natural than script selective –Cascading selective is more natural than linear –Script is more natural than linear (the data suggests)

18 of 18 Future Work Use a more complete application –Pilot study with presentation software already complete Use a richer set of dependencies Implement selective undo in a representative application

19 of 18 Validity threats –Scalability To complex tasks To other applications