An Empirical Evaluation of Undo Mechanisms Aaron G. Cass Chris S.T. Fernandes Andrew Polidore...

An Empirical Evaluation of Undo Mechanisms

Aaron G. CassChris S.T. Fernandes

Andrew Polidore

Computer Science Dept.Union College

Schenectady, NY, USA

2 of 18

Talk Outline

• Types of undo

• Problem statement

• Empirical evaluation design– Pilot study– Broad evaluation

• Results

• Future Work

• Conclusion

3 of 18

Linear Undo

4 of 18

Selective Undo

• Undo arbitrary action without requiring the undoing of subsequent steps [Berlage ’94]

5 of 18

Selective Undo

• Script model [Archer, et al ’84]

– Removing action Ai results in state:

– A1, A2, …, Ai-1, Ai+1, …, An

• Cascading model [Cass, Fernandes ’05]

– Accounts for dependencies between actions

– If Aj dependent on existence or result of Ai

(j > i), Aj also undone

Differing semantics:

Problem Statement• Selective undo is believed to be superior

to linear undo

But…

Will it be easier for users to use?

Is this a natural model for undo?

Does it match users’ mental models?

7 of 18

Goal

• Compare three different models of undo– Linear– Script selective– Cascade selective

Conduct a user study to find out!

8 of 18

The Study

• Want to determine users’ mental model of undo

• Design a task that elicits their mental model in the performance of that task

9 of 18

The Instrument

• Paper-based instrument– Not associated with a

computer application– User freedom– Selective undo not

widely implemented

10 of 18

The InstrumentSteps taken to create this picture:

1. Draw Circle2. Draw Square3. Draw Triangle4. Color Circle

Assume you have already done the steps above. Draw what you think the outcome should be if you were to undo step 4 (step 1).

11 of 18

Two Tasks

1. Draw Circle2. Draw Square3. Draw Triangle4. Color Circle

Undo Step 1

Linear ScriptCascade

Undo Step 4

All models

12 of 18

Pilot Study

• Concern that word “undo” may trigger response based on past experience

• Compare with “reverse the effects of”• 4 subjects

• Result: kept “undo” in instrument wording

13 of 18

Evaluation Methods

• 29 subjects– Undergraduate population– Experience with widespread applications– Little experience with specialized applications

• Within-subjects design

• Post-task questionnaire

• Single dependent variable2 analysis

14 of 18

Results (N=28)

linear script cascade other

1 (4%)

7 (25%)

18 (64%)

2 (7%)

15 of 18

2 Analysis

• Expected one model to be preferred

p<0.05 linear script cascade other

linear script cascade other

• Expected cascade to be preferred over linear

p<0.05

16 of 18

2 Analysis• Expected script to be

preferred over linear

2 not applicable

• Expected either script or cascade to be preferred over the other

p<0.05

linear script cascade other

linear script cascade other

17 of 18

Conclusions

• In familiar applications– Cascading selective is more natural than

script selective– Cascading selective is more natural than

linear– Script is more natural than linear

(the data suggests)

18 of 18

Future Work

• Use a more complete application– Pilot study with presentation software already

complete

• Use a richer set of dependencies

• Implement selective undo in a representative application