User Perceptions of Drawing Logic Diagrams with

Pen-Centric User Interfaces

Bo Kang, Jared N. Bott, and Joseph J. LaViola Jr.

Interactive Systems & User Experience LabDepartment of EECS

University of Central Florida

Outline

• Related Work• Motivation• Experiment• Results• Discussion• Conclusion

Related Work

• Pen-based Interfaces– DENIM (Lin et al. 2000)– CrossY (Apitz et al. 2004)– LogicPad (Kang and LaViola 2012)

• Evaluation and Perceptions– MacKenzie et al. (1991)– Wais et al. (2007)– Forsberg et al. (2008)– Vatavu et al. (2011)– Bott et al. (2011)

Motivation

• LogicPad– Hybrid interface for

Boolean logic problems

– Seemed faster than sketching

• Is speed more important for these diagrams?

Experiment

• Performed study comparing 3 pen-based interfaces for creating Boolean logic diagrams

SketchPure sketch, 100% accurate, “ideal”

Drag-and-DropTraditional WIMP-based, stylus and keyboard

HybridRadial menu for gates, sketch labels and wires

Subjects and Apparatus

• 18 college students participated– 3 female, 15 male– Ages 19 – 30

• Worked on tablet PC– HP EliteBook 2760p

Experimental Task

• 3 copy-and-verify tasks (one per interface)– 6 problems per task

• Given a diagram-equation pair

• Copy a diagram using interface, get a Boolean equation back

• Compare given equation with one from interface

Experimental Design

• Wizard of Oz approach– All 3 interfaces

programmed with ordering of tasks, which equation to show

• 3 by 2 within-subjects factorial design– Independent variables:

user interface (sketch, drag-and-drop, hybrid) and diagram complexity (low, high)

– Dependent variable: completion time

Metrics

• Measured completion time

• Rate each interface– Making gates– Making wires– Making labels– Arrange gates– Create diagrams– Speed– Frustration

• Rank interfaces– Ease of use– Speed– Naturalness– Overall preference

Hypotheses

• Primary: Participants will prefer the sketch interface over the hybrid and drag-and-drop interfaces

• Secondary– Hybrid interface will be faster than the

sketch and drag-and-drop interfaces– Sketch interface will be rated more

natural than the hybrid and drag-and-drop interfaces

Results - Rankings

Results – Completion TimeInterface Drag-and-Drop Hybrid Sketch

Complexity

σ σ σ

Low 92.9 19.5 70.8 12.1 70.0 14.8

High 226.4 38.7 186.6 33.3 202.4 34.6

Overall 159.7 74.1 128.7 63.7 136.2 72.1

• T-tests on completion time– Sketch faster than drag-and-drop– Hybrid faster than drag-and-drop– Hybrid faster than sketch, except at low

complexity (no significance)

Results – Ratings

• Significant tests– Ease of use in labeling, arranging, and

creating diagrams

• Easy label: sketch > hybrid > drag-and-drop

• Easy arrange: drag-and-drop > sketch

• Easy diagram: hybrid > drag-and-drop

Results – Hypotheses

• Primary hypothesis – Did they prefer sketch interface?– No

• Was sketch most natural?– Yes

• Was hybrid fastest?– Yes…

Discussion

• Speed and user perceptions– Difference in rankings/ratings and

completion time–Why?• No task switching with sketch interface• Internal versus external mistakes• Drawing style slows down sketching

– No easy way to spatial arrange drawing

– 100% sketch accuracy not as fast as hybrid

Discussion – cont.

• Why rank an interface as best overall?– Spearman’s rank correlation between

overall ranking and other rankings and ratings

– Highest correlations with ease of use ranking, naturalness ranking, speed ranking

– Sketch “was fast for small diagrams”– Sketch “was easy and natural”– Hybrid “easier than the others”

Conclusion

• Would users prefer a sketch interface over a faster interface?– Study comparing three pen-based interfaces for

creating logic diagrams– Sketch was well-liked, but not decisively so

• User perceptions and measurements– Perception of speed and our measurement differed

• Should we continue research into pen-based interfaces for structured 2D languages?– Yes

• Pure sketch might not be the most powerful, but clearly desirable traits

Acknowledgments

• This work is supported in part by NSF CAREER award IIS-0845921 and NSF awards IIS-0856045 and CCF-1012056.

QUESTIONS?

Bo Kang: bkang@cs.ucf.eduJared N. Bott: jbott@cs.ucf.eduJoseph J. LaViola Jr.: jjl@eecs.ucf.edu

RatingsInterface Drag-and-drop Sketch Hybrid

Statement σ σ σ

Easy gate 5.72 1.447 6.17 0.924 6.39 0.698

Easy wire 5.00 1.680 5.89 1.231 5.94 1.305

Easy label 4.67 1.815 6.89 0.323 6.33 1.085

Easy arrange

6.22 0.878 4.56 1.854 5.78 1.263

Easy diagram

5.39 1.243 5.67 1.138 6.28 0.752

Quick 5.61 1.335 5.83 1.249 6.11 1.023

Frustrating 3.06 1.731 2.50 1.249 2.11 1.023

CorrelationsCorrelation with Overall Ranking

ρ p

Ease of Use 0.778 0.000

Naturalness 0.694 0.000

Speed 0.639 0.000

Frustration 0.411 0.002

Easy Diagram -0.409 0.002

Easy Label -0.397 0.003

Completion Time 0.397 0.003

Quick -0.342 0.011

Easy Gate -0.314 0.021

Easy Wire -0.281 0.039

Easy Arrange 0.056 0.690