11
HAL Id: hal-00976507 https://hal.inria.fr/hal-00976507 Submitted on 9 Apr 2014 HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Running an HCI Experiment in Multiple Parallel Universes Pierre Dragicevic, Fanny Chevalier, Stéphane Huot To cite this version: Pierre Dragicevic, Fanny Chevalier, Stéphane Huot. Running an HCI Experiment in Multiple Parallel Universes. 2014, pp.607-618. 10.1145/2559206.2578881. hal-00976507

Running an HCI Experiment in Multiple Parallel Universes

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

HAL Id hal-00976507httpshalinriafrhal-00976507

Submitted on 9 Apr 2014

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents whether they are pub-lished or not The documents may come fromteaching and research institutions in France orabroad or from public or private research centers

Lrsquoarchive ouverte pluridisciplinaire HAL estdestineacutee au deacutepocirct et agrave la diffusion de documentsscientifiques de niveau recherche publieacutes ou noneacutemanant des eacutetablissements drsquoenseignement et derecherche franccedilais ou eacutetrangers des laboratoirespublics ou priveacutes

Running an HCI Experiment in Multiple ParallelUniverses

Pierre Dragicevic Fanny Chevalier Steacutephane Huot

To cite this versionPierre Dragicevic Fanny Chevalier Steacutephane Huot Running an HCI Experiment in Multiple ParallelUniverses 2014 pp607-618 10114525592062578881 hal-00976507

Running an HCI Experimentin Multiple Parallel Universes

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

P Dragicevic P Dragicevic P Dragicevic P DragicevicF Chevalier F Chevalier F Chevalier F Chevalier

S Huot S Huot S Huot S HuotRunning an HCI Experiment in Multiple Parallel Universes

In altchi (CHI rsquo14) Extended Abstracts of the 32nd International Conferenceon Human Factors in Computing Systems ACM April 2014

Authors Version

AbstractWe experimentally evaluated a haptic touch slider in 8parallel universes The results were overall similar butexhibited surprisingly high variability in terms of statisticalsignificance patterns We discuss the general implicationsof these findings for empirical HCI research

Author KeywordsEvaluation Replication Multiverse NHST p-value

ACM Classification KeywordsH52 [User Interfaces] Evaluationmethodology

IntroductionScientific knowledge in HCI largely builds on empiricalstudies But in a world where time funding and access toparticipants are limited researchers are often left withrunning studies only once on a few subjects Fortunatelythe existence of a multiverse [1] allows to parallelizeresearch efforts and alleviate these practical constraints

A multiverse experiment was conducted to assess thebenefits of haptic feedback on touch sliders Eachexperiment was conducted and analyzed separately in adifferent parallel universe using the same methods and bythe ldquosamerdquo investigators We first provide the eightindependent reports then propose a general discussion

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 3 Time by Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 22ndash36 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique but no significant effect ofDifficulty and no TechniquetimesDifficulty interaction(see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 127336 00044Difficulty 111 27084 01281TechniquetimesDifficulty 111 40402 00696

Our Anova analysis therefore confirms that technique HSyields significantly shorter completion times thantechnique S overall ie all task difficulties confoundedThe average Time is 109s for S and 104s for HS (seeFigure 3) This difference corresponds to a 48 increasein speed for technique HS compared to technique S

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(48 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Surprisingly we found no significant effect of Difficulty

overall so our hypothesis (H2) is not confirmed Thiscould be explained by the fact that differences in targetdifficulty were not large enough to significantly affectperformance We could have used different target sizesbut the limited input resolution of the device prevented usfrom using much smaller targets Conversely a very largetarget would occupy most of the slider range which doesnot capture realistic slider tasks Overall it seems that forsliders target size is not a crucial factor

To summarize our study provides strong evidence for thebenefits of tactile feedback when operating slidersAlthough moderate the effect of technique was found tobe highly significant Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Thisallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficulty

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (3 female) familiar with touch devicesaged 20ndash37 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of both Technique and Difficulty but nosignificant TechniquetimesDifficulty interaction effect (seeTable 1)

Table 1 Anova table

Source df F SigTechnique 111 51139 00450Difficulty 111 62892 00291TechniquetimesDifficulty 111 13669 02671

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 116s for S vs 110s forHS a 55 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 125s for Hard vs 101s for Easy corresponding to a 238 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(55 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a significant difference betweenthe two levels of difficulty all techniques confounded withEasy being as much as 238 faster than Hard Therefore our hypothesis (H2) is also supported Wederived our difficulty levels based on extensive pilotstudies so as not to favor any technique Our resultsvalidate our experimental design and confirm that targetsize is an adequate metric for task difficulty HS appearsto perform comparably well under two widely differenttask difficulties suggesting that its advantages may wellgeneralize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Difficulty

Av

erag

e T

ime

(s)

095

100

105

110

115

120

125

130

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash32 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a highly significant effect ofDifficulty with also a highly significantTechniquetimesDifficulty interaction effect (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 32748 00977Difficulty 111 142324 00031TechniquetimesDifficulty 111 149541 00026

Our analysis confirms the effect of difficulty (avg TimesEasy=098s Hard=125s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=096s HS=100s p = 01416) anda highly significant difference between techniques forHard with a 92 increase in speed with HS (avgTimes S=130s HS=119s p = 00069) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 92 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

105

110

115

120

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 21ndash50 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 72144 00212Difficulty 111 41479 00665TechniquetimesDifficulty 111 55941 00375

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 112s for S vs 106s forHS a 57 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=105s HS=103s p = 04065)and a highly significant difference between techniques forHard with a 82 increase in speed with HS (avgTimes S=119s HS=110s p = 00060) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(57 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (82 faster)In contrast the improvement was lower (19) under theEasy condition (also see Figure 3) One explanation isthat in the Hard condition the ldquofat fingerrdquo probleminterferes with the corrective phase of usersrsquo movementMultimodal feedback likely alleviates this by providingnon-visual guidance Under the Easy condition the targetwas larger and the fat finger issue not as pronouncedmaking haptic feedback still useful but less critical

Surprisingly we were not able to find a significant effectof Difficulty overall despite the trends visible inFigure 3 This could be explained by the fact thatdifferences in the target difficulty were not large enough tosignificantly affect performance In our pilot studies weconsidered tasks involving much smaller or much largertargets but dismissed them as unrealistic So it seemsthat overall target size is not a crucial factor for sliders

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Operatingsliders is hard on touch devices in general but even moreso when fine control is needed due to the ldquofat fingerrdquoproblem We show that haptic guidance greatly facilitatesthis task Overall based on our results we recommendthe use of sliders with haptic detents on touch devicesespecially when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

1000

1025

1050

1075

1100

1125

1150

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash39 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 60536 00317Difficulty 111 10392 03299TechniquetimesDifficulty 111 94480 00106

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 108s for S vs 101s forHS a 69 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=101s HS=101s p = 09601)and a highly significant difference between techniques forHard with a 129 increase in speed with HS (avgTimes S=114s HS=101s p = 00071) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(69 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (as much as129 faster) In contrast the two techniques seem toperform very similarly under the Easy condition (seeFigure 3) One explanation is that in the Hard conditionusers are deprived of visual cues during the correctivephase of their movement because of the ldquofat fingerrdquoproblem Multimodal feedback likely alleviates this byproviding non-visual guidance Under the Easy conditionthe target may have been large enough for users to rely onvisual feedback only making haptic feedback superfluous

Surprisingly we were not able to find a significant effect ofDifficulty overall A tentative explanation can be foundin Figure 3 while S seems to be affected by difficulty HSexhibits a stable performance across difficulty levels Thissuggests that with haptic feedback all targets are equallyeasy Although this seems to contradict Fittsrsquo Law recallthis law is about aimed movements with visual feedbackThe haptic channel may not be as sensitive to target sizepossibly due to the fact that it is a sensory modalitydirectly connected with kinesthetic and motor functions

To summarize our study shows that adding tactilefeedback in the form of simulated detents facilitates theprecise manipulation of sliders Precise control of sliders ischallenging on touch devices partly due to the ldquofat fingerrdquoproblem We show that with haptic guidance it becomespractically as easy as coarse control Overall based on ourresults we recommend the use of sliders with hapticdetents on touch devices when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

Running an HCI Experimentin Multiple Parallel Universes

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Pierre DragicevicInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Fanny ChevalierInria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

Stephane HuotUniv Paris Sud CNRS Inria

P Dragicevic P Dragicevic P Dragicevic P DragicevicF Chevalier F Chevalier F Chevalier F Chevalier

S Huot S Huot S Huot S HuotRunning an HCI Experiment in Multiple Parallel Universes

In altchi (CHI rsquo14) Extended Abstracts of the 32nd International Conferenceon Human Factors in Computing Systems ACM April 2014

Authors Version

AbstractWe experimentally evaluated a haptic touch slider in 8parallel universes The results were overall similar butexhibited surprisingly high variability in terms of statisticalsignificance patterns We discuss the general implicationsof these findings for empirical HCI research

Author KeywordsEvaluation Replication Multiverse NHST p-value

ACM Classification KeywordsH52 [User Interfaces] Evaluationmethodology

IntroductionScientific knowledge in HCI largely builds on empiricalstudies But in a world where time funding and access toparticipants are limited researchers are often left withrunning studies only once on a few subjects Fortunatelythe existence of a multiverse [1] allows to parallelizeresearch efforts and alleviate these practical constraints

A multiverse experiment was conducted to assess thebenefits of haptic feedback on touch sliders Eachexperiment was conducted and analyzed separately in adifferent parallel universe using the same methods and bythe ldquosamerdquo investigators We first provide the eightindependent reports then propose a general discussion

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 3 Time by Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 22ndash36 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique but no significant effect ofDifficulty and no TechniquetimesDifficulty interaction(see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 127336 00044Difficulty 111 27084 01281TechniquetimesDifficulty 111 40402 00696

Our Anova analysis therefore confirms that technique HSyields significantly shorter completion times thantechnique S overall ie all task difficulties confoundedThe average Time is 109s for S and 104s for HS (seeFigure 3) This difference corresponds to a 48 increasein speed for technique HS compared to technique S

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(48 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Surprisingly we found no significant effect of Difficulty

overall so our hypothesis (H2) is not confirmed Thiscould be explained by the fact that differences in targetdifficulty were not large enough to significantly affectperformance We could have used different target sizesbut the limited input resolution of the device prevented usfrom using much smaller targets Conversely a very largetarget would occupy most of the slider range which doesnot capture realistic slider tasks Overall it seems that forsliders target size is not a crucial factor

To summarize our study provides strong evidence for thebenefits of tactile feedback when operating slidersAlthough moderate the effect of technique was found tobe highly significant Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Thisallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficulty

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (3 female) familiar with touch devicesaged 20ndash37 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of both Technique and Difficulty but nosignificant TechniquetimesDifficulty interaction effect (seeTable 1)

Table 1 Anova table

Source df F SigTechnique 111 51139 00450Difficulty 111 62892 00291TechniquetimesDifficulty 111 13669 02671

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 116s for S vs 110s forHS a 55 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 125s for Hard vs 101s for Easy corresponding to a 238 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(55 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a significant difference betweenthe two levels of difficulty all techniques confounded withEasy being as much as 238 faster than Hard Therefore our hypothesis (H2) is also supported Wederived our difficulty levels based on extensive pilotstudies so as not to favor any technique Our resultsvalidate our experimental design and confirm that targetsize is an adequate metric for task difficulty HS appearsto perform comparably well under two widely differenttask difficulties suggesting that its advantages may wellgeneralize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Difficulty

Av

erag

e T

ime

(s)

095

100

105

110

115

120

125

130

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash32 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a highly significant effect ofDifficulty with also a highly significantTechniquetimesDifficulty interaction effect (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 32748 00977Difficulty 111 142324 00031TechniquetimesDifficulty 111 149541 00026

Our analysis confirms the effect of difficulty (avg TimesEasy=098s Hard=125s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=096s HS=100s p = 01416) anda highly significant difference between techniques forHard with a 92 increase in speed with HS (avgTimes S=130s HS=119s p = 00069) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 92 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

105

110

115

120

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 21ndash50 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 72144 00212Difficulty 111 41479 00665TechniquetimesDifficulty 111 55941 00375

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 112s for S vs 106s forHS a 57 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=105s HS=103s p = 04065)and a highly significant difference between techniques forHard with a 82 increase in speed with HS (avgTimes S=119s HS=110s p = 00060) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(57 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (82 faster)In contrast the improvement was lower (19) under theEasy condition (also see Figure 3) One explanation isthat in the Hard condition the ldquofat fingerrdquo probleminterferes with the corrective phase of usersrsquo movementMultimodal feedback likely alleviates this by providingnon-visual guidance Under the Easy condition the targetwas larger and the fat finger issue not as pronouncedmaking haptic feedback still useful but less critical

Surprisingly we were not able to find a significant effectof Difficulty overall despite the trends visible inFigure 3 This could be explained by the fact thatdifferences in the target difficulty were not large enough tosignificantly affect performance In our pilot studies weconsidered tasks involving much smaller or much largertargets but dismissed them as unrealistic So it seemsthat overall target size is not a crucial factor for sliders

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Operatingsliders is hard on touch devices in general but even moreso when fine control is needed due to the ldquofat fingerrdquoproblem We show that haptic guidance greatly facilitatesthis task Overall based on our results we recommendthe use of sliders with haptic detents on touch devicesespecially when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

1000

1025

1050

1075

1100

1125

1150

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash39 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 60536 00317Difficulty 111 10392 03299TechniquetimesDifficulty 111 94480 00106

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 108s for S vs 101s forHS a 69 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=101s HS=101s p = 09601)and a highly significant difference between techniques forHard with a 129 increase in speed with HS (avgTimes S=114s HS=101s p = 00071) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(69 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (as much as129 faster) In contrast the two techniques seem toperform very similarly under the Easy condition (seeFigure 3) One explanation is that in the Hard conditionusers are deprived of visual cues during the correctivephase of their movement because of the ldquofat fingerrdquoproblem Multimodal feedback likely alleviates this byproviding non-visual guidance Under the Easy conditionthe target may have been large enough for users to rely onvisual feedback only making haptic feedback superfluous

Surprisingly we were not able to find a significant effect ofDifficulty overall A tentative explanation can be foundin Figure 3 while S seems to be affected by difficulty HSexhibits a stable performance across difficulty levels Thissuggests that with haptic feedback all targets are equallyeasy Although this seems to contradict Fittsrsquo Law recallthis law is about aimed movements with visual feedbackThe haptic channel may not be as sensitive to target sizepossibly due to the fact that it is a sensory modalitydirectly connected with kinesthetic and motor functions

To summarize our study shows that adding tactilefeedback in the form of simulated detents facilitates theprecise manipulation of sliders Precise control of sliders ischallenging on touch devices partly due to the ldquofat fingerrdquoproblem We show that with haptic guidance it becomespractically as easy as coarse control Overall based on ourresults we recommend the use of sliders with hapticdetents on touch devices when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 3 Time by Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 22ndash36 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique but no significant effect ofDifficulty and no TechniquetimesDifficulty interaction(see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 127336 00044Difficulty 111 27084 01281TechniquetimesDifficulty 111 40402 00696

Our Anova analysis therefore confirms that technique HSyields significantly shorter completion times thantechnique S overall ie all task difficulties confoundedThe average Time is 109s for S and 104s for HS (seeFigure 3) This difference corresponds to a 48 increasein speed for technique HS compared to technique S

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(48 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Surprisingly we found no significant effect of Difficulty

overall so our hypothesis (H2) is not confirmed Thiscould be explained by the fact that differences in targetdifficulty were not large enough to significantly affectperformance We could have used different target sizesbut the limited input resolution of the device prevented usfrom using much smaller targets Conversely a very largetarget would occupy most of the slider range which doesnot capture realistic slider tasks Overall it seems that forsliders target size is not a crucial factor

To summarize our study provides strong evidence for thebenefits of tactile feedback when operating slidersAlthough moderate the effect of technique was found tobe highly significant Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Thisallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficulty

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (3 female) familiar with touch devicesaged 20ndash37 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of both Technique and Difficulty but nosignificant TechniquetimesDifficulty interaction effect (seeTable 1)

Table 1 Anova table

Source df F SigTechnique 111 51139 00450Difficulty 111 62892 00291TechniquetimesDifficulty 111 13669 02671

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 116s for S vs 110s forHS a 55 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 125s for Hard vs 101s for Easy corresponding to a 238 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(55 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a significant difference betweenthe two levels of difficulty all techniques confounded withEasy being as much as 238 faster than Hard Therefore our hypothesis (H2) is also supported Wederived our difficulty levels based on extensive pilotstudies so as not to favor any technique Our resultsvalidate our experimental design and confirm that targetsize is an adequate metric for task difficulty HS appearsto perform comparably well under two widely differenttask difficulties suggesting that its advantages may wellgeneralize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Difficulty

Av

erag

e T

ime

(s)

095

100

105

110

115

120

125

130

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash32 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a highly significant effect ofDifficulty with also a highly significantTechniquetimesDifficulty interaction effect (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 32748 00977Difficulty 111 142324 00031TechniquetimesDifficulty 111 149541 00026

Our analysis confirms the effect of difficulty (avg TimesEasy=098s Hard=125s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=096s HS=100s p = 01416) anda highly significant difference between techniques forHard with a 92 increase in speed with HS (avgTimes S=130s HS=119s p = 00069) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 92 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

105

110

115

120

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 21ndash50 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 72144 00212Difficulty 111 41479 00665TechniquetimesDifficulty 111 55941 00375

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 112s for S vs 106s forHS a 57 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=105s HS=103s p = 04065)and a highly significant difference between techniques forHard with a 82 increase in speed with HS (avgTimes S=119s HS=110s p = 00060) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(57 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (82 faster)In contrast the improvement was lower (19) under theEasy condition (also see Figure 3) One explanation isthat in the Hard condition the ldquofat fingerrdquo probleminterferes with the corrective phase of usersrsquo movementMultimodal feedback likely alleviates this by providingnon-visual guidance Under the Easy condition the targetwas larger and the fat finger issue not as pronouncedmaking haptic feedback still useful but less critical

Surprisingly we were not able to find a significant effectof Difficulty overall despite the trends visible inFigure 3 This could be explained by the fact thatdifferences in the target difficulty were not large enough tosignificantly affect performance In our pilot studies weconsidered tasks involving much smaller or much largertargets but dismissed them as unrealistic So it seemsthat overall target size is not a crucial factor for sliders

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Operatingsliders is hard on touch devices in general but even moreso when fine control is needed due to the ldquofat fingerrdquoproblem We show that haptic guidance greatly facilitatesthis task Overall based on our results we recommendthe use of sliders with haptic detents on touch devicesespecially when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

1000

1025

1050

1075

1100

1125

1150

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash39 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 60536 00317Difficulty 111 10392 03299TechniquetimesDifficulty 111 94480 00106

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 108s for S vs 101s forHS a 69 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=101s HS=101s p = 09601)and a highly significant difference between techniques forHard with a 129 increase in speed with HS (avgTimes S=114s HS=101s p = 00071) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(69 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (as much as129 faster) In contrast the two techniques seem toperform very similarly under the Easy condition (seeFigure 3) One explanation is that in the Hard conditionusers are deprived of visual cues during the correctivephase of their movement because of the ldquofat fingerrdquoproblem Multimodal feedback likely alleviates this byproviding non-visual guidance Under the Easy conditionthe target may have been large enough for users to rely onvisual feedback only making haptic feedback superfluous

Surprisingly we were not able to find a significant effect ofDifficulty overall A tentative explanation can be foundin Figure 3 while S seems to be affected by difficulty HSexhibits a stable performance across difficulty levels Thissuggests that with haptic feedback all targets are equallyeasy Although this seems to contradict Fittsrsquo Law recallthis law is about aimed movements with visual feedbackThe haptic channel may not be as sensitive to target sizepossibly due to the fact that it is a sensory modalitydirectly connected with kinesthetic and motor functions

To summarize our study shows that adding tactilefeedback in the form of simulated detents facilitates theprecise manipulation of sliders Precise control of sliders ischallenging on touch devices partly due to the ldquofat fingerrdquoproblem We show that with haptic guidance it becomespractically as easy as coarse control Overall based on ourresults we recommend the use of sliders with hapticdetents on touch devices when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficulty

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (3 female) familiar with touch devicesaged 20ndash37 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of both Technique and Difficulty but nosignificant TechniquetimesDifficulty interaction effect (seeTable 1)

Table 1 Anova table

Source df F SigTechnique 111 51139 00450Difficulty 111 62892 00291TechniquetimesDifficulty 111 13669 02671

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 116s for S vs 110s forHS a 55 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 125s for Hard vs 101s for Easy corresponding to a 238 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(55 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a significant difference betweenthe two levels of difficulty all techniques confounded withEasy being as much as 238 faster than Hard Therefore our hypothesis (H2) is also supported Wederived our difficulty levels based on extensive pilotstudies so as not to favor any technique Our resultsvalidate our experimental design and confirm that targetsize is an adequate metric for task difficulty HS appearsto perform comparably well under two widely differenttask difficulties suggesting that its advantages may wellgeneralize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Difficulty

Av

erag

e T

ime

(s)

095

100

105

110

115

120

125

130

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash32 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a highly significant effect ofDifficulty with also a highly significantTechniquetimesDifficulty interaction effect (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 32748 00977Difficulty 111 142324 00031TechniquetimesDifficulty 111 149541 00026

Our analysis confirms the effect of difficulty (avg TimesEasy=098s Hard=125s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=096s HS=100s p = 01416) anda highly significant difference between techniques forHard with a 92 increase in speed with HS (avgTimes S=130s HS=119s p = 00069) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 92 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

105

110

115

120

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 21ndash50 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 72144 00212Difficulty 111 41479 00665TechniquetimesDifficulty 111 55941 00375

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 112s for S vs 106s forHS a 57 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=105s HS=103s p = 04065)and a highly significant difference between techniques forHard with a 82 increase in speed with HS (avgTimes S=119s HS=110s p = 00060) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(57 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (82 faster)In contrast the improvement was lower (19) under theEasy condition (also see Figure 3) One explanation isthat in the Hard condition the ldquofat fingerrdquo probleminterferes with the corrective phase of usersrsquo movementMultimodal feedback likely alleviates this by providingnon-visual guidance Under the Easy condition the targetwas larger and the fat finger issue not as pronouncedmaking haptic feedback still useful but less critical

Surprisingly we were not able to find a significant effectof Difficulty overall despite the trends visible inFigure 3 This could be explained by the fact thatdifferences in the target difficulty were not large enough tosignificantly affect performance In our pilot studies weconsidered tasks involving much smaller or much largertargets but dismissed them as unrealistic So it seemsthat overall target size is not a crucial factor for sliders

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Operatingsliders is hard on touch devices in general but even moreso when fine control is needed due to the ldquofat fingerrdquoproblem We show that haptic guidance greatly facilitatesthis task Overall based on our results we recommendthe use of sliders with haptic detents on touch devicesespecially when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

1000

1025

1050

1075

1100

1125

1150

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash39 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 60536 00317Difficulty 111 10392 03299TechniquetimesDifficulty 111 94480 00106

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 108s for S vs 101s forHS a 69 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=101s HS=101s p = 09601)and a highly significant difference between techniques forHard with a 129 increase in speed with HS (avgTimes S=114s HS=101s p = 00071) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(69 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (as much as129 faster) In contrast the two techniques seem toperform very similarly under the Easy condition (seeFigure 3) One explanation is that in the Hard conditionusers are deprived of visual cues during the correctivephase of their movement because of the ldquofat fingerrdquoproblem Multimodal feedback likely alleviates this byproviding non-visual guidance Under the Easy conditionthe target may have been large enough for users to rely onvisual feedback only making haptic feedback superfluous

Surprisingly we were not able to find a significant effect ofDifficulty overall A tentative explanation can be foundin Figure 3 while S seems to be affected by difficulty HSexhibits a stable performance across difficulty levels Thissuggests that with haptic feedback all targets are equallyeasy Although this seems to contradict Fittsrsquo Law recallthis law is about aimed movements with visual feedbackThe haptic channel may not be as sensitive to target sizepossibly due to the fact that it is a sensory modalitydirectly connected with kinesthetic and motor functions

To summarize our study shows that adding tactilefeedback in the form of simulated detents facilitates theprecise manipulation of sliders Precise control of sliders ischallenging on touch devices partly due to the ldquofat fingerrdquoproblem We show that with haptic guidance it becomespractically as easy as coarse control Overall based on ourresults we recommend the use of sliders with hapticdetents on touch devices when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Difficulty

Av

erag

e T

ime

(s)

095

100

105

110

115

120

125

130

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash32 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a highly significant effect ofDifficulty with also a highly significantTechniquetimesDifficulty interaction effect (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 32748 00977Difficulty 111 142324 00031TechniquetimesDifficulty 111 149541 00026

Our analysis confirms the effect of difficulty (avg TimesEasy=098s Hard=125s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=096s HS=100s p = 01416) anda highly significant difference between techniques forHard with a 92 increase in speed with HS (avgTimes S=130s HS=119s p = 00069) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 92 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

105

110

115

120

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 21ndash50 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 72144 00212Difficulty 111 41479 00665TechniquetimesDifficulty 111 55941 00375

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 112s for S vs 106s forHS a 57 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=105s HS=103s p = 04065)and a highly significant difference between techniques forHard with a 82 increase in speed with HS (avgTimes S=119s HS=110s p = 00060) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(57 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (82 faster)In contrast the improvement was lower (19) under theEasy condition (also see Figure 3) One explanation isthat in the Hard condition the ldquofat fingerrdquo probleminterferes with the corrective phase of usersrsquo movementMultimodal feedback likely alleviates this by providingnon-visual guidance Under the Easy condition the targetwas larger and the fat finger issue not as pronouncedmaking haptic feedback still useful but less critical

Surprisingly we were not able to find a significant effectof Difficulty overall despite the trends visible inFigure 3 This could be explained by the fact thatdifferences in the target difficulty were not large enough tosignificantly affect performance In our pilot studies weconsidered tasks involving much smaller or much largertargets but dismissed them as unrealistic So it seemsthat overall target size is not a crucial factor for sliders

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Operatingsliders is hard on touch devices in general but even moreso when fine control is needed due to the ldquofat fingerrdquoproblem We show that haptic guidance greatly facilitatesthis task Overall based on our results we recommendthe use of sliders with haptic detents on touch devicesespecially when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

1000

1025

1050

1075

1100

1125

1150

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash39 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 60536 00317Difficulty 111 10392 03299TechniquetimesDifficulty 111 94480 00106

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 108s for S vs 101s forHS a 69 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=101s HS=101s p = 09601)and a highly significant difference between techniques forHard with a 129 increase in speed with HS (avgTimes S=114s HS=101s p = 00071) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(69 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (as much as129 faster) In contrast the two techniques seem toperform very similarly under the Easy condition (seeFigure 3) One explanation is that in the Hard conditionusers are deprived of visual cues during the correctivephase of their movement because of the ldquofat fingerrdquoproblem Multimodal feedback likely alleviates this byproviding non-visual guidance Under the Easy conditionthe target may have been large enough for users to rely onvisual feedback only making haptic feedback superfluous

Surprisingly we were not able to find a significant effect ofDifficulty overall A tentative explanation can be foundin Figure 3 while S seems to be affected by difficulty HSexhibits a stable performance across difficulty levels Thissuggests that with haptic feedback all targets are equallyeasy Although this seems to contradict Fittsrsquo Law recallthis law is about aimed movements with visual feedbackThe haptic channel may not be as sensitive to target sizepossibly due to the fact that it is a sensory modalitydirectly connected with kinesthetic and motor functions

To summarize our study shows that adding tactilefeedback in the form of simulated detents facilitates theprecise manipulation of sliders Precise control of sliders ischallenging on touch devices partly due to the ldquofat fingerrdquoproblem We show that with haptic guidance it becomespractically as easy as coarse control Overall based on ourresults we recommend the use of sliders with hapticdetents on touch devices when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

105

110

115

120

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 21ndash50 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 72144 00212Difficulty 111 41479 00665TechniquetimesDifficulty 111 55941 00375

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 112s for S vs 106s forHS a 57 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=105s HS=103s p = 04065)and a highly significant difference between techniques forHard with a 82 increase in speed with HS (avgTimes S=119s HS=110s p = 00060) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(57 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (82 faster)In contrast the improvement was lower (19) under theEasy condition (also see Figure 3) One explanation isthat in the Hard condition the ldquofat fingerrdquo probleminterferes with the corrective phase of usersrsquo movementMultimodal feedback likely alleviates this by providingnon-visual guidance Under the Easy condition the targetwas larger and the fat finger issue not as pronouncedmaking haptic feedback still useful but less critical

Surprisingly we were not able to find a significant effectof Difficulty overall despite the trends visible inFigure 3 This could be explained by the fact thatdifferences in the target difficulty were not large enough tosignificantly affect performance In our pilot studies weconsidered tasks involving much smaller or much largertargets but dismissed them as unrealistic So it seemsthat overall target size is not a crucial factor for sliders

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat Operatingsliders is hard on touch devices in general but even moreso when fine control is needed due to the ldquofat fingerrdquoproblem We show that haptic guidance greatly facilitatesthis task Overall based on our results we recommendthe use of sliders with haptic detents on touch devicesespecially when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

1000

1025

1050

1075

1100

1125

1150

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash39 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 60536 00317Difficulty 111 10392 03299TechniquetimesDifficulty 111 94480 00106

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 108s for S vs 101s forHS a 69 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=101s HS=101s p = 09601)and a highly significant difference between techniques forHard with a 129 increase in speed with HS (avgTimes S=114s HS=101s p = 00071) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(69 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (as much as129 faster) In contrast the two techniques seem toperform very similarly under the Easy condition (seeFigure 3) One explanation is that in the Hard conditionusers are deprived of visual cues during the correctivephase of their movement because of the ldquofat fingerrdquoproblem Multimodal feedback likely alleviates this byproviding non-visual guidance Under the Easy conditionthe target may have been large enough for users to rely onvisual feedback only making haptic feedback superfluous

Surprisingly we were not able to find a significant effect ofDifficulty overall A tentative explanation can be foundin Figure 3 while S seems to be affected by difficulty HSexhibits a stable performance across difficulty levels Thissuggests that with haptic feedback all targets are equallyeasy Although this seems to contradict Fittsrsquo Law recallthis law is about aimed movements with visual feedbackThe haptic channel may not be as sensitive to target sizepossibly due to the fact that it is a sensory modalitydirectly connected with kinesthetic and motor functions

To summarize our study shows that adding tactilefeedback in the form of simulated detents facilitates theprecise manipulation of sliders Precise control of sliders ischallenging on touch devices partly due to the ldquofat fingerrdquoproblem We show that with haptic guidance it becomespractically as easy as coarse control Overall based on ourresults we recommend the use of sliders with hapticdetents on touch devices when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

00

02

04

06

08

10

12

Slider Haptic Slider

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

1000

1025

1050

1075

1100

1125

1150

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (4 female) familiar with touch devicesaged 18ndash39 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a significanteffect of Technique and a significant interactionTechniquetimesDifficulty (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 60536 00317Difficulty 111 10392 03299TechniquetimesDifficulty 111 94480 00106

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 108s for S vs 101s forHS a 69 increase in speed (see Figure 2) Studentrsquost-tests reveal no significant difference between techniquesfor Easy (avg Times S=101s HS=101s p = 09601)and a highly significant difference between techniques forHard with a 129 increase in speed with HS (avgTimes S=114s HS=101s p = 00071) (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(69 faster) Our hypothesis (H1) is therefore confirmed

In addition we found a significant interaction betweentechnique and task difficulty with a higher performancegain brought by HS for the Hard condition (as much as129 faster) In contrast the two techniques seem toperform very similarly under the Easy condition (seeFigure 3) One explanation is that in the Hard conditionusers are deprived of visual cues during the correctivephase of their movement because of the ldquofat fingerrdquoproblem Multimodal feedback likely alleviates this byproviding non-visual guidance Under the Easy conditionthe target may have been large enough for users to rely onvisual feedback only making haptic feedback superfluous

Surprisingly we were not able to find a significant effect ofDifficulty overall A tentative explanation can be foundin Figure 3 while S seems to be affected by difficulty HSexhibits a stable performance across difficulty levels Thissuggests that with haptic feedback all targets are equallyeasy Although this seems to contradict Fittsrsquo Law recallthis law is about aimed movements with visual feedbackThe haptic channel may not be as sensitive to target sizepossibly due to the fact that it is a sensory modalitydirectly connected with kinesthetic and motor functions

To summarize our study shows that adding tactilefeedback in the form of simulated detents facilitates theprecise manipulation of sliders Precise control of sliders ischallenging on touch devices partly due to the ldquofat fingerrdquoproblem We show that with haptic guidance it becomespractically as easy as coarse control Overall based on ourresults we recommend the use of sliders with hapticdetents on touch devices when fine control is needed

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Slider Haptic Slider

Figure 2 Time by Technique

Av

erag

e T

ime

(s)

000

025

050

075

100

125

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (2 female) familiar with touch devicesaged 20ndash43 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals a highlysignificant effect of Technique and a very highlysignificant effect of Difficulty and noTechniquetimesDifficulty interaction (see Table 1)

Table 1 Anova table

Source df F SigTechnique 111 131323 00040Difficulty 111 219758 00007TechniquetimesDifficulty 111 39159 00734

Our analysis therefore confirms that HS is faster than Soverall with an average Time of 117s for S vs 110s forHS a 64 increase in speed (see Figure 2) Ouranalysis also confirms the effects of task difficulty with anaverage Time of 124s for Hard vs 103s for Easy corresponding to a 204 increase in speed (see Figure 3)

DiscussionOur user study shows that subjects completed the taskssignificantly faster in the presence of haptic feedback(64 faster) Our hypothesis (H1) is therefore confirmed

The superiority of haptic feedback seems to hold for alltarget difficulties as suggested by the lack of significantinteraction between Technique and Difficulty Eventhough large targets do not suffer from the ldquofat fingerrdquoproblem multimodal feedback still seems superior tovisual-only feedback This could be explained by the factthat the haptic channel is a sensory modality directlyconnected with kinesthetic and motor functions andtherefore capitalizes on our reflexive motor responses

Our analysis also shows a highly significant differencebetween the two levels of difficulty all techniquesconfounded with Easy being as much as 204 fasterthan Hard Therefore our hypothesis (H2) is alsosupported We derived our difficulty levels based onextensive pilot studies so as not to favor any techniqueOur results validate our experimental design and confirmthat target size is an adequate metric for task difficultyHS appears to perform comparably well under two widelydifferent task difficulties suggesting that its advantagesmay well generalize to other difficulty levels

To summarize our study confirms that adding tactilefeedback in the form of simulated detents facilitates theoperation of sliders Tactile guidance provides additionalproprioceptive cues when interacting with the glasssurface of the devicemdashotherwise uniformly flat This likelyallows users to maintain an accurate mental model of theslider thumbrsquos location speeding up the reaching ofspecific locations Overall based on our results werecommend the use of sliders with haptic detents on touchdevices both for fine and for coarse control

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Figure 2 A participantcompleting our study

Aver

age

Tim

e (s

)

000

025

050

075

100

125

150

Easy Hard

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (7 female) familiar with touch devicesaged 19ndash31 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S overall(H2) Easy tasks are faster than Hard tasks overall

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty Furthermore the Anova analysis did notreveal any significant TechniquetimesDifficulty interactioneffect (see Table 1 below)

Table 1 Anova table

Source df F SigTechnique 111 46215 00547Difficulty 111 48698 00495TechniquetimesDifficulty 111 18322 02030

Our analysis confirms the effects of task difficulty with anaverage Time of 129s for Hard vs 102s for Easy corresponding to a 265 increase in speed (see Figure 3)Thus our second hypothesis (H2) is confirmed

DiscussionOur initial hypothesis was that haptic feedback wouldfacilitate 1-D target acquisition tasks (H1) Our analysesfailed to support this hypothesis Yet our results suggestthat if haptic feedback may not help it does not harmeither Indeed HS was still on average 4 faster than S although this difference was not statistically significant

Participantsrsquo answers to our post-experimentquestionnaire suggest that haptic feedback may providequalitative benefits beyond pure task completion timesMany participants rated the technique high in hedonisticvalue (a median of 4 on a 5-point Likert scale) andfeedback on haptic detents was overall positive

The feedback collected during our study also helped usidentify directions for improvement for our currentprototype Some participants expressed discomfort whileinteracting with HS One mentioned ldquoa feeling similar asif the device was sending little electrical shocks to thefingerrdquo and thought the equipment was dysfunctionalWe believe this could easily be fixed by allowing users topersonalize the haptic signal One participant commentedthat haptic feedback ldquofeels weird [She] would ratherexpect [her] finger to smoothly glide on the glass surfacerdquoIndeed a flat screen provides conflicting affordances withhaptic feedback Visual techniques that emphasizephysicality (eg shadow or cushion effects to convey holesand bumps) could address this problem

In summary while our study did not reveal significantquantitative benefits of haptic detents over the traditionaltouch slider the qualitative feedback we received was verypositive and encouraging We were able to collectvaluable insights that shed light on the limitations ofcurrent haptic interfaces We hope that our results willinform and inspire further development in the area

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

User Study

Easy Hard

haptic signal

Figure 1 Example of Easy (left)and Hard (right) targets on thetouch slider In the hapticcondition (HS) 20 evenly-spaceddetents were simulated withvibrotactile feedback

Aver

age

Tim

e (s

)

000

025

050

075

100

125

Easy Hard

Figure 2 Time by Technique

Aver

age

Tim

e (s

)

100

105

110

115

120

125

Easy Hard

SliderHaptic Slider

Figure 3 Time by Difficultyand Technique

Our study examines the benefits of adding haptic detentsto touch sliders We used 1-D target acquisition tasksinvolving both easy and hard targets (see Figure 1)

A repeated measure full-factorial within-subject designwas used The factors were Technique = S=sliderHS=haptic slider and Difficulty = Easy HardTwelve volunteers (5 female) familiar with touch devicesaged 19ndash35 participated in the study We collected atotal of 12 Participant times 2 Technique times 2 Difficulty

times 128 repetitions = 6144 trials with completion Time

Hypotheses(H1) Technique HS is faster than technique S (H2) Easy tasks are faster than Hard

ResultsAn Anova on Time with the model TechniquetimesDifficultytimesRnd(Participant) reveals no significanteffect of Technique but a significant effect ofDifficulty with also a very highly significantTechniquetimesDifficulty interaction effect (see Table1)

Table 1 Anova table

Source df F SigTechnique 111 21350 01719Difficulty 111 51621 00442TechniquetimesDifficulty 111 226791 00006

Our analysis confirms the effect of difficulty (avg TimesEasy=102s Hard=119s see Figure 2) Studentrsquos t-testsreveal no significant difference between techniques forEasy (avg Times S=101s HS=104s p = 02757) anda very highly significant difference between techniques forHard with a 88 increase in speed with HS (avgTimes S=124s HS=114s p = 00061) (see Figure 3)

DiscussionWhile we did not observe a significant main effect ofTechnique an analysis of simple effects reveals that HSsignificantly outperformed S in the Hard condition withas much as 88 in speed improvement Therefore ourhypothesis (H1) is only partially confirmed

Although we did not find a significant difference betweentechniques in the Easy condition Figure 3 exhibits anintriguing trend raising the possibility of HS being worsethan S under the Easy condition This seems to beconfirmed by the very strong interaction observed betweenTechnique and Difficulty A possible explanation couldbe that the regular bursts generated by the haptic detentsis distracting to some users which in turn slightly impairstheir performance Indeed some participants expresseddiscomfort while interacting with HS

In the Hard condition however the situation is verydifferent due to the ldquofat fingerrdquo problem users are likelydeprived of visual cues during the corrective phase of theirmovement In this case multimodal feedback likelyalleviates this issue by providing non-visual guidance Inother terms when the target is small the benefits broughtby haptic feedback largely outweigh discomfort issuesallowing users to acquire these targets much more easily

To summarize our study shows that adding tactilefeedback in the form of simulated detents can be aneffective solution to the ldquofat fingerrdquo problem whenmanipulating sliders on touch devices However hapticfeedback can also be distracting and in some cases impairperformance when the task is easy (large 1-D targets)Overall based on our results we recommend the usehaptic detents on touch sliders for tasks that require finecontrol but not for tasks where coarse control is sufficient

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References

Methods and DiscussionSetting up a multiverse experiment is impractical todaydue to the current difficulty of communicating acrossuniverses [1] We therefore simulated the data that couldhave been produced by such an experiment We assumed8 universes sharing identical characteristics in terms of thepopulation of interest the true effects the investigatingresearchers the experimental protocol and the dataanalysis methods Only population sampling was assumedto be subject to random variations ie the 12 subjectswho signed up for the study differed across universes

A mean Time measure was generated for all 48combinations of (subject Technique and Difficulty) asfollows Time(iHS Easy) = exi Time(iS Easy) = exi+xprime

i Time(iHS Hard) = exi+zi Time(iS Hard) = exi+yi+zi with XX prime sim N (0 01)Y sim N (008 01) Z sim N (01 02) N (micro σ2) denotes anormal distribution and xi refers to the realization of therandom variable X for the subject i This method yieldslognormal time distributions and correlated measureswithin subjects Values of micro and σ2 have been chosen toyield statistical powers of 04 to 07 (see Figure 4) Thetwo techniques have identical means for Easy

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

1 01 001 0001 00001

Main effect

of TECHNIQUE

Main effect

of DIFFICULTY

Interaction

Effect of

TECHNIQUE

for Hard

Effect of

TECHNIQUE

for Easy

53

58

43

71

05

47

42

57

29

995

p =

p =

p =

p =

p =

ns

ns

ns

ns

ns

Figure 4 Probabilitydistributions for p valuesestimated using Monte Carlomethods Red indicatesnon-significant green indicatessignificant Except for the nulleffect (bottom) about any p

value can be obtained

This exercise is meant to illustrate the extent to whichexperiment analyses and conclusions are determined bychance Our analysis methods are typical of HCI withstatistical powers typical of psychology [3] and HCI [4]

Researchers know about sampling error but are overlyobsessed with Type I errors (which did not occur in any ofour 8 universes) Our analyses highlight a more generaland widespread pitfall the overreliance on p values If pis small means are reported and discussed as if they wereexact A large p value (ie larger than the standard butnonetheless arbitrary cutoff of 005) is often taken as a

sign that there is no effect But p values simply cannot betrusted (see Figure 4 and [2] for a demo) Althoughtraditional statistical practices have started to bequestioned in CHI [4] this issue has been disregarded Werefer the reader to [3] for a more extensive discussion andan alternative relying on estimation rather than p valueswhen analyzing and interpreting experimental results

Note that our simulated multiverse experiment isequivalent to simulating multiple replications of anexperiment in a single universe [3] There are indeed anumber of analogies like the multiverse theory theprinciple of scientific replication has theoretical supportbut has been hardly observed in practice In the contextof HCI we thought that a multiverse scenario would beslightly more believable [5] It also captures the idea thatwhile many outcomes are possible for an experiment wetypically only have access to one of them Hopefully wewill always keep the multiverse in mind

References[1] Carr B Ed Universe or multiverse Cambridge

University Press 2007[2] Cumming G Dance of the p values (video)

tinyurlcomdanceptrial2 2009[3] Cumming G The new statistics why and how

Psychological science 25 1 (2014) 7ndash29[4] Kaptein M and Robertson J Rethinking statistical

analysis methods for CHI In Proc CHIrsquo 12 ACM(2012) 1105ndash1114

[5] Wilson M L Mackay W Chi E Bernstein MRussell D and Thimbleby H RepliCHI - CHI shouldbe replicating and validating results more discuss InCHI Extended abstracts ACM (2011) 463ndash466

  • Introduction
  • User Study
    • Hypotheses
    • Results
    • Discussion
      • User Study
        • Hypotheses
        • Results
        • Discussion
          • User Study
            • Hypotheses
            • Results
            • Discussion
              • User Study
                • Hypotheses
                • Results
                • Discussion
                  • User Study
                    • Hypotheses
                    • Results
                    • Discussion
                      • User Study
                        • Hypotheses
                        • Results
                        • Discussion
                          • User Study
                            • Hypotheses
                            • Results
                            • Discussion
                              • User Study
                                • Hypotheses
                                • Results
                                • Discussion
                                  • Methods and Discussion
                                  • References