STIC - Sensory and Tactile Improved CaneJulia Hertel, Armin Schaare, Philipp Feuerbach, Oscar Ariza, Frank Steinicke

Universität HamburgGermany

3hertel,3schaare,3feuerba,ariza,steinicke@informatik.uni-hamburg.de

ABSTRACTNavigation and obstacle avoidance, especially regarding head-level objects, are of major concern to visually impaired peo-ple. This paper describes the advances on the design of anaugmented white cane, STIC (Sensory and Tactile ImprovedCane), making use of an ultrasonic distance sensor for head-level and a laser-based distance sensor for ground-level ob-stacle detection. The distance information is encoded viavibrotactile actuators. Additionally, a novel approach to re-tain obstacle information with the help of haptic Afterimagesis presented. Finally, the discriminability of distances basedon the vibrotactile feedback was measured via a psychome-tric study. Objects shiftings by more than 13.58 cm yield a90% detection accuracy.

KEYWORDSvisually impaired, obstacle detection, portable device, whitecane, smart cane, psychometric study, tactile feedback

1 INTRODUCTIONDetermining the position of obstacles is a challenging task inthe daily life of the visually impaired. According toManduchiand Kurniawan, "Head-level and fall accidents represent anon-negligible risk associated with walking without sight."[9]. They report that head-level accidents leading to medicalconsequences are often compensated by a lower walkingspeed. A white cane is the de-facto tool for visually impairedpersons (VIPs) to navigate and avoid crashes [9]. Neverthe-less, it lacks the ability to detect objects above floor level.Therefore so-called smart canes make use of ultrasonic orinfrared sensors which obtain distance and position of ob-stacles [8]. Auditory or vibrotactile feedback patterns areneeded to encode and deliver relevant information to the userin a comprehensible way. This work introduces the Sensoryand Tactile Improved Cane (STIC), another smart cane de-sign and a novel approach to maintain obstacle informationin an actuator array via haptic Afterimages. A psychome-tric study is conducted evaluating the discernability of anobject’s longitudinal motion relative to the participant basedon the vibrotactile feedback.

2 RELATEDWORKResults from recent studies suggest that VIPs experienceimproved perception throughout other senses such as hear-ing and the sense of touch [4, 6]. White canes make useof the haptic sense but typically lack in detecting objectsabove floor-level. Therefore attempts have been made toaugment white canes with additional sensors for detect-ing obstacles e.g. hanging signs, low door frames, etc. (seeUltraCane[2], WeWALK[1], GuideCane [12]. Wang et al. im-plemented HALO, an attachable device for white canes con-sisting of an ultrasonic sensor and a vibrating motor. Itresponds to low-hanging obstacles by providing vibrotac-tile feedback [15]. Additionally, navigating along predefinedpaths has been realized by tactile and auditive feedback via asmart cane [10]. The NavBelt, an obstacle-avoidance system,consisting of 8 ultrasonic sensors to scan the environment,provides binaural audio feedback. Its downside is the aggra-vation of putting it on [11].

3 DEVELOPING THE STICAs discussed in RelatedWork, vibrotactile, and auditory feed-back stand out to be the most commonly used for smartcanes. A study, comparing both guidance methods whilesubjecting participants to street noise, indicates a similarperformance of either method [7]. However, devices that useregular headphones are inadvisable since in a real-world situ-ation, hearing is important for orientation and traffic dangeridentification. A possible solution could be Bone-ConductionHeadphones which do not cover the outer ear, instead cre-ating sound by vibration through the skull [14]. Still, thequestion remains whether multiple audio cues could causeauditory overload. Also, due to working memory restrictions,vibrotactile, spatial feedback is expected to interfere less withenvironmentally based auditory stimuli [3].Kim and Cho defined principles regarding vibrotactile

feedback and handle design. Problematic were weight, mis-match of sensor and stick orientation, feedback insensitivity,habituation, and price [8]. Through a survey, Dakopoulos etal. point out four major characteristics for electronic travelaids: free-hands interaction, no ear coverage, wearable tech-nology, no hard learning curve [5].

MuC ’19, September 8–11, 2019, Hamburg, Germany Hertel et al.

STIC LayoutThe current STIC prototype (Figure 1) mimics a typical whitecane and consists of a wooden stick with a plastic ball at-tached to the lower end and a 3D printed plastic ergonomichandle attached to the upper end.Furthermore, two sensors- an infrared sensor and an ultrasonic sensor - are part ofthe cane. The infrared sensor aims to detect obstacles atground-level and is placed near the stick’s bottom end andaligned horizontally. The ultrasonic sensor is located at theheight of the user’s knee and points orthogonally from thestick in a diagonally upwards direction, detecting objectsplaced in front of the user’s head. The STIC’s electronics arewrapped in a case, placed near the handle on its lower sideto minimize the force needed to balance and sweep the cane.

Figure 1: The design of the STIC. The image shows the place-ment of the sensors, the Arduino board, and the handle.

All in all, the STIC is 151cm long and weighs 477 grams.The wooden sticks diameter is 1.5cm and the ballpoint diam-eter is 5cm. The container for the Arduino board is formedin length 17cm, width 8.5cm and height 3cm.

Handle LayoutThe two-point discriminability on the palm amounts to about(∼9mm) to (∼10mm)with similar numbers having been foundfor point localization distances. Also, pressure sensitivityevoked by vibrotactile stimuli is reported to be at a max-imum between 220Hz and 300Hz [16]. The frequency forvibrotactile stimuli was set to 240Hz and the handle layoutwas designed to leave at minimum 11mm space between thefive used piezo actuators.

Feedback FunctionBased on the sensor capabilities, we defined the detectionrange to be 20cm to 300cm. A function maps the sensor’smeasured distance to a frequency of the actuator activations.

f (x) = amax ∗

(amin

amax

) x−dmindmax −dmin

(1)

f (x1)

f (x1 − a)=

f (x2)

f (x2 − a)(2)

Figure 2: Placement of the actuators on the handle front(left), back side (center) and their positions on the hand(right) depicting their positions on the handle.

This exponential decay function maps the minimum dis-tance to the maximum activation frequency and vice versa(Equation 1), for the maximum and minimum activation fre-quency amax , amin as well as the maximum and minimumdistances dmax , dmin . Based on tests with the STIC and soft-ware limitations, the activation frequencies minimum andmaximum were set to 1Hz and 7Hz. The previous functionwas chosen because it satisfies the “same offset — same ratio”property (Equation 2), which is essential since the perceiveddifferences of activation frequencies correspond exponen-tially with the measured distances.

Afterimage TechnologyThis section aims to introduce a novel technology which isreferred to as Afterimage Technology. Typical smart canes[1, 2, 12] are excellent approaches for obstacle detection, butthey are limited to obstacle information directly in front of acane user. When the cane is turned away, the information islost. Afterimage Technology adds some obstacle memory tothe feedback encoded by three additional actuators placedon the handle’s bottom side (see Figure 2: west, north andeast), which form a 1-D haptic display.On detection, an obstacle’s distance is recorded and remainsin memory for a specified lifetime. Since the goal of the caneis to detect knee-level up to head-level obstacles, the datafrom the ultrasonic sensor is used for the Afterimages. If theultrasonic sensor detects an object and the cane is rotatedaround the user’s z-axis, the Afterimage wanders preciselyin the opposite direction given the same angle. Hence theactuator that now points at the obstacle, is activated the most.The information about the cane’s orientation is accessed viaan IMU placed on the Arduino board and is updated continu-ously. For determining the strength at which an Afterimageactuator shall vibrate, the amplitude is manipulated accord-ing to the match between actuator placement and amountof shifting, as well as the time since the Afterimage wasrecorded. The activation frequency stays the same for all ac-tuators for one Afterimage over its lifetime and is determined

STIC - Sensory and Tactile Improved Cane MuC ’19, September 8–11, 2019, Hamburg, Germany

by the recorded distance of the detected object as describedin the Feedback Function section. To determine the matchbetween the angle in which an Afterimage was recordedand the angle a given actuator is assigned to, a Gauss-likefunction is used given by the following equation:

AT ,σ ,ϕ (x) = T ∗ exp

(−0.5 ∗

(1

σ ∗ (ϕ − x)

)2), (3)

where A ∈ [0, 1] is the feedback amplitude, T = 1 −Tl ived

Tl i f et imeis the lifetime dependent falloff factor, σ is the

spread of the Gauss-like curve (in this example σ = 90Nact

with Nact the number of actuators used) and ϕ the offsetrelative to the canes global rotation. Figure 3(left) showsthe Gauss-like function directly after the initialization of anAfterimage i.e., an object detected by the ultrasonic sensor.The following rotation around 45 to the right results in thecorresponding Gauss-like function shifting along its x-axisby −45 and diminishing by the elapsed time since initializa-tion, as seen in Figure 3(right). In this case, the west actuator(allocated to −60) will provide feedback by about 40% of themaximum amplitude and the north actuator by about 10%.When the Afterimages lifetime is over the device will beready to receive the distance of the next detected obstaclewhich generates a new Afterimage.

4 PSYCHOMETRIC STUDYA study was conducted to calculate the psychometric func-tion, which depends on the vibrotactile feedback. Feedbackwas given by one single vibrating actuator which was relatedto the detection of head-level obstacles. The psychometricfunction determines the minimum change in stimulus inten-sity that is perceptible by a person. Regarding a smart cane,the function shows the perceptible distance change in thedepth relative to the user that can be detected with a certainprobability. The goal was to calculate the minimum distanceshift that is necessary to be at least 90% sure whether theobject moves towards or away from the sensor.

Equipment and SoftwareThe embedded code is executed on an Arduino-compatiblemicrocontroller1 based on an ARM Cortex M0 processor,clocked at 48 MHz at 3.3V, and featuring Bluetooth Low En-ergy. Two sensors support distance estimation. First, a high-performance optical distance measurement sensor2, based onan edge-emitting, 905nm (1.3 watts), single-stripe laser trans-mitter with a 0-40m range and an accuracy of +/- 2.5cm. Sec-ond, an ultrasonic distance-sensor3 with a calibrated beam1https://www.adafruit.com/product/29952https://www.sparkfun.com/products/140323https://www.maxbotix.com/Ultrasonic_Sensors/MB1010.htm

Figure 3: Feedback amplitude function based on relative ro-tation of an Afterimage. Initial state (left) and the functionafter rotating 45 degrees to the right (right). Depicted are thethree actuators and their amplitude value.

pattern with a resolution of 1 inch, a 20Hz reading rate, and arange starting from 6 inches to 254 inches. In order to providevibrotactile feedback, the unit contains five haptic drivers4controlling piezo discs with a 9mm diameter. The piezo hap-tic driver integrates a 105-V boost switch, an amplifier, andsupports overdriving and active-breaking. The tactile signalscan be parameterized by frequency and amplitude simulta-neously, as well as envelopes and the beginning/end of thesignal.

For the current prototype, the Unity3D game engine run-ning on an external PC is used to compute the feedbackusing the feedback function described above. The sensorsare connected to the microcontroller, which reads their dataand calculates distances. These distances are transmitted toUnity3D via Bluetooth. After computing the tactile feedbackbased on the measured distances, signal are sent back to themicrocontroller, which then controls the piezo actuators.

DesignFor measuring the discriminability of the feedback function,an experimental room was set up, in which the total distance

4http://www.ti.com/product/DRV2667

MuC ’19, September 8–11, 2019, Hamburg, Germany Hertel et al.

that is covered by the distance sensor was marked on thefloor. Four reference points from near to far (55, 125, 195and 265 centimeters) were determined to cover the wholespectrum of possible object distances. A box was placed onthe respective reference point and shifted by 5, 10, 15, 20, 25or 30 centimeters closer to the participant (negative shift) orfurther afar (positive shift). After each shift, the participanthad to decide whether the box was shifted further away orcloser in depth. The participant was blindfolded and receivedwhite noise over noise-canceling headphones. The only cuepresented was feedback given by one actuator on the handle,and every participant was instructed to place the thumb oftheir dominant hand on the actuator. Before shifting the boxanother time, it was placed back on the reference point to of-fer participants the opportunity to memorize the vibrationalpattern once again. Per reference point, each shift rangingfrom 5 until 30 centimeters was presented two times as wellas in both the negative and positive direction. The closestpossible obstacle distance was 25cm, and the farthest pos-sible was 295cm. The order was always the same, startingfrom the closest reference point up to the last reference point.After 24 decisions, the reference point was changed, and theparticipant could have a short break if preferred. A total of96 decisions have been recorded per participant.

Figure 4: Study Setup. Blindfolded participant receivingfeedback via actuators on the handle relative to object dis-tance. The handle was held, such that the thumb could beplaced on the top actuator to perceive the feedback.

ResultsEach participant experienced 48 positive and 48 negativeshiftings. Data from 6 (4 male, 2 female; age: 23-36, M =26.33, SD = 4.93) participants were collected for a total of 288negative as well as the same amount of positive shiftings. Apaired T-Test comparing the two shifting directions positive(M = 0.91,C SD = 0.08) and negative (M = 0.93, SD = 0.08)presented no significant difference, t(5) = -1.22, p > 0.05.

Figure 5: Error rate by reference point (left). Mean error rateand SD by obstacle difference to reference point(right).

Since there were no significant differences according tothe previous test, the Pearson correlation was computed re-garding the absolute value of the shift and the probability ofcorrect answers (M = 0.92, SD = 0.08). The result indicates astrong correlation (Pearson’s r(4)= 0.91, p < 0.05). In total, 96trials were completed by every participant (24 per referencepoint). The mean number of wrong answers was 7.33 (SD =2.92) across all participants. The best and worst performingparticipants had an error rate of about 4% and about 12%respectively. Figure 5 (left) shows the error rates across allreference points. There were 144 conducted trials for eachreference point. Mean probability of false decisions acrossall reference points is given with 7.64% meaning 11.00 (SD =3.74) wrong decisions. Figure 5 (right) shows the error rateacross all distance differences from the object to the pointof reference. Each distance of the object to a reference pointhad 48 corresponding trials. The mean of the number of er-rors across all distance differences was 3.67 (SD = 3.59). Anon-parametric Friedman test yielded no significant differ-ence regarding the different reference points concerning theerror rate, χ 2(3) = 5.95, p > 0.1. In order to determine the psy-chometric function, a model was used which predicted thevalues between the discrete shifting steps of 5cm. Also, a shiftlower than 5cm has not been assessed and the probability isexpected to reach 50% for an infinite small shift close to 0cm.The data has been extended with the respective values forthe 0cm shift. This procedure resulted in a probabilistic dis-tribution which allowed to compute the threshold at whichat least 90% of forced choices were expected to be correct.The threshold for negative shifts is given at 12.34cm, 95%CI [10.15cm, 15.88cm]. Whereas the threshold for positiveshifts is given at 14.56cm, 95% CI [12.22cm, 17.48cm]. Sincethere is no difference assumed between negative and positiveshifting, the psychometric function for absolute shifts wascalculated, with a threshold at 13.58cm, 95% CI [11.64cm,15.56cm]. Using the same model for absolute shifts, the 80%threshold for theminimumnecessary distance shift decreasesto 8.24cm, 95% CI [6.70cm, 10.09cm].

STIC - Sensory and Tactile Improved Cane MuC ’19, September 8–11, 2019, Hamburg, Germany

Negative Positive AbsoluteModel mean 6.40 7.60 5.94Model SD 7.06 8.27 9.08Threshold 12.34 14.56 13.58

Figure 6: Overview of the 90% thresholds for shiftingsregarding the probabilistic model. The mean depicts the75% threshold (top). Psychometric function for ABSOLUTEvalue of shiftings, with Threshold = 0.9 (bottom).

Taking into account the model that was calculated for theabsolute shifting, the psychometric function describes therelationship between the difference in obstacle distances andthe probability to decide correctly whether the obstacle wasmoved closer or further afar. Given a minimum distance shiftof 12.33cm, the participants could determine the direction ofshifts with a probability of 90% on a whole spectrum rangingfrom 20cm - 300cm. Most notable is the higher error rate forthe first reference point in comparison to the other ones asseen in Figure 5 and the counter-intuitive distribution of er-rors in the negative distance differences i.e., when the objectwas moved closer to the sensor. Although not significant,the hypothesis is formulated that one reason for this was thelimitations of the hardware and software implementation atthe time of the study. Empirical testing and feedback frommost participants suggested an issue with the feedback inwhich the activation frequency was more irregular when anobject is placed in the front, closer to the sensor. The effectwas only notable in the range of the first reference point,hence the higher corresponding error rate. Furthermore, theeffect also slightly biased negative distance differences to-wards a higher error rate, which partly explains the observeddistribution of error rates seen in Figure 5. Another possi-ble reason for the higher error rate while conducting trialsconcerning the first reference point is the order of referencepoints for each participant.

5 DISCUSSIONAccording to a pilot study, the use of stronger actuators isvital to avoid a masking of the tactile feedback by the motionof the ballpoint over rough ground. Another related topic isthe use of STIC in cold weather. If the user decides to weargloves, the sensitivity to the actuator activations diminishesseverely.So far, STIC uses a constant frequency for its vibrotactilefeedback; this bears the risk of an adaption effect [13]. In thiscase, the user’s subjective magnitude could decrease, whichcould lead to a weakened perception of possible dangers.However, such an adaption effect is avoidable by switchingbetween different frequencies.

Due to limitations of the layout formost sensitive locationsfor vibrotactile feedback on the human hand and the numberof actuators the prototype should use, it was not possible todesign a handle which accommodates to the usual way a VIPholds a white cane. However, STIC’s ergonomic designedhandle creates a haptic affordance and is comfortable to hold.Whether users of traditional white canes prefer conventionalhandles over STIC’s one, has not been determined yet.

6 CONCLUSION AND FUTUREWORKThis article presented the design and implementation of STIC- an enhanced white cane. Making use of an ultrasonic dis-tance sensor and tactile actuators, STIC aims to help visuallyimpaired people to avoid obstacles by encoding the distanceof detected objects as an activation frequency of the tactileactuators. In the course of a study, participants were asked todiscriminate whether a presented obstacle was moved closeror further apart. The results indicate a high rate of correct an-swers (90%). Additionally, Afterimages – a theoretical novelapproach to improve obstacle avoidance – was presented.Further research will include an obstacle course study totest the Afterimage Technology and obstacle avoidance ca-pability. Additional optimizations are ongoing, related tohandle ergonomics, weight reduction, quality improvementof the vibrotactile feedback as well as testing novel actuators(e.g., audio-based haptic actuators/reactors) and wearabletime-of-flight (ToF) distance sensors.

REFERENCES[1] 2018. WeWalk. http://wewalk.io/en. Accessed: 2018-05-04.[2] Invented 2004. Ultracane. https://www.ultracane.com. Accessed:

2018-05-04.[3] Alan Baddeley. 1992. Working memory. Science 255, 5044 (1992),

556–559.[4] Corinna Bauer, Lindsay Yazzolino, Gabriella Hirsch, Zaira Cattaneo,

Tomaso Vecchi, and Lotfi B.Merabet. 2015. Neural correlates associatedwith superior tactile symmetry perception in the early blind. Cortex63 (2015), 104 – 117. https://doi.org/10.1016/j.cortex.2014.08.003

[5] Dimitrios Dakopoulos and Nikolaos G Bourbakis. 2010. Wearableobstacle avoidance electronic travel aids for blind: a survey. IEEE

MuC ’19, September 8–11, 2019, Hamburg, Germany Hertel et al.

Transactions on Systems, Man, and Cybernetics, Part C (Applicationsand Reviews) 40, 1 (2010), 25–35.

[6] Daniel Goldreich and Ingrid M. Kanics. 2003. Tactile Acuity is En-hanced in Blindness. Journal of Neuroscience 23, 8 (2003), 3439–3445. arXiv:http://www.jneurosci.org/content/23/8/3439.full.pdf http://www.jneurosci.org/content/23/8/3439

[7] Olinkha Gustafson-Pearce, Eric Billett, and Franjo Cecelja. 2007. Com-parison between audio and tactile systems for delivering simple navi-gational information to visually impaired pedestrians. British Journalof Visual Impairment 25, 3 (2007), 255–265.

[8] S. Kim and K. Cho. 2013. Usability and Design Guidelines of SmartCanes for Users with Visual Impairments. International Journal ofDesig (2013), 1–7.

[9] Roberto Manduchi and Sri Kurniawan. 2011. Mobility-Related Ac-cidents Experienced by People with Visual Impairment. Insight:Research and Practice in Visual Impairment and Blindness 4 (2011).http://users.soe.ucsc.edu/~manduchi/papers/MobilityAccidents.pdf

[10] Yuriko Shiizu, Yoshiaki Hirahara, Kenji Yanashima, and KazushigeMagatani. 2007. The development of a white cane which navigates thevisually impaired. In Engineering in Medicine and Biology Society, 2007.EMBS 2007. 29th Annual International Conference of the IEEE. IEEE,5005–5008.

[11] Yoram Koren Shraga Shoval, Johann Borenstein. 1998. The Navbelt -A Computerized Travel Aid for the Blind Based on Mobile RoboticsTechnology. IEEE Transactions on Biomedical Engineering (1998).arXiv:https://ykoren.engin.umich.edu/wp-content/uploads/sites/122/2014/05/51.-The-NavBelt-aid-for-the-blindbased-on-mobile-robot-technology.pdf

[12] Iwan Ulrich and Johann Borenstein. 2001. The GuideCane-applyingmobile robot technologies to assist the visually impaired. IEEE Trans-actions on Systems, Man, and Cybernetics-Part A: Systems and Humans31, 2 (2001), 131–136.

[13] Jan BF Van Erp. 2002. Guidelines for the use of vibro-tactile displays inhuman computer interaction. In Proceedings of eurohaptics, Vol. 2002.Citeseer, 18–22.

[14] Bruce N Walker and Jeffrey Lindsay. 2005. Navigation performance ina virtual environment with bonephones. Georgia Institute of Technol-ogy.

[15] Yunqing Wang and Katherine J Kuchenbecker. 2012. HALO: Hapticalerts for low-hanging obstacles in white cane navigation. In HapticsSymposium (HAPTICS), 2012 IEEE. IEEE, 527–532.

[16] S WEINSTEIN. 1968. Intensive and Extensive Aspects of Tactile Sen-sitivity as a Function of Body Part, Sex and Laterality. The First Int’l symp. on the Skin Senses (1968), 199–222. https://ci.nii.ac.jp/naid/10017541995/en/