A NOVEL HAPTIC INTERFACE FOR EXTENDED RANGETELEPRESENCE: CONTROL AND EVALUATION

Antonia Perez Arias and Uwe D. HanebeckIntelligent Sensor-Actuator-Systems Laboratory (ISAS)

Institute for AnthropomaticsUniversitat Karlsruhe (TH), Germany

Email: aperez@ira.uka.de and uwe.hanebeck@ieee.org

Keywords: Extended range telepresence, motion compression, haptic interface, force control

Abstract: A novel haptic interface for extended range telepresence is presented that allows the user simultaneous widearea motion and haptic interaction in remote environments. To achieve an extended workspace, the hapticinterface consists of a haptic manipulator for precise haptic rendering and a large portal carrier system thatenlarges the workspace by prepositioning the end-effector. As the prepositioning unit is grounded and drivenby three linear drives, our approach has the advantages of high force capability and an accurate positioning ofthe haptic interface. The use of this haptic interface with Motion Compression permits to explore large remoteenvironments even from small user environments. As a result, not only has the user visual, acoustic, and hapticfeedback, but can also control the teleoperator or avatar by natural walking, which considerably increases thesense of immersion. A prototype system for haptic extended range telepresence was designed, implemented,and tested.

1 INTRODUCTION

Telepresence aims at creating the impression ofbeing present in an environment, which is inacces-sible to a human user. Such an environment can bereal or virtual, and will be referred to in the follow-ing as target environment. The feeling of presence isachieved by visual and acoustic sensory informationrecorded from the target environment and presentedto the user on an immersive display.

The more of the user’s senses are involved, thebetter the immersion in the target environment. Inorder to use the sense of motion as well, which isespecially important for human navigation and wayfinding, the user’s motion can be tracked and trans-ferred to the teleoperator, a mobile robot or an avatar,in the target environment. As a result, in extendedrange telepresence the user can additionally use theproprioception, the sense of motion, to navigate theteleoperator by natural walking, instead of using de-vices like joysticks, pedals or steering wheels.

However, without further processing the motioninformation, the motion of the user is restricted tothe size of the user environment, which is limited, for

Figure 1: User and haptic interface for interaction with ex-tended target environments.

example by the range of the tracking system or theavailable space. Motion Compression (Nitzsche et al.,2004) is an algorithmic approach that provides a non-linear transformation, mapping the path in the targetenvironment to a feasible path in the user environmentby minimizing proprioceptive and visual inconsisten-cies.

Extended range telepresence can be applied inmany fields, especially in those that require the humannavigation skills to solve the task, for example tele-

exploration of remote environments, visualization ofcomplex structures, training of emergency evacua-tions, etc. An extended range telepresence systemthat uses Motion Compression to teleoperate a mo-bile robot is presented in (Roßler et al., 2005). Sincehaptic information is indispensable, amongst others,for haptic exploration and manipulation of objects inthe target environment, a novel haptic interface for theextended range telepresence system was built. A pic-ture of the system is shown in Fig. 1.

Force reflecting telepresence systems usually as-sume an immobile user and a restricted workspace.For example, industrial robots have often been usedas haptic interfaces due to their accuracy and relativehigh force capability (Hoogen and Schmidt, 2001)but their limited workspace makes them unfeasiblefor extended range telepresence. In the last years,several haptic interfaces that allow a dexterous feed-back and fairly high forces have been designed to en-large their workspaces, e.g. a string-based haptic in-terface (Bouguila et al., 2000), or a grounded hyper-redundant haptic interface in (Ueberle et al., 2003).Portable haptic interfaces like exoskeletons (Bergam-asco et al., 1994) solve the problem of wide area mo-tion, since the interface is carried along by the user.However, working with exoskeletons can be fatiguingfor the user due to the high weight of the system. Inaddition, they can display lower forces than groundeddisplays (Richard and Cutkosky, 1997). The onlygroup of systems that really allow haptic interactionduring wide area motion are mobile haptic interfaces(Nitzsche et al., 2003), (Formaglio et al., 2005), (Peeret al., 2007). These are usually small haptic devicesmounted on a mobile platform. Drawbacks of such in-terfaces are a difficult control and a dependency of theforce display quality on the localization of the mobileplatform.

In this paper, we present a novel haptic device thatallows haptic interaction in extended range telepre-sence and combines the advantages of grounded andmobile haptic interfaces. It consists of a grounded lin-ear prepositioning unit that moves along with the userand a manipulator arm attached to the prepositioningunit that is used to display forces at any position in theuser environment. This haptic interface allows in con-junction with Motion Compression unrestricted widearea motion and the possibility of effectively guidingthe user in the target environment by means of hap-tic information. The control of the haptic interface isbased on the decoupling of force control and preposi-tioning of the haptic device, which takes both the op-timal manipulator’s configuration and the user’s posi-tion into consideration.

The remainder of this paper proceeds as follows.

The following section presents the extended range te-lepresence system. Motion Compression is reviewed,since it determines the requirements of the haptic in-terface, and the mechanical setup of the haptic inter-face is presented. In section 3, a detailed descriptionof the control design is given. Experimental resultsare shown in section 4. Finally, a summary and anoutlook can be found in section 5.

2 EXTENDED RANGETELEPRESENCE SYSTEM

2.1 Motion Compression

Motion Compression provides a nonlinear mappingbetween the user’s path in the user environment andthe path in the target environment. It consists of threemodules: Path prediction tries to predict the desiredpath of the user in the target environment by meansof tracking data and, if possible, information of thetarget environment. The resulting path is called targetpath. Path transformation transforms the target pathso that it fits in the user environment. The resultinguser path conserves the length and turning angles ofthe target path while there is a minimum difference incurvature. Fig. 2 shows these paths in both environ-ments. Finally, the user guidance module guides theuser on the user path, while he has the impression ofwalking on the original target path.

4 m

4 m

8 m

7 m

User Environment Target Environment

Figure 2: The corresponding paths in both environments.Left: user path in the user environment. Right: target pathin the target environment.

The result of Motion Compression is a trans-formation between the user’s position in the user en-vironment and the teleoperator’s position in the tar-get environment at any time and position. This trans-formation can also be used to map the user’s hand po-sition, or to transform force vectors recorded by theteleoperator back into the user environment.

The use of Motion Compression for extendedrange telepresence puts a number of demands on thedesign of a haptic interface. The haptic interface must

be able to reach all configurations in a user environ-ment of 4×4 m2, in which the user may move with anatural speed of up to 2 m/s. Especially the rotationalmotion around the vertical axes must be unlimited.

2.2 A Novel Haptic Interface: Setup

Fine haptic rendering and wide area motion requirevery different characteristics regarding mechanics aswell as control. Therefore a novel haptic interfacewas designed that consists of two subsystems: a linearprepositioning unit that accompanies the user alongthe user path so that he does not perceive the hapticdisplay, and a manipulator arm attached to the prepo-sitioning unit that is used to display defined forces atany position in the user environment. In this way, theworkspace of the haptic interface covers the wholeuser environment. Fig.3 shows a CAD drawing of thecomplete setup.

The motion subsystem is realized as groundedportal carrier system of approximately 5× 5× 2 m3

with three translational degrees of freedom, which arerealized by three independent linear drives. Theselinear drives are built using a commercially availablecarriage-on-rail system. The carriages are driven by atoothed belt. The x- and y-axis consist of two parallelrails each for stability reasons, while the z-axis is onlya single rail. As a result, the system is driven by fivesynchronous AC-motors with helical-bevel servo gearunits of 120 Nm maximal torque, that allow a maxi-mum speed of 2 m/ s and an acceleration of 2 m/ s2.As the configuration space equals cartesian space, for-ward kinematics can be expressed by means of anidentity matrix. Thus position control is extremelyeasy to handle and very robust (Roßler et al., 2006).

This construction has the advantages of a a highforce capability and an accurate positioning of themanipulator, which is determined directly through en-coder’s information with relative accuracy 0.1 mm.By using a position control with high gains, theuser does not perceive the motion subsystem, andthe transparency depends only on the force-controlledsubsystem (Nitzsche et al., 2003).

Because the acceleration of the human hand is typ-ically much higher than the acceleration of the portalcarrier, a fast manipulator was used. It covers the hu-man arm workspace and has planar movement. It isimplemented as a planar SCARA arm, which is at-tached to the z-axis of the portal carrier. The redun-dant planar degrees of freedom permit the separationof positioning and force display. Two active rotationaljoints driven by two 150 W DC-motors are integratedinto the base, so that all moving joints are passive.Circular drives allow infinite motion around the z-

Linear prepositioning

unit

Manipulator arm

Figure 3: CAD drawing of the complete setup with linearpositioning unit and manipulator arm.

axis. The manipulator arm was designed to displaya force of 50 N at the end-effector. More details canbe found in (Roßler et al., 2006).

3 CONTROL DESIGN OF THEHAPTIC INTERFACE

3.1 Kinematic Model

The control of this haptic interface is based on the de-coupling of force control at the end-effector and po-sition control of the haptic device. The position ofthe end-effector with respect to the basis coordinateframe xE is described by the global position of thelinear prepositioning unit, xL, and xS, the relative po-sition of the manipulator arm with respect to the linearprepositioning unit as xE = xL + xS.

α

ψβ

L

E

x

y

R

l2

l1

ϕ

Figure 4: Geometrical SCARA-Model.

Fig. 4 shows the geometrical SCARA-model usedto derive the kinematic equations. L represents thelinear prepositioning unit, E the end-effector, and l1

and l2 the lengths of the inner and outer segments,respectively. If only the joints at the angles α and β

are actively driven, the end-effector position xE canbe expressed as

xE = xL + xS =[

xLyL

]+

cos(

α+β

2

)·R

sin(

α+β

2

)·R

, (1)

where R, the radial travel, is calculated as

R = l1 cos(

β−α

2

)+

√l22 − l2

1 sin2(

β−α

2

)(2)

With this equation the direct kinematics of the ma-nipulator is defined. The Jacobian of the manipulatorJ(γ) on the configuration space γ =

[α β

]T, whichwill be used next for the control of the haptic inter-face, is defined as

J(

γ

)=

∂xS

(γ

)∂γ

(3)

3.2 Control Structure

Fig. 5 illustrates the block diagram of the controlscheme with force feedback, as well as the positioncontrol of the haptic display. The end-effector ve-locity of the haptic interface xE is transmitted via thecommunication channel and acts as reference velocityat the end-effector of the teleoperator xE,re f ,T . The en-vironment reacts according to its impedance with a re-action FT , which is measured by the teleoperator, andtransmitted to the haptic interface as reference inputFT,re f ,U . This architecture represents a two-channelforce-velocity bilateral control.

In our system, the haptic interface is modelled asan admittance, which transforms FU,re f , the referenceforce to be displayed, into the reference motion of theend-effector as

FU,re f = M · xE,re f +D · xE,re f , (4)

where M is the desired mass matrix and D the de-sired damping matrix. The admittance control schemeis very well suited for systems with nonlinearitiesand large dynamic properties because the admittancemodel shapes the desired dynamic behaviour of thesystem by compensating the natural device dynamics(Ueberle and Buss, 2004).

The resolved acceleration control (J−1 control) isapplied to calculate the commanded motor torque ofthe manipulator τre f :

τre f = M ·J−1 · xC +h(

γ, γ)· γ+g

(γ

), (5)

where M is an approximation of the device joint iner-tia matrix and J is the Jacobian. The vectors h, repre-senting the friction effects, and g can be approximatedthrough experimental identification.

The user, while moving the end-effector, applies aforce consisting of a voluntarily applied force and areaction force induced by the arm impedance. In or-der to reconstruct the applied force from the measuredforce F∗U and the velocity of the end-effector, a modelof the human arm impedance is applied:

FU = F∗U +Mu · xE +Du · xE +Ku ·xE (6)

It is known that the arm impedance varies with theuser and the arm configuration. Hence, the mean val-ues of multiple users and planar configurations wereused: Mu = 2 Kg, Du = 6 Ns/m, and Ku = 10 N/ m.

The reference position of the linear prepositioningunit xL,re f , which can be easily controlled in cartesiancoordinates, is calculated by optimizing the manipu-lator’s configuration according to some performancemeasure.

3.3 Prepositioning

When attaching the SCARA manipulator to the portalcarrier, there is a redundancy in the planar directionsthat may be resolved by optimizing the manipulabil-ity of the SCARA. The manipulability is usually rep-resented as

w(

γ

)=√

det(

JT (γ) ·J(γ))

. (7)

For l1 = 0.285 m and l2 = 0.708 m, the SCARArobot’s manipulability was found to be optimal whenψ = β−α = 2.048.

Let’s consider the polar coordinates ofthe end-effector’s position: R = l1 cos

(ψ

2

)+√

l22 − l2

1 sin2 (ψ

2

), and φ = α+β

2 . Since the manipu-lability w is independent of φ, another criterion mustbe found to optimize this parameter. It is also crucialto avoid collisions with the user, therefore the angleφ is chosen that maximizes the distance d betweenthe user and the prepositioning unit by adoptingRopt = R(ψopt). By designating xEH the vector fromthe end-effector’s position to the user’s position, andθ the angle between this vector and the x-axis, thedistance d can be expressed as

d = Ropt 2 + |xEH |2−2R |xEH |cos(θ−φ+π) , (8)

and it is maximal when φopt = θ, or in other words,when the linear prepositioning unit is situated in frontof the user, and lies on the connecting line betweenthe user’s head and the end-effector. The optimal joint

Tele

oper

ator

Sim

ulat

ion

PositionController

PositionOptimization

Position Control

Force Control

F T,ref,U

FU

F ∗U

F T

xSτ ref

xL,refxH

xL

Com

mun

icat

ion

&M

otio

n C

ompr

essi

on

AdmittanceModel

ddt

ArmImpedance

PositionController

InverseDynamics

HapticDisplay

ddt

xE,ref,TxExE

xE,ref

xC

Figure 5: Control scheme of the haptic interface.

angles are finally αopt = φopt− ψopt

2 and βopt = φopt +ψopt

2 .

With xoptS =

[cos(φopt) ·Ropt sin(φopt) ·Ropt]T

being the optimal configuration of the manipulator,the reference position of the linear prepositioning re-sults xL,re f = xE − xopt

S .

4 EXPERIMENTS

Two kinds of experiments were performed in or-der to evaluate the proposed haptic interface. First,the proposed force control was tested and second, thesimultaneous wide area motion with haptic interac-tion was validated.

The force at the end-effector, and the positionsof both, end-effector and prepositioning unit, wererecorded during free motion and during a hard con-tact. In order to achieve transparency, the referenceforce during free motion is FT,re f ,U = 0 N. An admit-tance of M = 4 kg was simulated. The control gainsof the prepositioning and the admittance position con-troller were obtained experimentally using standardZiegler-Nichols.

Fig. 6 shows the force-position plots for the x-direction, when a user walks 15 seconds back andforth about 2 m in x-direction. Analogously, Fig. 7represents the reference and the measured force whena user walks against a wall at position −0.5 m withrigidity K = 700 N/m. The maximal displayed forceis limited to 60 N. Both figures also show the motionof the linear positioning unit at an optimal distance of

the end-effector.The main advantage of the admittance control is

that the desired mass and damping of the device canbe shaped. However, it is known that the admittancecontrol reduces the force bandwidth of the haptic sys-tem.

The prepositioning was tested together with thehaptic interaction to validate the entire concept of thehaptic interface. For this purpose, the virtual andthe user environment were supposed coincident, i.e.4× 4 m2 large, and two virtual walls were placed in-side. Fig. 8 shows the results of this experiment. Themotion of the user can be divided into four segments:a) the user moves toward the wall, b) the user walksalong the wall 1, c) the user walks along the wall 2,and d) the user turns on place. The haptic interface isalways on the opposite side of the end-effector, so thatthe danger of a collision with the user is avoided. Atthe same time, the distance between the end-effectorand the basis of the haptic interface is kept constanton the optimal value that maximizes the manipulabil-ity of the haptic display.

5 CONCLUSIONS

This paper presents a novel multi-modal telepre-sence system for extended range telepresence, whichenables the user to explore remote environments bynatural walking. To achieve this goal, a novel hapticinterface was developed, which consists of a hapticmanipulator mounted on a linear prepositioning unitthat follows the user by keeping the optimal configu-

0 5 10 15−2

−1

0

1

2

t/s

x/m

(a)

0 5 10 15−120

−100

−80

−60

−40

−20

0

20

t/s

F x /N

(b)

Figure 6: Position and force during free motion. (a) End-effector position xE (red), linear system position xL (blue). (b)Reference force Fx,re f = 0 N (red), actual force Fx (blue).

0 5 10 15−2

−1

0

1

2

t/s

x/m

(a)

0 5 10 15−120

−100

−80

−60

−40

−20

0

20

t/s

F x/N

(b)

Figure 7: Position and force by hard contact. (a) End-effectorposition xE (red), linear system position xL (blue). (b) Refe-rence force Fx,re f (red), actual force Fx (blue).

−2 −1 0 1 2−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

x /m

y/m

a

b

cd

Wall 1

Wall 2

Figure 8: Positions of user xH (blue), end-effector xE(green) and linear system xL (red) in presence of virtualwalls during wide area motion.

ration of the manipulator and avoiding collisions withthe user. For the haptic feedback, a dedicated forcecontrol was implemented and tested. It uses an admit-tance model to shape the dynamics of the system, aswell as a model of the impedances of arm and mani-pulator to compensate their undesired dynamics. Ex-periments show the suitability of this haptic interfacefor extended range telepresence.

The use of haptic information in extended rangetelepresence to improve the user guidance is a promis-ing application of the presented haptic interface,which is currently being investigated. For this ap-plication, the simultaneous compression of head andhand motion represents a further challenge.

REFERENCES

Bergamasco, M., Allotta, B., Bosio, L., Ferretti, L., Perrini,G., Prisco, G. M., Salsedo, F., and Sartini, G. (1994).”An arm exoskeleton system for teleoperation and vir-tual environment applications”. In Proceedings of theIEEE Intl. Conference on Robotics and Automation.

Bouguila, L., Ishii, M., and Sato, M. (2000). ”Multi-modal

haptic device for large-scale virtual environment”. InProceedings of the 8th ACM Intl. Conference on Mul-timedia.

Formaglio, A., Giannitrapani, A., Barbagli, F., Franzini, M.,and Prattichizzo, D. (2005). ”Performance of mobilehaptic interfaces”. In Proceedings of the 44th IEEEConference on Decision and Control and the Euro-pean Control Conference.

Hoogen, J. and Schmidt, G. (2001). ”Experimental resultsin control of an industrial robot used as a haptic inter-face”. In Proceedings of the IFAC Telematics Appli-cations in Automation and Robotics.

Nitzsche, N., Hanebeck, U. D., and Schmidt, G. (2003).”Design issues of mobile haptic interfaces”. Journalof Robotic Systems, 20(9):549–556.

Nitzsche, N., Hanebeck, U. D., and Schmidt, G. (2004).”Motion compression for telepresent walking in largetarget environments”. Presence, 13(1):44–60.

Peer, A., Komoguchi, Y., and Buss, M. (2007). ” Towards amobile haptic interface for bimanual manipulations”.In Proceedings of the IEEE/RSJ Intl. Conference onIntelligent Robots and Systems.

Richard, C. and Cutkosky, M. R. (1997). ”Contact forceperception with an ungrounded haptic interface”. InProceedings of ASME IMECE 6th Annual Symposiumon Haptic Interfaces.

Roßler, P., Armstrong, T., Hessel, O., Mende, M., andHanebeck, U. D. (2006). ”A novel haptic interface forfree locomotion in extended range telepresence sce-narios”. In Proceedings of the 3rd Intl. Conference onInformatics in Control, Automation and Robotics.

Roßler, P., Beutler, F., Hanebeck, U. D., and Nitzsche, N.(2005). ”Motion compression applied to guidance ofa mobile teleoperator”. In Proceedings of the IEEEIntl. Conference on Intelligent Robots and Systems.

Ueberle, M. and Buss, M. (2004). ”Control of kinesthetichaptic interfaces”. In Proceedings of the IEEE/RSJIntl. Conference on Intelligent Robots and Systems,Workshop on Touch and Haptics.

Ueberle, M., Mock, N., and Buss, M. (2003). ”Towardsa hyper-redundant haptic display”. In Proceedings ofthe International Workshop on High-Fidelity Telepre-sence and Teleaction.