6
Passivity-based control for haptic teleoperation of a legged manipulator in presence of time-delays Mattia Risiglione 1,2 , Jean-Pierre Sleiman 1 , Maria Vittoria Minniti 1 , Burak C ¸ izmeci 1 , Douwe Dresscher 3 and Marco Hutter 1 Abstract— When dealing with the haptic teleoperation of multi-limbed mobile manipulators, the problem of mitigating the destabilizing effects arising from the communication link between the haptic device and the remote robot has not been properly addressed. In this work, we propose a passive control architecture to haptically teleoperate a legged mobile manip- ulator, while remaining stable in the presence of time delays and frequency mismatches in the master and slave controllers. At the master side, a discrete-time energy modulation of the control input is proposed. At the slave side, passivity constraints are included in an optimization-based whole-body controller to satisfy the energy limitations. A hybrid teleoperation scheme allows the human operator to remotely operate the robot’s end- effector while in stance mode, and its base velocity in locomotion mode. The resulting control architecture is demonstrated on a quadrupedal robot with an artificial delay added to the network. I. INTRODUCTION Teleoperation offers solutions to extend the sensing and manipulation capabilities of human operators performing long-distance tasks. Recent advances of robotic platforms with loco-manipulation skills represent a promising resource for enlarging the operational workspace at the remote site. In particular, legged robots have the ability to locomote over rough terrains, interact with objects and perceive their surroundings. Hence, these platforms can be used as avatar systems to enhance the human’s full immersion in the remote environment and to safely perform complex operations in hazardous scenarios. Among all the feedback information that stimulates the human senses, haptics has been shown to be of paramount importance [1]. When signals flow bilaterally between the device handled by the human operator, named master de- vice, and the remote robot being teleoperated, named slave, we refer to bilateral teleoperation. In such an application, computing control laws with delayed and/or incomplete information can easily destabilize the system. Besides latency and packet losses, other components can negatively influence the stability of the system, such as the grasping force exerted This research was supported in part by the Swiss National Science Foun- dation through the National Centre of Competence in Research Robotics (NCCR Robotics), in part by the Swiss National Science Foundation through the National Centre of Competence in Digital Fabrication (NCCR dfab), in part by TenneT, and in part by Armasuisse Science and Technology. 1 Authors are with Robotics System Lab (RSL), ETH Zurich, Switzerland. 2 Author is with Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia (IIT). [email protected]. 3 Author is with Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, The Netherlands. Fig. 1: Experimental setup with Omega6 as haptic device at the master side (left) and the ANYmal C platform equipped with a robotic arm at the slave side (right). by the user, haptic feedback gains and environment stiff- ness. Closed-loop stability in haptics has been investigated since the late 80s[2]. In this context, passivity theory has been applied, with many proposed solutions differing in the way energy flows are monitored and limited [3], [4]. Among these, the Two-Layer architecture from [5] uses virtual storage elements to observe the energy exchanged between the master and slave device. Such an approach is made of two stages: first, the control law is computed, next energy dissipation is applied to act against a passivity loss of the bilateral controller. With respect to other methods such as wave variables [6] or time-domain approaches [7], passive controllers based on the Two-Layer approach (i.e., [5], [8], [9]) do not require any encoding of the power variables, nor rely on any assumptions on the sampling rate. Additionally, the transparency of the system can be improved by optimizing a transparency metric in the passivity layer [9]. For multi-degree of freedom (DOF) floating-base sys- tems, such as quadrupeds or humanoids, optimization-based techniques are prominently adopted for whole-body control (WBC) design [10], [11]. Such control schemes optimize control objectives for multiple tasks, while handling physical constraints, such as actuation limits, friction cone constraints, etc. Passive whole-body controllers have been proposed to compliantly and safely interact with the environment [12], [13]. However, in a haptic teleoperation scenario, existing work has focused on bounding [14] and mapping [15] the operator commands to ensure safe teleoperation and maintain balance, rather than designing a controller that is inherently robust against the effects of time-delays. One way of address- ing this problem is through an energy dissipation performed at the output of the whole-body controller, however this could interfere with the constrained optimal solution and render arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

Passivity-based control for haptic teleoperation of a legged manipulatorin presence of time-delays

Mattia Risiglione1,2, Jean-Pierre Sleiman1, Maria Vittoria Minniti1,Burak Cizmeci1, Douwe Dresscher3 and Marco Hutter1

Abstract— When dealing with the haptic teleoperation ofmulti-limbed mobile manipulators, the problem of mitigatingthe destabilizing effects arising from the communication linkbetween the haptic device and the remote robot has not beenproperly addressed. In this work, we propose a passive controlarchitecture to haptically teleoperate a legged mobile manip-ulator, while remaining stable in the presence of time delaysand frequency mismatches in the master and slave controllers.At the master side, a discrete-time energy modulation of thecontrol input is proposed. At the slave side, passivity constraintsare included in an optimization-based whole-body controller tosatisfy the energy limitations. A hybrid teleoperation schemeallows the human operator to remotely operate the robot’s end-effector while in stance mode, and its base velocity in locomotionmode. The resulting control architecture is demonstrated on aquadrupedal robot with an artificial delay added to the network.

I. INTRODUCTION

Teleoperation offers solutions to extend the sensing andmanipulation capabilities of human operators performinglong-distance tasks. Recent advances of robotic platformswith loco-manipulation skills represent a promising resourcefor enlarging the operational workspace at the remote site.In particular, legged robots have the ability to locomoteover rough terrains, interact with objects and perceive theirsurroundings. Hence, these platforms can be used as avatarsystems to enhance the human’s full immersion in the remoteenvironment and to safely perform complex operations inhazardous scenarios.

Among all the feedback information that stimulates thehuman senses, haptics has been shown to be of paramountimportance [1]. When signals flow bilaterally between thedevice handled by the human operator, named master de-vice, and the remote robot being teleoperated, named slave,we refer to bilateral teleoperation. In such an application,computing control laws with delayed and/or incompleteinformation can easily destabilize the system. Besides latencyand packet losses, other components can negatively influencethe stability of the system, such as the grasping force exerted

This research was supported in part by the Swiss National Science Foun-dation through the National Centre of Competence in Research Robotics(NCCR Robotics), in part by the Swiss National Science Foundation throughthe National Centre of Competence in Digital Fabrication (NCCR dfab), inpart by TenneT, and in part by Armasuisse Science and Technology.

1Authors are with Robotics System Lab (RSL), ETH Zurich, Switzerland.2Author is with Dynamic Legged Systems Lab, Istituto Italiano di

Tecnologia (IIT). [email protected] is with Faculty of Electrical Engineering, Mathematics and

Computer Science, University of Twente, The Netherlands.

Fig. 1: Experimental setup with Omega6 as haptic device atthe master side (left) and the ANYmal C platform equippedwith a robotic arm at the slave side (right).

by the user, haptic feedback gains and environment stiff-ness. Closed-loop stability in haptics has been investigatedsince the late 80s[2]. In this context, passivity theory hasbeen applied, with many proposed solutions differing inthe way energy flows are monitored and limited [3], [4].Among these, the Two-Layer architecture from [5] usesvirtual storage elements to observe the energy exchangedbetween the master and slave device. Such an approach ismade of two stages: first, the control law is computed, nextenergy dissipation is applied to act against a passivity lossof the bilateral controller. With respect to other methodssuch as wave variables [6] or time-domain approaches [7],passive controllers based on the Two-Layer approach (i.e.,[5], [8], [9]) do not require any encoding of the powervariables, nor rely on any assumptions on the sampling rate.Additionally, the transparency of the system can be improvedby optimizing a transparency metric in the passivity layer [9].

For multi-degree of freedom (DOF) floating-base sys-tems, such as quadrupeds or humanoids, optimization-basedtechniques are prominently adopted for whole-body control(WBC) design [10], [11]. Such control schemes optimizecontrol objectives for multiple tasks, while handling physicalconstraints, such as actuation limits, friction cone constraints,etc. Passive whole-body controllers have been proposed tocompliantly and safely interact with the environment [12],[13]. However, in a haptic teleoperation scenario, existingwork has focused on bounding [14] and mapping [15] theoperator commands to ensure safe teleoperation and maintainbalance, rather than designing a controller that is inherentlyrobust against the effects of time-delays. One way of address-ing this problem is through an energy dissipation performedat the output of the whole-body controller, however this couldinterfere with the constrained optimal solution and render

arX

iv:2

108.

0765

8v1

[cs

.RO

] 1

7 A

ug 2

021

Page 2: arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

it infeasible. Here, instead of having a control computinglayer (transparency layer) and a dissipation layer (passivitylayer), we directly create an optimal passivity controller thatrespects the energy constraints.

Hence, we propose a control framework for the time-delayed haptic teleoperation of a legged robot, that is inparticular capable of mitigating the destabilizing effectscaused by time delays in the communication link. To thisend, we provide new insights for the formulation of energyconstraints in optimization-based whole-body controllers.We then validate the effectiveness of our approach with aset of hardware experiments performed on a quadrupedalmobile manipulator, where the robot is teleoperated duringend-effector or base control while introducing artificial yetrealistic delays in the network. It is worth noting that theresulting architecture is not specific for quadrupedal robots,but is general enough to be applied to any floating-basesystem, such as humanoids or wheeled robots.

II. BILATERAL TELEOPERATION PROBLEM

In this section, we first provide some necessary back-ground on passivity theory. Afterwards, we describe theconsidered bilateral teleoperator, and how the passivity re-quirements can be satisfied with the use of energy tanks.

A. Background on passivity theory

Consider a state-space system given by:

x = f(x,u)

y = h(x,u),(1)

where x ∈ X ⊆ Rnx , u ∈ U ⊆ Rnu , y ∈ Y ⊆ Rny .Definition 2.1 ([16]): System (1) is said to be passive

with respect to the input u and output y if there exists acontinuous function, H(·) ≥ 0, called storage function, suchthat

H(t)−H(0) ≤∫ t

0

uT (s)y(s) ds, (2)

∀ t ≥ 0, for all input signals u ∈ U ⊆ Rnu , for all initialstates x(0) ∈ X and nu = ny .Equivalently, we can define system (1) to be passive if∃H(·) ≥ 0 such that H(t) ≤ uT (t)y(t).

Theorem 2.1 ([17]): Consider the two systems

Σi :

{xi = f i(xi,ui)

yi = hi(xi,ui),(3)

where i = 1, 2, nu = ny , interconnected with the followingpower-preserving feedback interconnection:[

u1

u2

]=

[0nu×nu W−W 0nu×nu

] [y1

y2,

](4)

where W := diag(w1, ..., wnu), W < 0. If Σi is passivewith respect to the mapping ui → yi with storage functionHΣi for i = 1, 2, then the interconnected system Σ1 ∧Σ2 isalso passive and the overall storage function is such that{

HΣ1∧Σ2= HΣ1

+HΣ2

HΣ1∧Σ2= 0.

(5)

B. Control by interconnection

The closed-loop system in Fig. 2 is passive if its interactingcomponents satisfy (2), and are interconnected accordingto (4). Being mechanical systems, the master and slaveare passive, as it can be deduced considering the systemHamiltonian as a storage function. Therefore, a passivecontrol unit needs to be designed to convey energy betweenthe haptic interface and the remote robot. This can be done byensuring that the assumptions of Theorem (2.1) are satisfied,i.e. that the passivity requirement (2) holds for the bilateralteleoperator and this results passively interconnected to themaster and slave devices. In the next section, we show howto preserve the passivity requirements using the concept ofvirtual energy tanks [18].

C. Passivity control with energy tanks

Energy tanks are dynamical systems that are connected tothe master and slave systems and exchange energy betweeneach other. The energy stored inside each tank is equal to theone transferred from the other tank, and exchanged with thephysical world, and represents the energy budget that can beused to generate desired active behaviours. We denote thesetwo storage elements as: Σmt(umt, ymt) and Σst(ust, yst).The energy tanks are modeled as dynamical systems withthe following dynamics:

Σmt :

{xmt = umt +

(P+mt−P

−mt

xmt

)ymt = ∂xmtHmt = xmt

(6a)

Σst :

{xst = ust +

(P+st−P

−st

xst

)yst = ∂xstHst = xst,

(6b)

where Hmt = 12xmt

2, Hst = 12xst

2 are the tanks storagefunctions, and P+ ≥ 0 and P− ≥ 0 indicate the incomingand outgoing power exchanged between the two tanks. Inthis work, energy is exchanged only if present inside thetank, according to the energy transfer protocol described in[5]. The systems Σmt and Σst are connected to the masterand slave ports, respectively, as follows:

umcumtuscust

=

0 wm

xmt0 0

−wTmxmt

0 0 0

0 0 0 wsxst

0 0 −wTsxst

0

ymymtysyst

Dm

Ds

, (7)

where wm ∈ Rnym and ws ∈ Rnys are steer valuescomputed by the master and slave controllers, based on thestates of the two tanks xmt and xst. The matrices Dm andDs that interconnect the tanks to the respective systems, arein the form of Eq. (4). The resulting bilateral teleoperatorenergy rate matches the power flowing from master and slavedevices into the master and slave controllers, indeed

Hmt + Hc + Hst = −uTmcym − uTscys. (8)

where Hc = P−mt−P+mt+P

−st−P+

st is the energy stored in thecommunication channel. Therefore, the bilateral teleoperator

Page 3: arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

Σmt Σst

Tank-basedmaster controller

Tank-basedWBC

Dm Dsym

umc

usc

ys

BILATERAL TELEOPERATOR

Fig. 2: Diagram of main elements interacting, from left to right: the human operator and the haptic device, the bilateralteleoperator, the slave and the environment.

is passive with respect to input (−umc,−usc) and output(ym,ys). However, if the energy tanks are depleted, thereis no possibility to passively implement the desired action.Therefore, the only requirement that we need to impose topreserve the passivity of the system is that the energy levelinside each tank is positive. In the next section, we will showhow we impose this requirement in the master and slavecontrollers.

III. HYBRID TELEOPERATION CONTROLIn the following sections we provide the implementation

details of the proposed setup. First, we introduce the modelof the slave, which is described by the equations of motionof a multi-limbed floating-base system:

Mu(q)v + bu(q,v) = JTcu(q)λ

Ma(q)v + ba(q,v) = JTca(q)λ + usc,(9)

where q ∈ SE(3) × Rna and v ∈ R6+na are the robotgeneralized coordinates and velocities, M is the mass matrix,b includes the nonlinear effects (i.e. Coriolis, centrifugal,and gravitational terms), and usc is the vector of actuationtorques. Additionally, Jc is a matrix of stacked contactJacobians, while λ is the vector of contact wrenches. Thesubscripts u and a correspond to the unactuated and actuatedparts of the defined quantities, respectively. We formulatethe slave whole-body control problem as in [11], wherea hierarchical QP is solved to determine the vector ofdesired generalized accelerations (v∗), and desired contactforces (λ∗). We denote the vector of optimization variableswith ξ = (v∗,λ∗). A set of equality tasks encode dynamicconsistency, no slipping or separation of stance legs, as wellas tracking of base and limb motion references. Additionally,a set of inequalities is defined to encode friction coneconstraints, joint torque limits, in addition to the passivityconditions. The priorities of each task are reported in Table Iand will be discussed later in this section. It is worthmentioning that the reference commands from the masterdevice are not directly sent to the whole-body controller,but are given as inputs to an intermediate motion planner[19]. Briefly, this planner solves a model predictive control(MPC) problem to generate optimal motion references forthe robot’s base and limbs. These references are then trackedby the passivity-preserving QP hierarchical controller. Giventhe optimal solution ξ of the WBC, the joint torques arecomputed by inverting the desired dynamics:

usc = [Ma(q) − JTca(q)] · ξ + ba(q,v). (10)

A. End-effector position control

1) Master controller: Since the controller is implementedon an embedded unit, a discrete-time rule needs to becomputed to update the energy tank level [20]:

Hmt(k) =

∫ k∆T

0

(umt +

P+mt − P−mtxmt

)ymtdt

=

∫ k∆T

0

−wTm(t)ym(t)dt+

∫ k∆T

0

[P+mt(t)− P−mt(t)]dt

=

k∑j=1

−wTm(j − 1)∆xm(j) + ∆H+

mt(j)−∆H−mt(j)

= Hmt(k − 1)−wTm(k − 1)∆xm(k) + ∆Hmt(k), (11)

with ∆xm(k) = xm(k)− xm(k − 1), ym = xm, and∆Hmt(k) is the overall exchanged energy at time k. LetFc be the desired haptic feedback, which is updated basedon feedback information from the slave’s end-effector. Wecompute the master-side steer value wm as a function ofboth the state of the tank and the desired haptic feedback:

wm(k) =

{Fc(k) if Hmt(k) ≥ ζx2mt(k)γ2 Fc(k)− Γ · xm if 0 < Hmt(k) < ζ

(12)where ζ > 0 is the minimum energy level in the tankintroduced to avoid singularities in the solution, γ =

√2 · ζ,

and Γ � 0 is a diagonal matrix whose entries are equal toα · (ζ −Hmt(k)) with α > 0. When the value of the tankdecreases below this threshold, Fc is modulated accordingto the state of the tank and, additionally, a dissipative termis added to extract energy from the master. The type ofmodulation proposed here offers the advantage of beingsmooth, and does not strictly require any estimate of thevelocity at the next sample. Generally, up until the nextsample is available, there is no way to act on a loss ofpassivity. However, if ζ is chosen as the maximum allowableenergy to spend, this problem is avoided.

2) Slave controller: At the slave side, a similar update forthe tank can be obtained as follows:

Hst(k) = Hst(k − 1)−wsT (k − 1)∆qs(k) + ∆Hst(k).

(13)

At each sample, Hst is updated with the previously appliedtorques ws(k − 1) = usc(k − 1), the joint displacement∆qs(k) = q(k)−q(k−1), and the exchange with the mastertank ∆Hst(k) = ∆H+

st(k)−∆H−st(k). The control law (10)

Page 4: arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

TABLE I: List of tasks used in the WBC. Each task isassigned a priority. The tasks introduced in this work arehighlighted, with reference to the corresponding equations.

Priority Task1 Equations of motion

Torque limitsFriction coneZero acceleration at the contact

2 Base motion direction (21)Arm motion direction (18)

3 Base passivity constraint (20)Arm passivity constraint (15)Base motion trackingLimbs motion tracking

needs to impose that the energy level inside the tank at eachinstant of time is larger than zero. Since the energy exchangewith the master tank ∆Hst at sample k+ 1 is unknown, weimpose the following inequality constraint:

Hst(k + 1) = Hst(k)− uTsc(k)q(k)∆T ≥ ε, (14)

where Hst is an estimate of the tank level at time k + 1,ε > 0 is a threshold for the minimum possible energy level,and the joint displacements ∆qs(k + 1) are estimated fromthe current joint velocities.

During teleoperation of the arm end-effector, we toleratesmall variations of the torso, resulting in (14) being mainlyaffected by the joints of the arm. Under this consideration,Eq. (14) can be rewritten taking (10) into account, as:[

qTr Mr −qT

r JTcr

]ξ ≤ Hst(k)− ε

∆T− qT

r (br − gr) (15)

where the subscript r corresponds to the arm degrees offreedom, Mr includes the last nr rows of Ma, and gr are thearm gravitational torques. These can be subtracted from theenergy budget since such a compensation preserves passivity.

Let q∗r and q∗r be the arm joint references computed bythe MPC planner. A desired acceleration is generated with aPD control law:

qdesr = Kp(q∗r − qr) + Kd(q

∗r − qr). (16)

An equality task is then formulated to track the resultingdesired acceleration

[Onr×nz Inr×nr Onr×nλ ] · ξ = qdesr , (17)

where nz = na + 6− nr. To avoid having the passivity taskinvert the direction of the desired accelerations, we imposean additional constraint on the arm’s motion, given by:

[Onr×nz Qr Onr×nλ ] · ξ ≥ 0, (18)

where Qr ∈ Rnr×nr is a diagonal matrix with diagonalentries equal to the desired joint accelerations as in (16).A higher priority (see Table I) is assigned to (18) whencompared to the passivity and tracking tasks. In this waywe ensure that the desired joint-space motion direction ispreserved, while jointly optimizing for passivity and trans-parency (i.e tracking) with a lower priority. One could expectthat the passivity task should be at the highest priority.

However, as pointed out in [21], since passivity is onlysufficient for stability, strictly enforcing passivity could leadto an overly conservative behavior. Hence, for our controlarchitecture, enforcing stability at a lower priority allows torespect the stability requirements, while still guaranteeing agood tracking performance.

B. Base velocity control

1) Master controller: The base velocity teleoperationproblem is challenging since the haptic device has a limitedworkspace. Ideally, it would be desirable to map mas-ter positions to slave velocity commands. However, themaster device is passive with respect to the power port(umc, xm) rather then (umc,xm). As in [22], we overcomethis issue by rendering the master passive with respect tothe port (umc, rm), where the output rm is defined as:rm = xm + Λxm, with Λ � 0; and a local control actionis added to umc as −Bxm−Kxm. The matrix B is chosensuch that B � MmΛ, with Mm being the mass matrix ofthe haptic device, K � 0. As shown in [23], the signal rmcan be made proportional only to the scaled position if thecontribution of velocity is negligible due to the choice of Λ.

2) Slave controller: The slave degrees of freedom con-trolled in velocity mode are the lateral and longitudinalvelocities of the base. No references are sent along thevertical direction, however we still allow for feedback. In thisway, if the base of the robot is disturbed along this direction,the operator could still perceive the applied force. Sincethe floating-base of a legged robot is driven by generatingcontact forces on the environment, we impose the energeticlimitation with respect to the xy (non-vertical) forces exertedby the stance legs:

Hst(k + 1) = Hst(k)− (JTuxyλ)T xsxy∆T ≥ ε, (19)

where JTuxy includes the first two rows of (JTcu ), and xs isthe base velocity. Using (9), we can rewrite (19) as:

[xTsxyMxy 02×nλ ] · ξ ≤ Hst(k)− ε

∆T− xT

sxy (bxy − gxy), (20)

which can be added as a task in the hierarchical QP. As inSec. III-A, the desired linear accelerations calculated by themotion planner should be preserved. These accelerations areexpressed in a specific frame, named control frame C [11],which changes according to the slope of the terrain and thebase orientation. To prevent changes in desired references,we add the following additional inequality constraint to thestack of tasks:[

QxyJuxy 02×nλ]· ξ + QxyJuxyv ≥ 0, (21)

where Qxy ∈ R2×2 is a diagonal matrix with entries equal tothe elements of the desired linear acceleration C rdesxy , whichis computed as follows:

C rdesxy = C r∗xy+Kd(C r∗xy−C rxy)+Kp(Cr∗xy−Crxy). (22)

Page 5: arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

0 2 4 6−0.1

−0.050

0.050.1

y[m

]

master slave

0 5 10 15 20

−0.1

0

0.1

0 2 4 6

−10

0

10

Forc

efe

edba

cky

[N]

Fcwm

0 5 10 15 20

−2

0

2

0 2 4 6

−0.5

0

t [s]

Ener

gyta

nk[J]

Hm

Hs

0 5 10 15 200

0.1

0.2

t [s]

Fig. 3: Master and slave end-effector position, force andenergy tanks levels with passivity layer (right), and without(left), during an experiment of end-effector control.

IV. EXPERIMENTAL RESULTS

We validated our proposed teleoperation frameworkthrough experiments performed on a real robot platform withan artificial delay introduced in the network. Videos of allthe experiments are available in the attached multimediamaterial. The master device used for the experiments isthe Force Dimension Omega.6 haptic device1. The slaverobot is ANYmal – a torque-controlled quadrupedal robot –equipped with a 4 DOF robotic arm. The control loop runsat 1 kHz on the haptic device, while the ANYmal whole-body controller and the state estimator run at 400 Hz on therobot’s onboard computer (Intel Corei7-8850H CPU@4GHzhexacore processor). Each experiment is performed withoutand with the proposed passivity framework. We provideresults for end-effector position control and base velocitycontrol, with an end-to-end added delay of 60 ms for thetwo scenarios. This delay was observed as the averagecommunication delay between the University of Twente andour laboratory in Zurich.

A. End-effector position control

In this experiment, the human operator moves the hapticdevice along a predefined trajectory, defined in such a wayto span the whole workspace up to the boundaries of thehaptic device. For this experiment, the local force feedbackused in Eq. (12) was generated from the displacementerror between the master and the slave measured position:Fc(k) = Kp(xs(k − kd) − xm(k)) where kd is the timedelay. Results for this experiment are displayed in Fig. 3where, for the sake of conciseness, only the plots along they direction are reported. It can be seen that, without thepassivity layer, the two devices show oscillatory signals thatare out-of-phase. In this condition, to avoid damages to thehardware, the system is frozen as soon as the oscillations startto grow. Another indicator of unstable behavior is observedin the plot of the energy tank level, where a drop to anegative value occurs. Conversely, with passivity layer, thisbehavior is avoided and leads to smoother master and slavetrajectories.

1https://www.forcedimension.com/products/omega

0 5 10 15 20

−0.2

0

0.2

Velo

city

x[m

/s]

masterslave

0 10 20 30 40

−0.2

0

0.2

0 5 10 15 20

−0.2

−0.1

0

0.1

0.2

t [s]

Velo

city

y[m

/s]

0 10 20 30 40

−0.2

0

0.2

t [s]

Fig. 4: Master and slave x-y velocity profiles with passivitylayer (right), and without (left) in base velocity control.

0 5 10 15 20

−5

0

5

Forc

efe

edba

ckx

[N]

Fcwc

0 10 20 30 40

−5

0

5

0 5 10 15 20

−5

0

5

Forc

eFe

edba

cky

[N]

0 10 20 30 40

−4

−2

0

2

4

0 5 10 15 20

−2

−1

0

t [s]

Ener

gyta

nk[J

]

Hm

Hs

0 10 20 30 400

0.1

0.2

0.3

t [s]

Fig. 5: Force and energy tanks levels with passivity layer(right), and without (left) in base velocity control.

B. Base velocity control

In this experiment, we control the base velocity ofthe slave. To avoid hardware damage we limit the max-imum commanded velocity to 0.2 m/s. For this test,the local force feedback used in Eq. (12) was gener-ated from the velocity tracking error at the slave side:Fc(k) = Kv(vm(k − kd)− vs(k)), where vm, and vs de-note the master velocity, and the slave base velocity, respec-tively. vm is computed from the master position using themapping from Section III-B. This haptic feedback allowsto realistically perceive external forces, and to capture theireffect without the need for any haptic sensory feedback.It can be noted from Fig. 4 that, with passivity layer, thevelocity tracking error increases as the master velocity isabout to reach 0.2 m/s. While the master velocity is aboutreaching this peak, the mean absolute percentage trackingerror in velocity is 6.98% without passivity layer, and only1.1% with passivity layer. The former scenario happens inorder to avoid violating the passivity condition as seen inthe energy level plots of Fig. 5, which in turn leads tothe energetic penalization of the contact forces along thedirections where they create motion. As shown in Fig. 4, ifpassivity is not enforced, the force feedback causes the hapticdevice to become unstable, and this in turn compromises

Page 6: arXiv:2108.07658v1 [cs.RO] 17 Aug 2021

Fig. 6: 3D position trajectories for the slave end-effectorduring two experiments of haptic interaction with a human,with energy tanks off (left), and on (right).

tracking. Enforcing passivity allows to avoid this problemand results in a stable teleoperation system.

C. Haptic interaction during end-effector control

In this experiment, we perform a haptic interaction testwith a human at the slave side shaking the robot’s hand.Haptic feedback was generated as in Sec. IV-A. For the sakeof conciseness, we only present results for the slave response(Fig. 6). From both the plots and the video attachment,it can be verified that, with passivity, the trajectories aresmoother and the system remains stable. Gains for the mastercontroller were tuned in such a way that enables the operatorto perceive low magnitude forces acting on the slave end-effector. We realized that, if it is desired that the operatorfeels the hand-shake, these gains tend to be high, leading toa more unstable behavior. Indeed, in haptic teleoperation,instabilities are due not only to time delays, but also toother sources, such as those deriving from a controller tuningthat aims to enhance transparency. However, it can be notedfrom Fig. 6 that our formulation is also effective in handlingthis additional source of instability.

V. CONCLUSIONS

In this paper, we presented an approach to ensure thestability of a haptic teleoperation system for a quadrupedalmobile manipulator. The crucial point to ensure such aproperty is to manage a potential active behavior of thebilateral teleoperator. We addressed such a problem for thebilateral teleoperation of a general floating-base system, inthe presence of destabilizing time delays. To this end, energytanks were used as energy observers and included in theformulation of a passive QP controller at the slave side.We experimentally demonstrated that, in the presence oftime delays, the proposed passivity formulation based onenergy tanks effectively leads to more stable operations.One interesting direction for future work includes addingfeedback from force sensor measurements, and extending theproposed method to perform tele-manipulation tasks, such asremotely turning valves or opening doors.

REFERENCES

[1] J. G. Wildenbeest, D. A. Abbink, C. J. Heemskerk, F. C. VanDer Helm, and H. Boessenkool, “The impact of haptic feedbackquality on the performance of teleoperated assembly tasks,” IEEETransactions on Haptics, vol. 6, no. 2, pp. 242–252, 2012.

[2] R. J. Anderson and M. W. Spong, “Bilateral control of teleoperatorswith time delay,” in IEEE International Conference on Systems, Man,and Cybernetics, vol. 1. IEEE, 1988, pp. 131–138.

[3] P. F. Hokayem and M. W. Spong, “Bilateral teleoperation: An histor-ical survey,” Automatica, vol. 42, no. 12, pp. 2035–2057, 2006.

[4] R. Muradore and P. Fiorini, “A review of bilateral teleoperationalgorithms,” Acta Polytechnica, vol. 13, no. 1, pp. 191–208, 2016.

[5] M. Franken, S. Stramigioli, S. Misra, C. Secchi, and A. Macchelli,“Bilateral telemanipulation with time delays: A two-layer approachcombining passivity and transparency,” IEEE transactions on robotics,vol. 27, no. 4, pp. 741–756, 2011.

[6] G. Niemeyer and J.-J. Slotine, “Stable adaptive teleoperation,” IEEEJournal of oceanic engineering, vol. 16, no. 1, pp. 152–162, 1991.

[7] J.-H. Ryu, D.-S. Kwon, and B. Hannaford, “Stable teleoperation withtime-domain passivity control,” IEEE Transactions on robotics andautomation, vol. 20, no. 2, pp. 365–373, 2004.

[8] E. Sartori, C. Tadiello, C. Secchi, and R. Muradore, “Tele-echographyusing a two-layer teleoperation algorithm with energy scaling,” in In-ternational Conference on Robotics and Automation, ICRA, Montreal,QC, Canada. IEEE, 2019, pp. 1569–1575.

[9] O. A. M. Franco, J. Bimbo, C. Pacchierotti, D. Prattichizzo, D. Bar-celli, and G. Bianchini, “Transparency-optimal passivity layer designfor time-domain control of multi-dof haptic-enabled teleoperation,” in2018 IEEE/RSJ International Conference on Intelligent Robots andSystems (IROS). IEEE, 2018, pp. 4988–4994.

[10] L. Sentis and O. Khatib, “Synthesis of whole-body behaviors throughhierarchical control of behavioral primitives,” International Journal ofHumanoid Robotics, vol. 2, no. 04, pp. 505–518, 2005.

[11] C. D. Bellicoso, C. Gehring, J. Hwangbo, P. Fankhauser, and M. Hut-ter, “Perception-less terrain adaptation through whole body control andhierarchical optimization,” in IEEE-RAS 16th International Conferenceon Humanoid Robots (Humanoids). IEEE, 2016, pp. 558–564.

[12] S. Fahmi, C. Mastalli, M. Focchi, and C. Semini, “Passive whole-bodycontrol for quadruped robots: Experimental validation over challengingterrain,” IEEE Robotics and Automation Letters, vol. 4, no. 3, pp.2553–2560, 2019.

[13] B. Henze, M. A. Roa, and C. Ott, “Passivity-based whole-body balanc-ing for torque-controlled humanoid robots in multi-contact scenarios,”The International Journal of Robotics Research, vol. 35, no. 12, pp.1522–1543, 2016.

[14] G. Xin, J. Smith, D. Rytz, W. Wolfslag, H.-C. Lin, and M. Mistry,“Bounded haptic teleoperation of a quadruped robot’s foot posture forsensing and manipulation,” in 2020 IEEE International Conference onRobotics and Automation (ICRA). IEEE, 2020, pp. 1431–1437.

[15] W. Gomes, V. Radhakrishnan, L. Penco, V. Modugno, J. Mouret, andS. Ivaldi, “Humanoid whole-body movement optimization from retar-geted human motions,” in 19th IEEE-RAS International Conference onHumanoid Robots, Humanoids 2019, Toronto, ON, Canada, October15-17, 2019. IEEE, 2019, pp. 178–185.

[16] T. Gilead, L. Rogelio, B. Bernard, E. Olav, and M. Bernhard, “Dissipa-tive systems analysis and control,” Autom., vol. 41, no. 1, pp. 177–179,2005.

[17] C. Ebenbauer, T. Raff, and F. Allgower, “Dissipation inequalities insystems theory: An introduction and recent results,” in Invited lecturesof the international congress on industrial and applied mathematics,vol. 2007, 2009, pp. 23–42.

[18] T. S. Tadele, T. J. De Vries, and S. Stramigioli, “Combining energy andpower based safety metrics in controller design for domestic robots,” inIEEE International Conference on Robotics and Automation (ICRA).IEEE, 2014, pp. 1209–1214.

[19] J.-P. Sleiman, F. Farshidian, M. V. Minniti, and M. Hutter, “A unifiedmpc framework for whole-body dynamic locomotion and manipula-tion,” IEEE Robotics And Automation Letters, 2021.

[20] S. Stramigioli, C. Secchi, A. J. van der Schaft, and C. Fantuzzi, “Sam-pled data systems passivity and discrete port-hamiltonian systems,”IEEE Transactions on Robotics, vol. 21, no. 4, pp. 574–587, 2005.

[21] M. Selvaggio, P. R. Giordano, F. Ficuciellol, and B. Siciliano, “Passivetask-prioritized shared-control teleoperation with haptic guidance,” inInternational Conference on Robotics and Automation (ICRA). IEEE,2019, pp. 430–436.

[22] D. Lee and D. Xu, “Feedback r-passivity of lagrangian systems formobile robot teleoperation,” in IEEE International Conference onRobotics and Automation. IEEE, 2011, pp. 2118–2123.

[23] A. Franchi, C. Secchi, H. I. Son, H. H. Bulthoff, and P. R. Giordano,“Bilateral teleoperation of groups of mobile robots with time-varyingtopology,” IEEE Transactions on Robotics, vol. 28, no. 5, pp. 1019–1033, 2012.