6
Adaptive Nonlinear Teleoperation Control in Multi-master/Multi-slave Environments (Invited Paper) Shahin Sirouspour and Peyman Setoodeh Department of Electrical and Computer Engineering, McMaster University Hamilton, Ontario, L8S 4K1, Canada [email protected], [email protected] Abstract— An adaptive nonlinear control framework for multi-master/multi-slave teleoperation is proposed. A fully con- nected communication architecture is considered, which allows for transmission of position and force information between all master and slave robots. The operators are presented with a virtual intervening tool in order to collaboratively interact with the task environment. Models of operators, master and slave robots, tool, and environment are incorporated in the design. The performance and stability of cooperative teleoperation are guaranteed under dynamic interactions between slave robots and in the presence of model uncertainty. The robustness of the control system with respect to communication latency is also analyzed. Experimental studies demonstrate that the proposed approach is highly effective in all phases of a teleoperation task, i.e. in free motion and in contact with both flexible and rigid environments. Index Terms— Cooperative Teleoperation, Multi- master/Multi-slave Teleoperation, Robot Control, Nonlinear Control, Adaptive Control. I. I NTRODUCTION Over the past two decades, applications of teleoperation technology have been steadily growing in areas such as space operation, underwater exploration, mining, toxic and nuclear material handling, the entertainment industry, and more recently in health care [1]. Telerobotic systems deliver the human intelligence and manipulation skills combined with robot precision, repeatability and power to inaccessi- ble and/or remote environments. This is achieved through coordinated control of a master robotic arm, locally used by the operator, and a slave manipulator which mimics the operator’s actions in the task environment. Cooperative robotic manipulation has advantages over single-robot manipulation, such as increased dexterity, im- proved handling capability, increased loading capacity, and enhanced robustness due to redundancy. Cooperative tele- operation combines advantages of human collaboration and robotic manipulation. It can be employed in traditional tele- operation applications and in emerging areas such as robot- assisted surgery, e.g. a surgeon may use two robotic arms to This work was supported by Natural Sciences and Engineering Research Council of Canada, Canada Foundation for Innovation, and Ontario Innova- tion Trust. telemanipulate a surgical instrument. Despite the extensive amount of research in teleoperation, e.g. [2]–[7], and coor- dinated control of robots, e.g. [8], [9], multi-master/multi- slave teleoperation has received little attention in the past. This includes the work in [10] which proposes the use of heuristic methods such as predictive graphic displays to address the time delay in the communication channels in cooperative telerobotic environments. Approaches based on Petri Net models and event-based planning and control have been recently developed in [11] for remote operation in multi- robot environments. An Internet-based distributed, multiple- telerobot system that enables operators to use remote robots in order to perform cooperative tasks has been introduced in [12]. In cooperative teleoperation, multiple slave robots dynam- ically interact, directly or through an intervening tool dynam- ics. Designs based on conventional single-master/single-slave architectures fail to address the performance and stability re- quirements of cooperative teleoperation as they often neglect these interactions. The issues of performance and stability for such systems must be addressed under these constraints. This paper follows upon the first author’s earlier work [13] in which a general architecture for cooperative teleoperation control was introduced. In that work, a µ-synthesis based robust linear controller was developed assuming dynamically linearized models for the operators, masters, slaves, and envi- ronment. The controller utilizes all possible communication channels between the master and slave manipulators. This paper proposes adaptive nonlinear control to accommodate nonlinearity and parametric uncertainty in these dynamics. The control laws are inspired by those given in [6] for con- ventional single-master/single-slave teleoperation. The major contribution of this paper is extending such control approach to cooperative teleoperation applications where multiple mas- ters and slaves are involved. The proposed multilateral teleoperation controller estab- lishes position-position kinematic correspondence between the masters and slaves. It also synthesizes an adjustable intervening tool impedance between the operators and the environment. The stability of the closed-loop system in free Proceedings of the 2005 IEEE Conference on Control Applications Toronto, Canada, August 28-31, 2005 WA3.5 0-7803-9354-6/05/$20.00 ©2005 IEEE 1263

[IEEE 2005 IEEE Conference on Control Applications, 2005. CCA 2005. - Toronto, Canada (Aug. 29-31, 2005)] Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005

Embed Size (px)

Citation preview

Page 1: [IEEE 2005 IEEE Conference on Control Applications, 2005. CCA 2005. - Toronto, Canada (Aug. 29-31, 2005)] Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005

Adaptive Nonlinear Teleoperation Control inMulti-master/Multi-slave Environments

(Invited Paper)

Shahin Sirouspour and Peyman Setoodeh

Department of Electrical and Computer Engineering, McMaster UniversityHamilton, Ontario, L8S 4K1, Canada

[email protected], [email protected]

Abstract— An adaptive nonlinear control framework formulti-master/multi-slave teleoperation is proposed. A fully con-nected communication architecture is considered, which allowsfor transmission of position and force information between allmaster and slave robots. The operators are presented with avirtual intervening tool in order to collaboratively interact withthe task environment. Models of operators, master and slaverobots, tool, and environment are incorporated in the design.The performance and stability of cooperative teleoperation areguaranteed under dynamic interactions between slave robotsand in the presence of model uncertainty. The robustness of thecontrol system with respect to communication latency is alsoanalyzed. Experimental studies demonstrate that the proposedapproach is highly effective in all phases of a teleoperation task,i.e. in free motion and in contact with both flexible and rigidenvironments.

Index Terms— Cooperative Teleoperation, Multi-master/Multi-slave Teleoperation, Robot Control, NonlinearControl, Adaptive Control.

I. INTRODUCTION

Over the past two decades, applications of teleoperationtechnology have been steadily growing in areas such asspace operation, underwater exploration, mining, toxic andnuclear material handling, the entertainment industry, andmore recently in health care [1]. Telerobotic systems deliverthe human intelligence and manipulation skills combinedwith robot precision, repeatability and power to inaccessi-ble and/or remote environments. This is achieved throughcoordinated control of a master robotic arm, locally usedby the operator, and a slave manipulator which mimics theoperator’s actions in the task environment.

Cooperative robotic manipulation has advantages oversingle-robot manipulation, such as increased dexterity, im-proved handling capability, increased loading capacity, andenhanced robustness due to redundancy. Cooperative tele-operation combines advantages of human collaboration androbotic manipulation. It can be employed in traditional tele-operation applications and in emerging areas such as robot-assisted surgery, e.g. a surgeon may use two robotic arms to

∗This work was supported by Natural Sciences and Engineering ResearchCouncil of Canada, Canada Foundation for Innovation, and Ontario Innova-tion Trust.

telemanipulate a surgical instrument. Despite the extensiveamount of research in teleoperation, e.g. [2]–[7], and coor-dinated control of robots, e.g. [8], [9], multi-master/multi-slave teleoperation has received little attention in the past.This includes the work in [10] which proposes the useof heuristic methods such as predictive graphic displays toaddress the time delay in the communication channels incooperative telerobotic environments. Approaches based onPetri Net models and event-based planning and control havebeen recently developed in [11] for remote operation in multi-robot environments. An Internet-based distributed, multiple-telerobot system that enables operators to use remote robotsin order to perform cooperative tasks has been introducedin [12].

In cooperative teleoperation, multiple slave robots dynam-ically interact, directly or through an intervening tool dynam-ics. Designs based on conventional single-master/single-slavearchitectures fail to address the performance and stability re-quirements of cooperative teleoperation as they often neglectthese interactions. The issues of performance and stability forsuch systems must be addressed under these constraints.

This paper follows upon the first author’s earlier work [13]in which a general architecture for cooperative teleoperationcontrol was introduced. In that work, a µ-synthesis basedrobust linear controller was developed assuming dynamicallylinearized models for the operators, masters, slaves, and envi-ronment. The controller utilizes all possible communicationchannels between the master and slave manipulators. Thispaper proposes adaptive nonlinear control to accommodatenonlinearity and parametric uncertainty in these dynamics.The control laws are inspired by those given in [6] for con-ventional single-master/single-slave teleoperation. The majorcontribution of this paper is extending such control approachto cooperative teleoperation applications where multiple mas-ters and slaves are involved.

The proposed multilateral teleoperation controller estab-lishes position-position kinematic correspondence betweenthe masters and slaves. It also synthesizes an adjustableintervening tool impedance between the operators and theenvironment. The stability of the closed-loop system in free

Proceedings of the2005 IEEE Conference on Control ApplicationsToronto, Canada, August 28-31, 2005

WA3.5

0-7803-9354-6/05/$20.00 ©2005 IEEE 1263

Page 2: [IEEE 2005 IEEE Conference on Control Applications, 2005. CCA 2005. - Toronto, Canada (Aug. 29-31, 2005)] Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005

Fig. 1. Schematic of a cooperative teleoperation system.

motion, and in contact with flexible and rigid environmentsis demonstrated via a Lyapunov analysis. The novelty ofthe proposed cooperative controller is due to: (i) explicitlyaddressing the issues of performance and stability in thepresence of dynamic interaction between the slaves as wellas in the presence of unknown operators, environment, andmaster/slave parameters. This is in contrast with the fewreported work in cooperative teleoperation which either doignore such dynamic interactions or only study the problemof free motion coordination; (ii) its data communicationarchitecture which allows for all possible information routes.

The rest of this paper is organized as follows. Dynam-ics of a cooperative teleoperation system are introduced inSection II. The adaptive nonlinear controllers for master andslave robots are briefly discussed in Section III. The coordi-nating teleoperation control laws are presented in Section IV.The stability of the closed-loop system in the presence ofcommunication delay is analyzed in Section V. Experimentalresults for a single-axis two-master/two-slave design exampleare given in Section VI. The paper will be concluded inSection VII.

II. COOPERATIVE TELEOPERATION

Figure 1 displays a conceptual cooperative teleoperationsystem in which m operators collaborate in performing a taskthrough controlling m slave robots and a tool. The operatorscan move the tool in free motion, interact with a flexibleenvironment, and make contact with a rigid environment.

In [13], the authors have proposed to establish data linksbetween all master and slave units. Within such framework,each master and slave robot can receive position and forceinformation from all master and slave units. The extra com-munication links can be utilized to facilitate coordinationamong operators. For example, the controller can assist theoperators in simultaneous grasping of an object throughimposing virtual constraints on positions. Another advantageof the proposed architecture over traditional approaches isin applications where communication delays are differentthroughout various channels. The local information links canbe utilized to reduce instability due to links with large delays.

Dynamics of master and slave robots are governed bysecond-order nonlinear differential equations, and tool dy-namics as well as operators’ arm dynamics are approximatedby second-order linear time-invariant differential equations.When the operator holds the master handle, i.e. xi

m = xih, the

dynamics of the i’th master and operator can be integratedas follows

M imhΦi + Ci

mhΦi + Gimh = Y i

mhθimh = f i

m + f∗ih (1)

where Φi(xim) is the generalized master’s position trans-

formed to the virtual tool frame and xim ∈ R6 is the

generalized position of a frame attached to the i’th master’shandle w.r.t. its base frame; f i

m is the transformed controlaction and f∗i

h is the operator’s intentional force, which isassumed to be bounded; Y i

mhθimh is the linear-in-parameter

representation of the dynamics with θimh being the vector of

unknown parameters of the operator and possibly the masterarm [14].

It is assumed that the slave robots are rigidly in contactwith the tool and have formed a closed kinematic chain. Thisassumption imposes constraints on the positions of the slavesand the tool, i.e. xt = Ψ1(x1

s) = · · · = Ψm(xms ); xt is

the generalized position of a frame attached to the tool atits potential contact point with the environment, xi

s is thegeneralized position of a frame attached to the contact pointof the i’th slave and tool represented in the slaves’ base frame,and Ψ’s are appropriate nonlinear coordinate transformations.

The tool-environment contacts are assumed to be rigidalong some a priori known directions and flexible along therest. It is assumed that rigid and flexible coordinates aredecoupled in generalized tool position vector xt. Therefore,xt can be written as xt = ∆rx

re +(I − ∆r)xt where xr

e is avector of constant contact positions along the rigid directionsand ∆r is a 6×6 diagonal matrix that its i’th diagonal elementis one when there is rigid contact in the i’th direction andotherwise is zero. Hence, ∆rxt = 0 and ∆rxt = 0, and theenvironment reaction force fe can be written in terms of itsflexible and rigid components as follows

fe = ∆rfe + ∆f (Mext + Bext + Kext) (2)

where fe is a generalized force exerted on the environmentby the tool at its contact point and ∆rfe is a vector of en-vironment reaction forces along the rigid coordinates. Again∆f is a 6 × 6 diagonal matrix that its i’th diagonal elementis one when there is flexible contact in the i’th direction andzero otherwise; Me, Be, Ke ≥ 0 are constant matrices thatmodel the dynamics of the flexible part of the environment.Since the rows and columns associated with rigid coordinatesare zero, these matrices are positive semi-definite.

Dynamics of the slave robots, tool and the environment

1264

Page 3: [IEEE 2005 IEEE Conference on Control Applications, 2005. CCA 2005. - Toronto, Canada (Aug. 29-31, 2005)] Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005

can be combined to obtain

Mstext + Cstext + Gste = Ysteθste (3)

=m∑

i=1

J itTf i

s − ∆rfe (4)

where θste includes unknown parameters of the flexible envi-ronment, tool and possibly slave robots; f i

s is the equivalent

control action in the slaves’ base frame and J it =

(∂Ψi

∂xis

)−1

.

III. ADAPTIVE NONLINEAR CONTROL FOR MASTERS

AND SLAVES

The first step in deriving the cooperative teleoperationcontrollers is to design adaptive nonlinear control laws formaster and slave robots. These controllers are similar tothose in [6] with two exceptions: First, there are multipleslave manipulators and a tool here which form a kinematicchain and therefore their dynamics are coupled. In [6] onlyone slave is considered. Second, in [6] the contact spaceis divided into two rigid and flexible subspaces while themodelling in this paper permits several flexible and rigidcontact coordinates.

A. Control Laws

The control laws for the slave and the i’th master robotsare given by

Fs = J†t us + F int

s , Fs =[f1

s · · · fms

]T

us = Ysteθste + Ksρt + A−1s ∆r

(C−1vd

t + vdt

)

vrt = vd

t − Asfe , ρt = vrt − vt (5)

f im = Y i

mhθimh + Ki

mρim + f i

maxsgn(ρim)

vrim = vdi

m + K−1p Ai

mf ih , ρi

m = vrim − Φi (6)

where Ks, C, As, Kim, Λ, Kp, Am > 0 are diagonal matrices,

vdt is tool velocity command, f i

h and fe are filtered hand andenvironment reaction forces respectively; Fs is the slaves’control action, and F int

s is an internal force component thatdoes not affect the motion of the tool, i.e. JtF

ints = 0.

The internal force can help maintain the contact between theslave manipulators and the tool if the end-effectors are notequipped with grippers.

The slave/environment and the i’th master/operator param-eter adaptation are driven by

˙θste = ΓsY

Tsteρt = Γsyθ,

˙θi

mh = ΓimY T

mhρim = Γi

mziθ (7)

where Γs is a diagonal matrix with elements

γii =

⎧⎪⎨⎪⎩

0 θis ≤ θmin

si, yi

θ ≤ 00 θi

s ≥ θmaxs

i, yiθ ≥ 0

γis otherwise

(8)

γi’s are positive and θmins and θmax

s are known lower andupper bounds on the parameters; Γi

m is also a diagonal matrixwith elements similar to those of Γs.

B. Stability Analysis

The following closed-loop error dynamics are obtainedfor the slaves/tool/evironment and the i’th master/operatorsubsystems by substituting control laws in (5) and (6) into(3) and (1), respectively.[Mste + ∆rC

−1A−1s

]ρt +

[Cste + ∆rA

−1s + Ks

]ρt

−Ysteθste = 0 (9)

M imhρi

m + Cimhρi

m + Kimρi

m + f∗ih + f i

maxsgn(ρim)

−Y imhθi

m = 0 (10)

with θste = θste − θste and θim = θi

m − θim. Lyapunov

functions for the error dynamics in (9) and (10) are

Vs =12ρT

t

[Mste + ∆rC

−1A−1s

]ρt +

12θT

s Γs−1θs (11)

V im =

12ρi

mTM i

mhρim +

12θi

m

TΓi

m−1

θim (12)

It is not difficult to show that

Vs ≤ −ρt

[∆rA

−1s + Ks

]ρt, V

im ≤ −ρi

m

TKi

mρim (13)

The results in (11)-(13) yield

ρim = vdi

m − Φ + K−1p Ai

mf ih ∈ L2

⋂L∞, i = 1, · · · , m

ρt = vdt − vt − Asfe ∈ L2

⋂L∞ (14)

The switching terms in the masters’ control law (6) elim-inate the error due to unknown bounded external forces f∗

hi.

However, high frequency switching activities are undesirable.In practice, this can be resolved by replacing the sgn() witha smooth function. Alternatively, f∗

h may be included in θm

and estimated on-line. Such approach is effective if the rateof change in f∗

h is slow with respect to the time constantof parameter adaption. The latter approach is adopted in thiswork.

IV. TELEOPERATION COORDINATING CONTROLLERS

Local adaptive nonlinear controllers in (5) and (6) guaran-tee the stability of ρi

m and ρt. The operation of masters andslaves are coordinated through the design of desired velocitycommands vdi

m in (6) and vdt in (5) as follows:

vdim =

Λ2m − 2

Σmk=1, =i

[Φk − Φi

]− K−1

p Asfe

+K−1

p

2mΛΣm

k=1

[Ψk − KpΦi

]+ K−1

p Σmk=1, =iA

kmfk

h

+1

2m − 2Σm

k=1, =i˙Φk +

K−1p

2mΣm

k=1˙Ψk (15)

vdt =

Kp

mΣm

k=1˙Φk + KpΛ

[1m

Σmk=1Φ

k − K−1p xt

]

+ Σmk=1A

kmfk

h (16)

1265

Page 4: [IEEE 2005 IEEE Conference on Control Applications, 2005. CCA 2005. - Toronto, Canada (Aug. 29-31, 2005)] Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005

In the above formulation˙Φk + CΦk = CΦk, ˙fk

h + Cfkh = Cfk

h

˙Ψk + CΨk = CΨk,˙fe + Cfe = Cfe (17)

Using (5),(6),(15), and (16), it can be shown that

ρim − ρj

m = Λ(Φj − Φi

)+ Φj − Φi+

12m − 2

[ ˙Φj − ˙Φi + Λ(Φj − Φ

)] (18)

Equation (18) can be written as

ρim − ρj

m =1

2m − 2Ω + Ω (19)

with Ω = Φj − Φi +Λ(Φj − Φi

)and ˙Ω+CΩ = CΩ. Using

ρim − ρj

m ∈ L2

⋂L∞, we have Ω ∈ L2

⋂L∞, which yields

Φj − Φi, Φj − Φi ∈ L2

⋂L∞ (20)

and therefore the position tracking error between the i’th andj’th master devices is stable. Furthermore,

ρt − Kp

mΣm

k=1ρkm =

12m

Σmk=1

[Kp

˙Φk − ˙Ψk]+

[Kp

mΣm

k=1Φk − vt

]+ Λ

[Kp

mΣm

k=1Φk − xt

]

+1

2mΛΣm

k=1

[KpΦk − Ψk

](21)

Again, given that ρt− Kp

m Σmk=1ρ

km ∈ L2

⋂L∞ and following

similar steps as above, it can be shown that

Kp

mΣm

k=1Φk − xt,

Kp

mΣm

k=1Φk − vt ∈ L2

⋂L∞ (22)

Therefore, the stability of tracking errors between scaledmasters’ and slaves’ positions is guaranteed. Next, the forcetracking behavior of the proposed cooperative teleoperationcontroller is studied.

ρt +Kp

mΣm

k=1ρkm = 2Σm

k=1Akmfk

h − 2Asfe+[1m

Σmk=1

˙Ψk − vt

]+

Kp

mΛΣm

k=1

[Φk − Φk

]+

Λ[

1m

Σmk=1Ψ

k − xt

]+

12m

Σmk=1

[Kp

˙Φk − ˙Ψk]+

Kp

mΣm

k=1

[˙Φkvk

m − Φk]

+1

2mΛΣm

k=1

[KpΦk − Ψk

](23)

It should be noted that

Φk − Φk = −C−1 ˙Φk, ˙Φk − Φk = −C−1 ¨Φk

Ψk − xt = −C−1 ˙Ψk, ˙Ψk − vt = −C−1 ¨Ψk (24)

Also from (21),

12mΣm

k=1

[Kp

˙Φk − ˙Ψk]∈ L2

⋂L∞

12mΛΣm

k=1

[KpΦk − Ψk

]∈ L2

⋂L∞ (25)

Using (23)-(25) results in

2Σmk=1Amfk

h − 2Asfe − KpC−1

mΣm

k=1¨Φk − C−1

mΣm

k=1¨Ψk

− KpC−1Λ

mΣm

k=1˙Φk − C−1Λ

mΣm

k=1˙Ψk ∈ L2

⋂L∞ (26)

Note that Asfe = As∆r fe +As∆f fe and from (5) and (16)

As∆r fe = ∆rvdt −∆rρt = ∆rΣm

k=1Amfkh −∆rρt −Λxt

+Kp

m∆rΣm

k=1˙Φk +

KpΛm

∆rΣmk=1Φ

k (27)

Using (2)

As∆f fe = As

(Me

¨xt + Be˙xt + Kext

)(28)

fkh = f∗k

h − Mkh

¨Φk − Bkh

˙Φk − KkhΦk (29)

with ˙f∗k

h + Cf∗kh = Cf∗k

h . yields

Φk, ˙Φk, Φk, Φk, Ψk, ˙Ψk, Ψk, Ψk ∈ L∞ (30)

This completes the proof of stability for the closed-loopsystem. Using a rigorous analysis similar to that in [6], itis possible to show that ρi

m’s and ρt also converge to zero.The convergence of position tracking errors to zero can thenbe concluded.

Now by letting Am = KfAf and assuming that thefrequency of operation is below the bandwidth of the first-order filter C/2π, (28) can be approximated by

Σmk=1f

kh − K−1

f fe ≈ K−1f KpA

−1C−1[Φ + ΛΦ

](31)

This can be interpreted as a virtual intervening tool dy-namics with a mass of K−1

f KpA−1C−1 and damping of

K−1f KpA

−1C−1Λ acting between the scaled environmentforce K−1

f fe and the operators’ net force Σmk=1f

kh . The mass

and stiffness of virtual tool are adjustable by the controlparameters A, C, Kp, Kf , and Λ.

V. THE EFFECT OF COMMUNICATION DELAY

In this section, the stability of the proposed controller inthe presence of constant time delay in the communicationchannels is analyzed. It is assumed that the time delay inthe local communication links between the masters as wellas between the slaves is negligible compared to the one-waytime delay, T , between the master and slave sites. Due to thepresence of the delay, (15) and (16) are modified as follows

vdim = e−sT

K−1p

2m

m∑k=1

˙Ψk +1

2m − 2

m∑k=1, =i

˙Φk+

K−1p

2mΛ

m∑k=1

[e−sT Ψk − KpΦi

]− e−sT K−1

p Asfe+

K−1p

m∑k=1, =i

Akmfk

h +Λ

2m − 2

m∑k=1, =i

[Φk − Φi

](32)

1266

Page 5: [IEEE 2005 IEEE Conference on Control Applications, 2005. CCA 2005. - Toronto, Canada (Aug. 29-31, 2005)] Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005

vdt = e−sT Kp

m

m∑k=1

˙Φk + e−sTm∑

k=1

Akmfk

h

+ KpΛ

[1m

e−sTm∑

k=1

Φk − K−1p xt

](33)

By substituting (32) and (33) into (14) and assuming thatfk

h = f∗kh − Zk

h˙Φk(xk

m), Zkh = Zk, Ak

m = Am, and fe =Zevt, it can be shown that

H1s(s)Ps = H2s(s)e−sT Pm − ρs (34)

H1m(s)Pm = H2m(s)e−sT Ps − ρm (35)

where Ps = xt, Pm =∑

Φk(xkm), and

ρs = ρt − e−sT Am

∑mk=1 fk

h

ρm =∑m

k=1 ρkm − mK−1

p Am

∑mk=1 fk

h

H2s(s) = Kp

m (sI + Λ) − AmZh

H1s(s) = (s + Λ)(C−1s + I

)+ AsZes

H2m(s) = mK−1p

2 [sI + Λ − 2AsZes]H1m(s) = (sI + Λ)

(sC−1 + 1

2I)

+ mK−1p AmZhs

Using (34) and (35),

Ps = H−11s (s)H2s(s)H−1

1m(s)H2m(s)e−2sT Ps − ρ∗sPm = H−1

1m(s)H2m(s)H−11s (s)H2s(s)e−2sT (Pm) − ρ∗m

ρ∗s = H−11s ρs + H−1

1s (s)H2s(s)H−11m(s)e−sT ρm

ρ∗m = H−11mρs + H−1

1m(s)H2m(s)H−11s (s)e−sT ρs

Since ρs and ρm are bounded according to (14), the systemremains stable for any time delay if:

∥∥H−11s (jω)H2s(jω)H−1

1m(jω)H2m(jω)∥∥∞ < 1 (36)∥∥H−1

1m(jω)H2m(jω)H−11s (jω)H2s(jω)

∥∥∞ < 1 (37)

The controller parameters can be obtained through formu-lating a min-max optimization problem characterized by (36)and (37). The maximization and minimization are performedover ω and controller parameters, respectively. Additionalconstraints can be imposed on the mass and damping of thevirtual tool in (31) to achieve a desired performance.

VI. EXPERIMENTAL RESULTS

This section presents the results of experimental studiesconducted for evaluation of the proposed cooperative teleop-eration scheme on a single-axis linear two-master/two-slavesystem. The experimental setup is displayed in Figure 2. Thetwo side carts on the linear track are motorized and are usedas the slaves in the experiments. The middle cart can freelymove on the track and is employed as the tool. Its motionis blocked in one direction by a rigid stop inserted in themiddle of the track under the tool. The position of the slavesare measured by rotary optical encoders. Two force sensorsmeasure the interaction force between the slaves and the tool.

Fig. 2. The cooperative teleoperation experimental setup.

The master devices are two identical planar twin pan-tograph haptic interfaces, each capable of producing threeactive degrees of motion. The motion of each device isrestricted to a single axis via the application of a workspaceproportional-derivative controller. Disturbance observers havebeen designed and implemented to estimate the operators’hand forces rather than using actual force sensors. The controlalgorithm has been implemented using Matlab RealtimeWorkshop Toolbox and Tornado 2.2/VxWorks 5.4. The controlrate is set to 1024Hz.

Two sets of experiments were conducted, one with flexiblecontact and another with rigid contact. In the experiments,two operators cooperatively moved the tool in free motionand made contact with the environment several times. Ex-periments were repeated with different operators and similarbehavior was observed.

A. Flexible Environment

Experimental results for flexible contact are shown inFigure 3. The system demonstrates accurate position trackingboth in free motion and in contact with the environment. Incontact situation, the force exerted on the environment bythe tool, tracks the operators’ net force after the velocityof the tool approaches zero. This is an expected behaviorsince the controller is supposed to render an intervening tooldynamics between the operators and the task environment.Also, the adaptive controller is able to estimate the envi-ronment stiffness effectively as shown in Figure 3(c). Theestimated stiffness is close to the a value obtained from off-line estimation. The small non-zero stiffness estimate in freemotion is most likely due to environment force observationerrors.

B. Rigid Environment

Experimental results for the rigid contact are presentedin Figure 4. The rigid contact behavior is stable and welldamped. The position tracking is accurate both in free motionand in contact phases. The system also demonstrates excellentforce tracking between the operators’ net force and the

1267

Page 6: [IEEE 2005 IEEE Conference on Control Applications, 2005. CCA 2005. - Toronto, Canada (Aug. 29-31, 2005)] Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005

Fig. 3. Experimental results for flexible environment: (a) position tracking(b) force tracking (c) environment’s stiffness estimate.

Fig. 4. Experimental results for rigid environment: (a) position tracking(b) force tracking.

environment force when the tool is pushed against the rigidenvironment with a variable force.

Experiments have also been conducted using two decou-pled four-channel teleoperation systems for performing thesame task. The system demonstrated poor contact stability.The operators were mostly unsuccessful in grasping the toolbecause of contact instability. Even in cases in which theforce gains were reduced to improve stability, the contactbetween the tool and slaves were frequently lost due to poorcoordination between the operators. In summary, the coop-erative controller provides a superior transparent responsein contact with the rigid environment and greatly facilitatesthe coordination between the operators when compared withdecoupled single-slave/single-master controllers.

VII. CONCLUSIONS

This paper studied the problem of coordinated control ofmulti-master/multi-slave teleoperation. The vast majority ofrelevant reported work in the literature are concerned withsingle-master/single-slave teleoperation applications. The fewprevious reports in cooperative teleoperation either ignore thedynamic interaction of the slaves through the tool or limittheir scope to free motion coordination. In this paper, thedynamics of master and slave manipulators, operators, tool,and environment were incorporated into an adaptive nonlinearcontrol approach. The stability of the closed-loop systemin the presence of parametric uncertainty and in contactwith flexible and rigid environments was proven through theLyapunov analysis. The robustness of the controller withrespect to communication delay was also analyzed. Theproposed controller establishes a desired position-positioncorrespondence between masters and slaves and at the sametime provides the operators with a controllable virtual inter-vening tool impedance. Experimental results demonstratedthe effectiveness of the proposed approach for coordinatedcontrol of multi-master/multi-slave teleoperation.

REFERENCES

[1] C. Melchiorri and A. Eusebi, “Telemanipulation: system aspects andcontrol issues ,” in Proc. Model. Cont. Mechan. Robot., pp. 149–183,1996.

[2] R. Anderson and M. Spong, “Bilateral control of teleoperators withtime delay,” IEEE Trans. Automat. Contr., vol. 34, no. 5, pp. 494–501,1989.

[3] D. Lawrence, “Stability and transparency in bilateral teleoperation,”IEEE Trans. Robot. Automat., vol. 9, no. 5, pp. 624–637, 1993.

[4] Y. Yokokohji and T. Yoshikawa, “Bilateral control of master-slave ma-nipulators for ideal kinesthetic coupling-formulation and experiment,”IEEE Trans. Robot. Automat., vol. 10, no. 5, pp. 605–620, 1994.

[5] G. Niemeyer and J.-J. Slotine, “Towards force-reflecting teleoperationover Internet,” in IEEE Int. Conf. Robot. Auto., pp. 1909–1915, 1998.

[6] W.-H. Zhu and S. Salcudean, “Stability guaranteed teleoperation: anadaptive motion/force control approach,” IEEE Trans. Automat. Contr.,vol. 45, no. 11, pp. 1951–1969, 2000.

[7] R. Jee-Hwan, K. Dong-Soo, and B. Hannaford, “Stable teleoperationwith time-domain passivity control,” IEEE Trans. Robot. Automat.,vol. 20, no. 2, pp. 365–373, 2004.

[8] J. Wang, S. Dodds, and W. Bailey, “Coordinated control of multiplerobotic manipulators handling a common object-theory and experi-ments,” IEEE Proc. Contr. Theory Appl., vol. 144, pp. 73–84, January1997.

[9] R. Bonitz and T. Hsia, “Internal force-based impedance control forcooperating manipulators ,” IEEE Trans. Automat. Contr., vol. 12,pp. 78–89, February 1996.

[10] N. Chong, T. Kotoku, K. Ohba, K. Komoriya, K. Tanie, J. Oaki,H. Hashimoto, F. Ozaki, K. Maeda, and N. Matsuhira, “A collaborativemulti-site teleoperation over an ISDN,” Mechatronics, vol. 13, pp. 957–979, 2003.

[11] I. Elhajj, A. Goradia, N. Xi, C. M. Kit, Y.-H. Liu, and T. Fukuda,“Design and analysis of internet-based tele-coordinated multi-robotsystems,” Autonomous Robots, vol. 15, pp. 237–254, 2003.

[12] X.-G. Wang, M. Moallem, and R. Patel, “An internet-based distributedmultiple-telerobot system,” IEEE Trans. Syst., Man, Cybern. A, vol. 33,no. 5, pp. 627–633, 2003.

[13] S. Sirouspour, “Robust control design for cooperative teleoperation,”in Proc. IEEE Int. Conf. Robot. Auto., pp. 1145–1150, 2005.

[14] L. Sciavicco and B. Siciliano, Modeling and Control of Robot Manip-ulators. Springer, 2000.

1268