Visual Servo Control TutorialPart 1: Basic Approaches

Chayatat RatanasawanyaDecember 2, 2009

Ref: Article by Francois Chaumette & Seth Hutchinson

Overview

•Introduction•Basic components of visual servoing•Image-based visual servo (IBVS)•Position-based visual servo (PBVS)•Stability analysis•Conclusion•Questions/comments

Introduction•Visual servo (VS) control – the use of computer

vision data to control the motion of a robot.

•Relies on techniques from image processing, computer vision, and control theory

•Two camera configurations:▫Eye-in-hand: camera is mounted on a robot

manipulator or on a mobile robot.▫Camera is fixed in the workspace

Basic components of VS

•Error function▫The goal is to minimize the error

•Design of s: ▫Consists of a set of features that are readily

available in the image data (IBVS), or▫Consists of a set of 3D parameters, which must

be estimated from image measurements (PBVS)

*)),(()( samse tt

Basic components of VS (Cont’d)•Interaction matrix (feature Jacobian)•Design of the controller

▫Can be done quite simply once s is selected▫The most straightforward approach is to design

a velocity controller

*)),(()( samse ttcsvLs

cevLe ee

eLv ec

eLv 1 ec eLv ec

ˆIn practice, it is

impossible to know perfectly Le or Le+

Image-based visual servo (IBVS)• The classical IBVS schemes use the image-plane

coordinates of a set of points to define s.• m - the pixel coordinates of a set of image points.• a - the camera intrinsic parameters.• For a 3D point X=(x,y,z) in the camera frame,

using the projection model the point is at x=(u,v) in the image, the interaction matrix is

uuvv

z

v

z

vuuvz

u

zx

2

2

11

0

101

L

cxvLx

Image-based visual servo (IBVS)• To control the 6DOF, at least three points are

necessary.• If the feature vector is chosen as x=(x1,x2,x3), the

Jacobian matrix can be

• However, more than 3 points are usually considered because there will exist cases for which Lx is singular. Moreover, it is not possible to differentiate the global minima (poses for which e=0) when they exist.

3

2

1

x

x

x

x

L

L

L

L

IBVS: Estimating the interaction matrix1. If L e is known; i.e., if

the current z of each point is available

2. L e is unknown, but the desired z is available

3. Same condition as in 1.

eLv ecˆ

Tee

Teee LLLLL

1ˆ

*ˆ

ee LL

)(2

1ˆ*eee LLL

Example of IBVS positioning

IBVS result: case 1

IBVS result: case 2

IBVS result: case 3

IBVS with a Stereovision system• A straightforward extension of the IBVS

approach.

• If a 3D point is visible in both left and right images, it is possible to use as visual features.

• The 3D coordinates of any point observed in both images can be estimated easily by a triangulation process, it is therefore possible and quite natural to use these 3D coordinates in the features set s.

Position-based visual servo (PBVS)• PBVS schemes use the pose of the camera w.r.t.

some reference coordinate frame to define s.

• Computing that pose from a set of measurements in an image requires the camera intrinsic parameters and the 3D model of the object observed.

• m - the pixel coordinates of a set of image points.

• a - the camera intrinsic parameters and the 3D model of the object.

PBVS: definition of ss=(t, θu)

1. If t is defined relative to the object frame, we have

Following the developments presented: determining Le and the estimate of its inverse, the control law is

utte

ts

uts

,

0,

,

*

**

oc

oc

oc

oc

uω

uttt

c

oc

oc

oc

cv*

PBVS result: case 1

PBVS: definition of ss=(t, θu)

2. If t is defined w.r.t. the current camera frame, we have

the corresponding control law is

se

s

uts

0

,*

*

cc

uω

tR

c

ccT

cv*

PBVS result: case 2

Stability analysis - IBVS

•Local asymptotic stability (vc=0 and e≠e*)

can be ensured when number of visual feature in the vector s is greater than 6.

•Global asymptotic stability cannot be guaranteed.

Stability analysis - PBVS• Global stability is achievable when all pose

parameters are perfect.• Robustness: small points position computation

errors in the image can lead to pose errors that may impact the accuracy and the stability of the system significantly.

Conclusion• IBVS or PBVS is better? – performance tradeoffs

• Stability: no strategy provides perfect properties

• Correct estimation of 3D parameters is important for IBVS, but crucial for PBVS.

• In PBVS, the vision sensor is considered as a 3D sensor, which leads to errors.

• In IBVS, the vision sensor is considered as a 2D sensor; therefore, it is robust to errors in calibration and image noise.

Questions/comments