ROBOT VISION Lesson 9: Robots & Vision Matthias Rüther

Robot Vision SS 2007 Matthias Rüther 1

ROBOT VISION Lesson 9: Robots & Vision

Matthias Rüther

Robot Vision SS 2007 Matthias Rüther 2

Contents

Visual Servoing– Principle

– Servoing Types

Robot Vision SS 2007 Matthias Rüther 3

Visual Servoing

Vision System operates in a closed control loop.

Better Accuracy than „Look and Move“ systems

Figures from S.Hutchinson: A Tutorial on Visual Servo Control

Robot Vision SS 2007 Matthias Rüther 4

Visual Servoing

Example: Maintaining relative Object Position

Figures from P. Wunsch and G. Hirzinger. Real-Time Visual Tracking of 3-D Objects with Dynamic Handling of Occlusion

Robot Vision SS 2007 Matthias Rüther 5

Visual Servoing

Camera Configurations:

End-Effector Mounted Fixed

Figures from S.Hutchinson: A Tutorial on Visual Servo Control

Robot Vision SS 2007 Matthias Rüther 6

Visual Servoing

Servoing Architectures

Figures from S.Hutchinson: A Tutorial on Visual Servo Control

Robot Vision SS 2007 Matthias Rüther 7

Visual Servoing

Position-based and Image Based control

– Position based: • Alignment in target coordinate system• The 3D structure of the target is rconstructed• The end-effector is tracked• Sensitive to calibration errors• Sensitive to reconstruction errors

– Image based:• Alignment in image coordinates• No explicit reconstruction necessary• Insensitive to calibration errors• Only special problems solvable• Depends on initial pose• Depends on selected features

target

End-effector

Image of target

Image of end effector

Robot Vision SS 2007 Matthias Rüther 8

Visual Servoing

EOL and ECL control

– EOL: endpoint open-loop; only the target is observed by the camera

– ECL: endpoint closed-loop; target as well as end-effector are observed by the camera

EOL ECL

Robot Vision SS 2007 Matthias Rüther 9

Visual Servoing

Position Based Algorithm:1. Estimation of relative pose

2. Computation of error between current pose and target pose

3. Movement of robot

Example: point alignment

p1

p2

Robot Vision SS 2007 Matthias Rüther 10

Visual Servoing

Position based point alignment

Goal: bring e to 0 by moving p1

e = |p2m – p1m|

u = k*(p2m – p1m)

pxm is subject to the following measurement errors: sensor position, sensor calibration, sensor measurement error

pxm is independent of the following errors: end effector position, target position

p1m p2m

d

Robot Vision SS 2007 Matthias Rüther 11

Visual Servoing Image based point alignment

Goal: bring e to 0 by moving p1

e = |u1m – v1m| + |u2m – v2m|

uxm, vxm is subject only to sensor measurement error

uxm, vxm is independent of the following measurement errors: sensor position, end effector position, sensor calibration, target position

p1 p2

c1 c2

u1

u2

v1 v2

d1d2

Robot Vision SS 2007 Matthias Rüther 12

Visual Servoing

Example Laparoscopy

Figures from A.Krupa: Autonomous 3-D Positioning of Surgical Instruments in Robotized Laparoscopic Surgery Using Visual Servoing

Robot Vision SS 2007 Matthias Rüther 13

Visual Servoing

Example Laparoscopy

Figures from A.Krupa: Autonomous 3-D Positioning of Surgical Instruments in Robotized Laparoscopic Surgery Using Visual Servoing