Robot Vision SS 2007 Matthias Rüther 1 710.088 ROBOT VISION 2VO 1KU Matthias Rüther

Robot Vision SS 2007 Matthias Rüther 1

710.088 ROBOT VISION 2VO 1KU

Matthias Rüther


Administrative Things

VO: Tuesday 14:30-16:00 HS i11

Strongly coupled with KU!!

www.icg.tu-graz.ac.at/courses

Exam: Written Exam Oral Exam if Requested

KU: Groups of three students Each group does the same project Effort: ~1week per student


Time Table


Literature

• Sciavicco, L., Siciliano, B., Modelling and Control of Robot Manipulators 2nd Ed., Springer, 2000

• Sonka M., Hlavac V., Boyle Image Processing, Analysis and Machine Vision, Chapman Hall, 1998

• Hartley R., Zissermann A., Multiple View Geometry in Computer Vision, Cambridge, 2001.


Student Project

Solve a Computer Vision Problem– From Hardware selection over 3D Measurement to Live Test


Goal

Measure 3D Geometry of Electrical Discharges

Impact Area

C1

C2 C3

C4


Tasks

Workpackage 1: select hardware, acquire images, segment flash

(xi, yi)


Tasks

Workpackage 2: camera calibration and pose estimation

Impact Area

C1

C2 C3

C4

RW, TW

R21, T21

R31, T31

R41, T41

K1

K2K3

K4


Tasks

Workpackage 3: correspondence & triangulation

(xi, yi) (xj, yj)

3D


Organization

The Project is divided in three workpackages which have to be delivered during the term:– 30.3.2007

– 1.6. 2007

– 22.6. 2007

Each group (3 students) does all three workpackages.

The workpackages build on top of the previous ones. After submission, the workpackages are published.

Each group is allowed to use previous workpackages of any other group.


Example

WP1:

WP2:

WP3:

Group 1 Group 2 … Group n

NO Collaboration during workpackage



YES

Each group may reuse previous workpackages of other groups


Rules

No collaboration between groups during a workpackage. Copying groups are removed from the KU.

Every group member is held responsible for every task in every workpackage.

Code reuse has no influence on the grade.

Each group must deliver at least two workpackages.

A “Sehr Gut” on the Project gives a 25% Bonus on the Lecture exam on 3.6.2007.


Robotics

What is a robot?"A reprogrammable, multifunctional manipulator designed to move

material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks"

Robot Institute of America, 1979

… in a three-dimensional environment.

Industrial– Mostly automatic manipulation of rigid parts with well-known shape in a

specially prepared environment.

Medical– Mostly semi-automatic manipulation of deformable objects in a

naturally created, space limited environment.

Field Robotics– Autonomous control and navigation of a mobile vehicle in an arbitrary

environment.


Experimental/Industrial/Commercial Robots


Industrial Robots


Challenging Environments


Service and Assistance


FRIEND Project


Robot vs Human

Robot Advantages:

– Strength

– Accuracy

– Speed

– Does not tire

– Does repetitive tasks

– Can Measure

Human advantages:

– Intelligence

– Flexibility

– Adaptability

– Skill

– Can Learn

– Can Estimate


Robotics: Goals and Applications

Goal: combine robot and human abilities.

Applications: – Automation (Production)

– Inspection (Quality control)

– Remote Sensing (Mapping)

– Man-Machine interaction („Cobot“)

– Robot Companion (Physically challenged people)

– See [Brady, M. et. al. (eds). „Robot Motion: Planning and Control“]


Statistics

Yearly installations of industrial robots, 2003-2004 and forecast for 2005-2008


Statistics

Estimated operational stock of industrial robots 2003-2004 and forecast for 2005-2008


Statistics

Number of robots per 10,000 production workers in the motor vehicle industry 2002 and 2004


Statistics

Service robots for professional use. Stock at the end of 2004 and projected installations in 2005-2008


Statistics

Service robots for personal and domestic use. Stock and value of stock at the end of 2004 and projected installations in 2005 -2008


What can Computer Vision do for Robotics?

Accurate Robot-Object Positioning

Keeping Relative Position under Movement

Visualization / Teaching / Telerobotics

Performing measurements

Object Recognition (see LV „Bildverarbeitung u. Mustererkennung“, „Bildverstehen“, „AK Computer Vision“)

Registration

Visual Servoing


Computer Vision

What is Computer Vision?

"Computer Vision describes the automatic deduction of the structure and the properties of a (possible dynamic) three-dimensional world from either a single or multiple two-dimensional images of the world" [Nalva VS, "A Guided Tour of Computer Vision"]

Measurement– Measure shape and material properties in a 3D environment. Accuracy

is important.

Recognition– Cognitive systems interpret a 3D environment (object classification,

categorization). Systems are allowed to fail to a certain extent (similar to humans).

Navigation– Navigation Systems orient themselves in a 3D environment.

Robustness and time are important.


Shape from Stereo


Shape from Stereo


Shape from Focus


Shape from Structured Light

Structured Light Sensor

Figures from PRIP, TU Vienna


Shape from Shading


Navigation

SLAM: Simultaneous Localization and Mapping. – Where am I on my map?

– If the place is unknown, build a new map, try to merge it with the original map.

Visual Odometry: calculate the relative motion of the camera between two frames. Summing up the motion gives the camera path. Error propagation!

Visual Servoing: move to / maintain a relative position between robot end effector and an object.

Tracking: continuously measure the position of an object within the sensor coordinate frame.


SLAM

Mapping:


SLAM

The final map:


SLAM

Navigation:


Visual Odometry


Visual Servoing


Tracking


Registration

Registration of CAD models to scene features:

Figures from P.Wunsch: Registration of CAD-Models to Images by Iterative Inverse Perspective Matching

Documents

Robot Vision SS 2007 Matthias Rüther 1 710.088 ROBOT VISION 2VO 1KU Matthias Rüther