Robot Vision SS 2013 Matthias Rüther 1

ROBOT VISION Lesson 1a: Structured Light 3D Reconstruction

Matthias Rüther, Christian Reinbacher

Robot Vision SS 2013 Matthias Rüther 2

Structured Light Methods

Goal: Robust 3D Reconstruction through triangulation

Project artificial pattern on the object

Pattern alleviates the correspondence problem

Variants:– Laser Pattern (point, line)

– Structured projector pattern (several lines, pattern sequence)

– Random projector pattern

Robot Vision SS 2013 Matthias Rüther 3

Structured Light Range Finder

1. Sender (projects plane)2. Receiver (CCD Camera)

X- directionGeometry Z- direction Sensor image

Robot Vision SS 2013 Matthias Rüther 4

1 plane -> 1 object profile

Object motion by conveyor band:=> synchronization: measure distance along conveyor=> y-accuracy determined by distance measurement

Scanning Units (e.g.: rotating mirror) are rare (accurate measurement of mirror motion is hard, small inaccuracy there -> large inaccuracy in geometry

Move the sensor: e.g. railways: sensor in wagon coupled to speed measurement

To get a 3D profile:• Move the object• Scanning Unit for projected plane• Move the Sensor

Robot Vision SS 2013 Matthias Rüther 5

Robot Vision SS 2013 Matthias Rüther 6

Commercially Available

Person Scanners

Cultural Heritage

Rapid Prototyping

Robot Vision SS 2013 Matthias Rüther 7

Problems of Laser Profile

Occlusions:

Object points need to be seen from Laser and Camera viewpoint

Sharpness and Contrast:

Both camera and laser need to be in focus

Speckle noise:

Laser always shows “speckle noise”, caused by interference of coherent light.

-> where is the center of the stripe?

Robot Vision SS 2013 Matthias Rüther 8

Multiple Sheets of Light

Project multiple Laser planes simultaneously to reduce measurement time.

Problem:Separation of stripes in the image

Application:Smoothness check of flat surfaces

Robot Vision SS 2013 Matthias Rüther 9

Pattern projection

CameraCamera: IMAG

CCD,

Res:750x590, f:16

mm

ProjectorProjector: Liquid Crystal Display (LCD 640), f: 200mm, Distance to object plane: 120cm

Projected light stripes

Range Image

Robot Vision SS 2013 Matthias Rüther 10

Projector

Lamp

Lens system

LCD - Shutter

Pattern structure

Example

Focusing lens (e.g.: 150mm)

Line projector (e.g.: LCD-640)

Robot Vision SS 2013 Matthias Rüther 11

Depth decoding

Project Temporal sequence of n binary masks. At each pixel, the temporal sequence of intensities (I1, …, In) gives a binary number which denoted the corresponding projector column.

Project Acquire Decode Triangulate

Robot Vision SS 2013 Matthias Rüther 12

Coded Light + Phase Shift

Binary code is limited to pixel accuracy (or less).

Increase accuracy to sub-pixel by projecting sine wave after code and measuring phase shift between projected and captured pattern. Decode phase from four samples of sine period, shifted by pi/2.

Robot Vision SS 2013 Matthias Rüther 13

Coded Light + Phase Shift

Increase accuracy to sub-pixel by projecting sine wave after code and measuring phase shift between projected and captured pattern. Decode phase from four samples of sine period, shifted by pi/2.

Image column (x)

code

Image column (x)

phase +

0

2

Robot Vision SS 2013 Matthias Rüther 14

Other Coding Methods Possible

Joaquim Salvi,

Pattern codification strategies in structured light systems

Robot Vision SS 2013 Matthias Rüther 15

The Kinect Working Principle

Triangulation based depth sensor

Static pattern projection

Heavy exploitation of redundancy

Extremely robust/conservative depth maps

Robot Vision SS 2013 Matthias Rüther 16

The Sensor System

IR Camera: CMOS, rolling shutter, 1.3MP, ½“, 10bit

RGB Camera: CMOS, rolling shutter, 1.3MP, 1/4“, 10bit

Accelerometer

IR Bandpass

IR Lens: F~6mm FOV~55°

RGB Lens: F~2.9mm, FOV~65°

Laser 830nm, 60mW class 3B without optics, 1 with optics, no amplitude modulation

Diffractive Optical Element (DOE)

Peltier ElementTemperature Stabilization

Microphone Array Tilt AxisStereo Processor

Robot Vision SS 2013 Matthias Rüther 17

The Sensor System

Tx ~75mm

DOF 0.5m – 8m

FOV ~55°

Res. 640x480 (at most)

Internal max 1280x1024

Robot Vision SS 2013 Matthias Rüther 18

The Projection Pattern

IR Laser and Diffractive Optical Element create interference pattern

Pattern is static and identical for all Kinects