Click here to load reader

Robot Vision SS 2009 Matthias Rüther 1 ROBOT VISION Lesson 7: State of the Art in 3D Reconstruction Matthias Rüther

  • View
    214

  • Download
    0

Embed Size (px)

Text of Robot Vision SS 2009 Matthias Rüther 1 ROBOT VISION Lesson 7: State of the Art in 3D...

  • Slide 1

Robot Vision SS 2009 Matthias Rther 1 ROBOT VISION Lesson 7: State of the Art in 3D Reconstruction Matthias Rther Slide 2 Robot Vision SS 2009 Matthias Rther 2 Overview Shape from X techniques Structured light Plane Sweep Stereo Global optimization methods Shape from (de)focus Specularities Shading, Photometric Stereo Alternative methods Slide 3 Robot Vision SS 2009 Matthias Rther 3 Structured Light Methods In principle same as multi-view/stereo Project artificial pattern on the object Pattern alleviates the correspondence problem Variants: Laser Pattern (point, line) Structured projector pattern (several lines, pattern sequence) Random projector pattern Slide 4 Robot Vision SS 2009 Matthias Rther 4 Structured Light Range Finder 1. Sender (projects plane) 2. Receiver (CCD Camera) X- directionGeometry Z- direction Sensor image Slide 5 Robot Vision SS 2009 Matthias Rther 5 1 plane -> 1 object profile Object motion by conveyor band: => synchronization: measure distance along conveyor => y-accuracy determined by distance measurement Scanning Units (e.g.: rotating mirror) are rare (accurate measurement of mirror motion is hard, small inaccuracy there -> large inaccuracy in geometry Move the sensor: e.g. railways: sensor in wagon coupled to speed measurement To get a 3D profile: Move the object Scanning Unit for projected plane Move the Sensor Slide 6 Robot Vision SS 2009 Matthias Rther 6 Slide 7 7 Commercially Available Slide 8 Robot Vision SS 2009 Matthias Rther 8 CAESAR TM Civilian American and European Surface Anthropometry Resource ProjectCAESAR TM 2400 Male/Female Americans 2000 Male/Female Europeans Slide 9 Robot Vision SS 2009 Matthias Rther 9 Problems Occlusions Sharpness and Contrast Speckle noise Slide 10 Robot Vision SS 2009 Matthias Rther 10 Gleichzeitige Projektion mehrerer Lichtschnitte Anstatt einer Lichtebene werden mehrere Lichtebenen auf das Objekt projeziert, um die Anzahl der aufzunehmenden Bilder zu reduzieren. Entfernungsberechnung: wie mit einer Lichtebene, jedoch mu jeder Lichtstreifen im Bild eindeutig identifizierbar sein. Problem: Aufgrund von Verdeckungen sind einzelne Streifen teilweise oder gar nicht im Kamerabild sichtbar -> keine eindeutige Identifikation der Lichtstreifen Anwendung: Glattheitsberprfung bei planaren Oberflchen ohne Tiefenwertberechnung. Slide 11 Robot Vision SS 2009 Matthias Rther 11 Pattern projection Camera Camera: IMAG CCD, Res:750x590, f:16 mm Projector Projector: Liquid Crystal Display (LCD 640), f: 200mm, Distance to object plane: 120cm Projected light stripes Range Image Slide 12 Robot Vision SS 2009 Matthias Rther 12 Projector Lamp Lens system LCD - Shutter Pattern structure Example Focusing lens (e.g.: 150mm) Line projector (z.b: LCD-640) Slide 13 Robot Vision SS 2009 Matthias Rther 13 Tiefenberechnung fr Streifenprojektor 1) Unterschiedlich breite Lichtstreifen werden zeitlich aufeinanderfolgend in die Szene projiziert und von der Kamera aufgenommen. 2) Fr jede Aufnahme wird fr jeden Bildpunkt festgestellt, ob dieser beleuchtet wird oder nicht. 3) Diese Information wird fr jeden Bildpunkt und fr jede Aufnahme im sog. Bit-Plane Stack abgespeichert. Verschiedene Lichtstreifen sind notwendig, um fr jeden Bildpunkt einen zugehrigen Lichtstreifen festzustellen zu knnen. Durch die zeitliche Abfolge der Aufnahmen wird es ermglicht, da jeder Lichtstreifen im Kamerabild identifiziert wird. 4) Findet man im Bit-Plane Stack fr einen Bildpunkt die Information, da er bei den Aufnahmen z.B. hell, dunkel, dunkel, hell war (Code 1001), dann folgt daraus, da dieser Bildpunkt vom vierten Lichtstreifen beleuchtet wird. 5) eindeutige Zuordnung Lichtstreifen Bildpunkt mglich Slide 14 Robot Vision SS 2009 Matthias Rther 14 Coded Light + Phase Shift Binary code is limited to pixel accuracy (at most). Increase accuracy by projecting sine wave and measuring phase shift between projected and captured pattern. Slide 15 Robot Vision SS 2009 Matthias Rther 15 Joaquim Salvi, Pattern codification strategies in structured light systems Slide 16 Robot Vision SS 2009 Matthias Rther 16 Random Texture Projection Slide 17 Robot Vision SS 2009 Matthias Rther 17 Moir Range Finder Project line structure, observe line structure through a grid 1. Sender (Projektor mit Linien) 2. Receiver (CCD Camera with line filter) Problem: identification of line ordering possible but hard, unsharp lines => inaccurate results Moir ImageMoir Pattern Slide 18 Robot Vision SS 2009 Matthias Rther 18 Moir Range Finder Y. Suenaga, 3D Measurements for Computer Animation Slide 19 Robot Vision SS 2009 Matthias Rther 19 Plane Sweep Stereo Sweep family of planes through volume each plane defines an image composite homography input image projective re-sampling of (X,Y,Z) Richard Szeliski, IBMR 1998 Slide 20 Robot Vision SS 2009 Matthias Rther 20 Plane Sweep Stereo For each depth plane compute composite (mosaic) image mean compute error image variance convert to confidence and aggregate spatially Select winning depth at each pixel Richard Szeliski, IBMR 1998 Slide 21 Robot Vision SS 2009 Matthias Rther 21 Voxel Coloring Slide 22 Robot Vision SS 2009 Matthias Rther 22 Voxel Coloring / Space Carving S={} /* initial set of colored voxels is empty for i = 1 to r do /* traverse each of r layers foreach V in the ith layer of voxels do project V into all images where V is visible if sufficient correlation of the pixel colors then add V to S Photorealistic Scene Reconstruction by Voxel Coloring Photorealistic Scene Reconstruction by Voxel Coloring S. M. Seitz and C. R. Dyer, Proc. Computer Vision and Pattern Recognition Conf., 1997, 1067-1073 Slide 23 Robot Vision SS 2009 Matthias Rther 23 Shape From Focus Recover shape of surfaces from limitied depth of view. Requires visibly rough surfaces Typical application: optical microscopy Slide 24 Robot Vision SS 2009 Matthias Rther 24 Shape From Focus Visibly Rough Surfaces Optical roughness: the smallest spatial variations are much larger than the wavelength of incident electromagnetic wave. Visible roughness: smallest spatial variations are comparable to viewing area of discrete elements (pixels). Magnification: Multi-facet level: w 1 >> w f, smooth texture Facet level: w 2 ~= w f, rough texture Slide 25 Robot Vision SS 2009 Matthias Rther 25 Shape From Focus Focused / Defocused images Focused: Defocused: object point is mapped to spot with radius => defocusing is equivalent to convolution with low pass kernel (pillbox function) Slide 26 Robot Vision SS 2009 Matthias Rther 26 Shape From Focus Changing Focus Displacing the sensor: changes sharp region, magnification and brightness Moving the lens: changes sharp region, magnification and brightness Moving the object: changes sharp region only => Object is moved in front of static camera Slide 27 Robot Vision SS 2009 Matthias Rther 27 Shape From Focus Overview: At facet level magnification, rough surfaces give texture-rich images A defocused image is equivalent to a low-passed image As S moves towards focused plane, its focus increases. When S is best focused, Challenges: How to measure focus? How to find best focus from finite number of measurements? Slide 28 Robot Vision SS 2009 Matthias Rther 28 Shape From Focus Focus measure operator Purpose: respond to high frequency variations in image intensity within a small image area produce maximum response when image area is perfectly focused Possible solution: determine high frequency content using Fourier transform (slow) Alternative: Laplacian operator (problem with elimination) Modified Laplacian Slide 29 Robot Vision SS 2009 Matthias Rther 29 Shape From Focus Sum Modified Laplacian Tenengrad Focus Measure Alternatives: variance of intensities, variance of gradients I NxM local intensity function (image window) Slide 30 Robot Vision SS 2009 Matthias Rther 30 Shape From Focus Example Infinite DOFDEM Slide 31 Robot Vision SS 2009 Matthias Rther 31 Shape From Focus Sampling the focus measure function Consider a single image point (x,y) Focus measure F is function of depth d: F(d) Goal find F peak from finite number of samples F 1 F 8 Slide 32 Robot Vision SS 2009 Matthias Rther 32 Shape from focus Sampling the focus measure function Possibility1: find highest discrete sample Slide 33 Robot Vision SS 2009 Matthias Rther 33 Shape from focus Sampling the focus measure function Possibility2: Gaussian interpolation Fit Gauss function to three strongest samples Slide 34 Robot Vision SS 2009 Matthias Rther 34 Shape from Specularity Suitable for highly reflective Surfaces Specular Reflection map of a single point source forms a sharp peak (Specular model, Phong model) Slide 35 Robot Vision SS 2009 Matthias Rther 35 Shape from Specularity Principle: If a reflection is seen by the camera and the position of the point source is known, the surface normal can be determined. => use several point sources with known position: structured highlight inspection Slide 36 Robot Vision SS 2009 Matthias Rther 36 Shape From Shadow Also: Shape from Darkness Reconstruct Surface Topography from self-occlusion E.g. Building reconstruction in SAR images, terrain reconstruction in remote sensing Slide 37 Robot Vision SS 2009 Matthias Rther 37 Shape From Shadow A static camera C observes a scene. Light source L travels over the scene x, position of L is given by angle. L and C are an infinite distance away (orthographic projection). Shadowgram: binary function f(x, ), stating whether scene point x was shadowed at light position. Slide 38 Robot Vision SS 2009 Matthias Rther 38 Photometric Stereo Multiple images, static camera, different illumination directions At least three images Known illumination direction Known reflection model (Lambert) Object may be textured Slide 39 Robot Vision SS 2009 Matthias Rther 39 REFLECTANCE MODELS albedo Diffuse albedo Specular albedo PHONG MODEL L = E (a COS b COS ) n a=0.3, b=0.7, n=2 a=0.7

Search related