3D RTI

Team Lead: Ben LauRyan Drury, Rene Gomez, Travis Carlson

MotivationRTI allows users to change the lighting in a scene

Light interacts with the surface features of an objectThe direction of light affects shadows and shades of colorBy varying the direction of light, users can better understand the geometry of the object

ProjectGoal:

Take RTI further by allowing user to better understand geometry of an object by using stereoscopic methods to see depth

ObjectivesCapture stereo RTI datasetsDevelop viewers to visualize stereo RTI data

Fullscreen 3D TV viewerAnaglyph viewerPoint cloud visualization

Overview of 3D TVsMost current 3D TVs rely on “active shutter” glassesTV alternates between showing left and right views while synchronized glasses block one eye at a time to prevent it from seeing the wrong viewCreates a 3D effect if FPS is high enough

Overview of 3D TVsTwo popular encodings of stereoscopic image pairs:

Side-by-sideFrame-sequential

Fullscreen 3D TV ViewerCan view stereo RTI datasets on a 3D TV naturally using side-by-side encodingWe also wrote a program to generate frame-sequential videos

Fullscreen 3D TV ViewerDemo of side-by-side mode

Anaglyph ViewerAnaglyphic 3D is passive, in contrast to the “active” 3D technology just discussedAnaglyphs rely on complementary color filters to pass the correct view to the correct eyeMost widely available method of stereoscopy

Classic red-cyan glasses

True Anaglyph

Formula: + Little Ghosting - Dark Image

- No Color Reproduction

Color Anaglyph

Formula: + Partial Color Reproduction

- Retinal Rivalry

Optimized Anaglyph

Formula:

Gamma Correction is normally applied to brighten up red channel

+ Partial Color Reproduction+ Almost no retinal rivalry

Point Cloud Viewer PipelineDevelopment of point cloud viewer required some preparatory work:

Camera calibrationImage rectificationDisparity map calculation

Camera CalibrationStereo camera is conceptually 2 regular cameras put together

Hence, calibrate both cameras individually and together

Need to calculate both intrinsic and extrinsic parameters

Intrinsic: focal length, skew, etc of each individual camera

Extrinsic: relative positions of 2 cameras, expressed as a pair of translation and rotation matrices

Image RectificationCamera calibration lets us rectify the images more accuratelyImage rectification is the process of projecting 2 images to the same plane

This makes all the epipolar lines parallelSimplifies the task of finding corresponding points in the scene that have been projected to different pixels

Image Rectification1 Picture = 1 Million Words

Image from Wikipedia

Disparity MapGiven rectified images, calculate distance between pixels that were projected from the same point in 3D spaceResult is a disparity map:

A grayscale image whose intensity in each pixel reflects the depth of the point in the scene that was projected to that pixel

Disparity Map for Dogbert

Left Right Disparity Map+ =

Point Cloud ViewerGiven disparity map, can re-project points in image back to 3D spaceResult is a point cloudWe generate a point cloud for each texture in the RTI and merge the results

Hence 9 mini point clouds for an order 3 RTI

Point Cloud ViewerDemo