Upload
willis-briggs
View
216
Download
1
Tags:
Embed Size (px)
Citation preview
Multi-Output Learningfor Camera Relocalization
Abner Guzmán-Rivera UIUC
Pushmeet Kohli Ben Glocker Jamie Shotton Toby Sharp Andrew Fitzgibbon Shahram Izadi
Microsoft Research
2
Camera Relocalizationfrom RGB-D images
World
Know 3D model
RGB-Depth
Observe single frame
Where is the camera?
6D camera pose H(rotation and translation)
3
Applications Large scale 3D model reconstruction
4
Applications Vehicle, robot, etc. localization
5
Applications Augmented Reality
6
Other Approaches to Localization Sparse key-point matching:
– Detectors: [Rosten et al. PAMI’10], [Holzer et al. ECCV’12]
– Descriptors: [Winder and Brown CVPR’07], [Calonder et al. ECCV’10], [Rublee et al. ICCV’11]
– Matching: [Lepetit and Fua PAMI’06], [Nistér and Stewénius CVPR’06], [Schindler et al. CVPR’07]
– Pose estimation: [Irschara et al. CVPR’09], [Dong et al. ICCV’09], [Yi et al. ECCV’10], [Baatz et al. IJCV’11], [Sattler et al. ICCV’11]
Whole key-frame matching[Klein and Murray ECCV’08], [Gee and Mayol-Cuevas BMVC’12]
Epitomic location recognition[Ni et al. PAMI’09]
7
Relocalization as Inverse Problem Find the pose H* minimizing the error in a
rendering of the model
3D model of sceneRendering error
View “renderer”Input RGB-D frame
8
Inverse Problem
DiscriminativePredictor
9
Inverse Problem
10
Single Predictor Not Powerful Enough Limited expressivity
The mapping is one-to-many
Input frame
11
Approx. Inverse Problem Stage 1
Portfolio ofDiscriminative
PredictorsWant complementary or “diverse” predictions
12
Approx. Inverse Problem Stage 2
13
How to train such portfolioof complementary predictors?
14
Discriminative Predictor[Shotton et al. CVPR’13]
15
Scene Coordinate Regression Forests
[Shotton et al. CVPR’13]
Pixel comparison features(Depth and RGB) (x,y,z) world coordinate
Regression tree:
Regression forest
. . .
16
Scene Coordinate Regression Forests
[Shotton et al. CVPR’13]
Inliers for several hypothesesfrom RANSAC
H1
H2
H3
H4
H5
H6
. . .Forest predicts 3Dworld coordinates
Sample pixels frominput RGB-D frame
17
Learning a portfolio of predictors
to output a set of hypotheses that:Would like to train a set of predictors
1. Are relevant, i.e., approx. local minimizers2. Summarize well the output space
18
Learning a portfolio: previous work Multiple Choice Learning
[Guzman-Rivera et al. NIPS’12, AISTATS’14]
Set min-loss Oracle penalizes portfolio for the errorin the best prediction in the output
– The portfolio is NOT penalized for being diverse– Set min-loss applies to standard datasets– Iterative training of fixed size portfolio
Standard task-loss
19
Learning a portfolio of predictors
Portfolio of predictors CVPR’13 SCoRe Forest
We already have the objective to optimize
and propose to approximate (1) by
20
– The portfolio is NOT penalized for being diverse– Learning procedure is able to tune portfolio to
the reconstruction error to be used at test-time– Next we describe one way to achieve diversity
Multi-Output LossStandard task-loss
21
Training Algorithm
22
Loss to Example Weights
Diversity parameter(“variance” of the weights)
Multi-output loss for example j
Intuition: Want next predictor to emphasize accuracy on examples difficult thus far
23
Rendering Error
24
L1 Rendering ErrorInput frame 1. Raycast depth frame for some hypothesis
2. Evaluate L1 distance between input depth and raycast depth
25
Results
26
7-Scenes Dataset
[Shotton et al. CVPR’13, Glocker et al. ISMAR’13]
27
Metric Proportion Correct (single prediction)
– Correct if translational error ≤ 5cm ANDrotational error ≤ 5o
Competing Approaches CVPR13: Scene Coordinate Regression Forests
[Shotton et al. CVPR’13]
CVPR13 + M-Best– Take M-Best RANSAC hypotheses
28
Office
Input frame
Multiple predictions:
Ground-truth (white),Prediction (magenta):
29
Stairs
Input frame
Multiple predictions:
Ground-truth (white),Prediction (magenta):
30
All Scene Average
1 2 3 4 5 6 7 8 9 100.66
0.68
0.70
0.72
0.74
0.76
0.78
0.80
CVPR13 + M-BestMulti-OutputCVPR13
Pro
port
ion
Cor
rect
Size of Portfolio
31
All Scene Average
1 2 3 4 5 6 7 8 9 100.66
0.68
0.70
0.72
0.74
0.76
0.78
0.80
CVPR13 + M-BestMulti-OutputCVPR13
Pro
port
ion
Cor
rect
Size of Portfolio
Usingaggregation
32
Summary Camera relocalization as inverse problem
Portfolio of complementarydiscriminative predictors
Method to learn suchportfolio
State-of-the-art camerarelocalization