Occluded Bilateral EPI Regularization2nd Workshop on Light Fields for Computer VisionJuly 26, 2017

Overview

OBER - Hendrik Schilling - July 26, 2017 1/15

Overview

standard minimization approach→ data term, smoothness term, solversome twists for crisper and smoother resultssome (over) simplifications

Key Insight

OBER - Hendrik Schilling - July 26, 2017 2/15

3D Reconstruction as Minimization Problem

inverse problemunderconstrainednon-convex

Key Insight

OBER - Hendrik Schilling - July 26, 2017 3/15

Key InsightsThe Problem is inherently non-differentiable due to

occlusions→ enforcing differentiabiliy will result in sub-optimal results

SecondaryIntermediate steps impair occlusion border & fine detail

performance→ formulate error metrics in input domain

Method

OBER - Hendrik Schilling - July 26, 2017 4/15

Data Term

Method

OBER - Hendrik Schilling - July 26, 2017 5/15

Smoothness Term

0.7

0.8

0.8

0.5 0.8 0.5

0.20.30.80.80.2

0.3

0.4

0.3

0.2 0.2

0.5

1.0

0.9

0.2

0.1

1.0

0.7 0.6

0.8

0.5

0.6

0.7

candidate d = candidate d =

modified bilateral filterevaluate both color and disparityhard thresholds → crisp occlusion boundaries

Method

OBER - Hendrik Schilling - July 26, 2017 6/15

Solver

Algorithm 1 Randomized Solver1: for 20 iterations do2: for Every disp map pixel do3: Calc error for current depth4: Calc error for depth candidates5: Keep best result6: end for7: end for

Method

OBER - Hendrik Schilling - July 26, 2017 7/15

Depth Candidates

random changerandom guessrandom (large range) neighbourdirect neighbour

Method

OBER - Hendrik Schilling - July 26, 2017 8/15

Propagationeven

pixel processing order neighbour candidates

odd

itera

tion

Results

OBER - Hendrik Schilling - July 26, 2017 9/15

groundtruth

disparitycenterview

disparity

BadPix(0.07)

OBER-cross+ANP SPO-MO2nd best BadPix(0.07)

mesh of OBER result(as viewed from above)

Results

OBER - Hendrik Schilling - July 26, 2017 10/15

0.00

18.51

3.01

1.76

0.27

0.42

11.17

4.80

10.12

37.0111.78

6.02

3.52

0.53

0.940.85

16.86

22.35

9.59

20.23

12.10

55.52

17.67

9.03

5.27

0.80

1.41

1.27

25.28

33.52

14.39

30.35

18.15

74.03

23.56

12.04

7.03

1.07

1.88

1.69

33.71

44.70

19.19

40.47

24.20

OBEROBER-cross+ANPOFSY_330/DNRPS_RFRM3DESPO-MO

BadPix(0.01)

BadPix(0.03)

BadPix(0.07)

MSE

Q25

Bumpiness

Contin.

SurfacesBumpiness

Planes

Discontinuities

Fine

Fattening

Fine

Thinning

MAE

Contin.

Surfaces

MAE

Planes

Results

OBER - Hendrik Schilling - July 26, 2017 11/15

Results

OBER - Hendrik Schilling - July 26, 2017 12/15

rereference result(OFSY_330/DNR)

OBER-cross+ANP(half-size imgs)

Outlook

OBER - Hendrik Schilling - July 26, 2017 13/15

OutlookA lot of things to improve:scale space, un-occlusions, heuristics → NN, disp map → mesh

Main strength:very flexible, per-pixel adaptable solver

Outlook

OBER - Hendrik Schilling - July 26, 2017 14/15

The End

OBER - Hendrik Schilling - July 26, 2017 15/15

The End