Unfolding an Indoor Origami World - Peopledfouhey/2014/origami/origami_v4.pdf · Constraints for...

Unfolding an Indoor Origami World

David Fouhey, Abhinav Gupta, Martial Hebert

1

2

3

Local Evidence

4 Hoiem et al. 2005, Saxena et al. 2005, Fouhey et al. 2013, etc.

Constraints

5

Constraints for Single Image 3D

6

Local Smoothness

Low Level, Generic

Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al., 2009, etc.

Constraints for Single Image 3D

7

Local Smoothness

Low Level, Generic

Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al., 2009, etc.

Constraints for Single Image 3D

8

Low Level, Generic

High Level, Physical

High Level, Physical

Constraints for Single Image 3D

9

Local Smoothness

Low Level, Generic

Coughlan and Yuille 2000, etc.

High Level, Physical

Constraints for Single Image 3D

10

Local Smoothness

Low Level, Generic

Hedau et al. 2009, Del Pero et al., 2011, Wang et al., 2012, Schwing et al. 2012, etc.

High Level, Physical

Constraints for Single Image 3D

11

Local Smoothness

Low Level, Generic

Lee et al. 2010, Xiao et al. 2012, Zhao et al. 2013, Schwing et al., 2013, etc.

High Level, Physical

Constraints for Single Image 3D

12

Low Level, Generic

Mid-level in the Past

13 Huffman 71, Clowes 71, Kanade 80, 81 Sugihara 86, Malik 87, etc.

Our Mid-Level Constraints

14

This Work

15

Input: Single Image

Output: Discrete Scene Parse

Overview

16

Parameterization Formulation

Experimental Results

Overview

17

Parameterization Formulation

Experimental Results

Parameterization

18

Parameterization

19

vp1

vp2

vp3

VP Estimator from Hedau et al., 2009

Parameterization

20

Two VPs give grid cell

Encoding Surface Normals

21

Encoding Surface Normals

22

Encoding Surface Normals

23

Encoding Surface Normals

24

x1,…, x400 x401,…, x800 x801,…, x1200

Related Parameterizations

25

vp1

vp3

x2

x1

x3

x4

Hedau et al., 2009; Wang et al. 2010, Schwing et al., 2012, 2013

vp2

Overview

26

Parameterization Formulation

Experimental Results

Parameterization

27

Formulation

28

Unaries

29

Unaries

30

Low c Any 3D Evidence

High c

Unaries

31

…

Input 3D Primitive Bank

Local: Data-Driven 3D Primitives

Fouhey, Gupta, Hebert, 2013

Unaries

32 Hedau, Hoiem, Forsyth, 2009

Global: Cuboid Fit + Clutter Mask

Input Predicted Walls Clutter Mask

Binaries

33

Convex/Concave Constraints

34

Convex (+) Concave (-)

Convex/Concave Constraints

35

Detected Concave (-)

Convex/Concave Constraints

36

Detected Concave (-)

Convex/Concave Constraints

37

Detected Concave (-)

Convex/Concave Constraints

38

Detected Concave (-)

Convex/Concave Constraints

39

Detected Concave (-)

Detecting Convex/Concave

40

Ground-Truth Discontinuities similar to Gupta, Arbelaez, Malik, 2013 3DP from Fouhey, Gupta, Hebert, 2013

Input 3D Primitive Bank

…

Use 3DP to Transfer Discontinuities

Smoothness

41

Constraints

42

Solving the Model

43

Overview

44

Parameterization Formulation

Experimental Results

Dataset

NYU Depth v2: 795 Train, 654 Test

45

Qualitative Results

46

Qualitative Results

47

Qualitative Results

48

Qualitative Results

49

Surface Connection Graphs

50

Convex Concave

Baseline

Primary Baseline: 3D Primitives

51 Fouhey, Gupta, Hebert, 2013

Output Input

Quantitative Results

52

Summary Stats (⁰) (Lower Better)

% Good Pixels (Higher Better)

11.25⁰ 22.5⁰ 30⁰ Mean Median

3DP 35.9 52.0 57.8 36.0 20.5 49.4

Hedau et al. 34.2 49.3 54.4 40.0 23.5 54.1

RMSE

Lee et al. 18.6 38.6 49.9 43.3 36.3 54.6

Proposed 37.6 53.3 58.9 35.1 19.2 48.7

Quantitative Results

53

Failure Modes

Mistaken but Confident Evidence

54

Failure Modes

55

Missing High-Level Modeling

Conclusions

56

Single Image

Parameterization

Formulation

Discrete Parse

Thank You

57