3D Shape Inference Computer Vision No.2-1. Pinhole Camera Model the camera center Principal axis the...

3D Shape Inference

Computer Vision

No.2-1

Pinhole Camera Model

x

y

z

the camera center

Principal axis

the image plane

Perspective Projection

the optical axis

the image plane

the camera centerz

Focal length

f

u

z

xfu

x

Orthographic Projection

the optical axis

the image plane

the camera center

xu

Weak Perspective Projection

the optical axis

the image plane

the camera center

0z

xfu

the reference plane

0z

Para Perspective Projection

the optical axis

the image plane

the camera center

0

00

0

z

xzz

xx

fu

the reference plane

),,( 000 zyx

Orthographic Projection

the optical axis

the image plane

the camera center

xu

Obtain a 3D Information form Line Drawing

Given – Line drawing(2D)

Find– 3D object that projects to given lines

Find– How do you think it’s a cube, not a

painted pancake?

Line Labeling Significance

– Provides 3D interpretation(within limits)– Illustrates successful(but incomplete)approach– Introduces constraints satisfaction

Pioneers– Roberts(1976)– Guzman(1969)– Huffman&Clows (1971)– Waltz (1972)

Outline

Types of lines types of vertices Junction Dictionary Labeling by constraint

propagation Discussion

Line Types

convex concave

occluding occluding

Labeling a Line Drawing

Easy to label lines for this solid→Now invert this in order to understand shape

Enumerating Possible Line Labeling without Constraints

•9 lines•4 labels each

→4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4= 250,000 possibilities

We want just one reality must reduce surplus possibilities

→Need constraints 　 (by 3D relationship)

Vertex TypesDivide junctions into categories

Need some constraints to reduce junction types

Restrictions

No shadows, no cracks Non-singular views At most three faces meet at vertex

Fewer Vertex Types

Vertex LabelingThree planes divide space into octants

Enumerate all possibilities 　 (Some full, some empty)

Trihedral vertex at intersection of 3 planes

Enumerating Possible Vertex Labeling(1)

0or8octants full--no vertex2,4,6 octants full

singular view7octants full

1FORK5octants full

2L,1ARROW

3octants full– upper behind L

– right above L

– left above L

– straight above ARROW

– straight below FORK

Enumeration(2)

Enumeration(3)

１ octant--Seven viewing octants supply

Huffman&Clows Junction Dictionary

Any other

arrangements cannot

arise

Have reduced

configuration from

144 to 12

Constraints on Labeling

Without constraints-- 250,000possibilities Consider constraints→3 ｘ 3 ｘ 3 ｘ 6 ｘ 6 ｘ 6 ｘ 5 ＝ 29,000possibilities

We can reduce more by coherency/consistency along line.

Labeling by Constraint Propagation

“Waltz filtering”

By coherence rule, line label constrains

neighbors

Propagate constraint through common vertex

Usually begin on boundary

May need to backtrack

Example of Labeling

Ambiguity

Line drawing can have multiple labelings

Necker Reversal(1)

Wire-frame cube– Human perception flips from one to the other– (After Necker 1832,Swiss naturalist)

Necker Reversal(2)

Impossible Objects

No consistent labeling But some do have a consistent labeling

– What’s wrong here?

Limitations of Line Labeling

Only qualitative;only gets topology Something wrong

Summary(1)

Preliminary 3D analysis of shape

1. Identify 3D constraint

2. Determine how constraint affects images

3. Develop algorithm to exploit constraint

--> General method for 3D vision

Tool:constraint propagation/satisfaction

Summary(2)

Problems

1. Significant ambiguity possible

2. Assumes perfect segmentation

3. Can be fooled without quantitative analysis

Gradient Space

Computer Vision

No. 2-2

Gradient Space and Line Labeling

Last time: line labeling by constraint propagation

Use gradient space to represent surface orientation

－ －

＋

＋ ＋

Review of Line Labeling

Problem Given a line drawing, label all the lines with

one of 4 symbols ＋ convex edge － concave edge←→ occluding edges

Approach Narrow down the number of possible labels

with a vertex catalog

＋ ＋＋

－－－ ＋ ＋

Surface Normal

Normal of a plane

Rewrite

0 DCzByAx

0x 1x

Normal vector (A,B,C)

0),,(),,( 010101 zzyyxxCBA

)1,,(),,(C

B

C

ACBA )1,,( qp

Surface Gradient

Gradient of surface is

Gradient of plane

),( yxfz

),(),(),(y

z

x

z

y

f

x

fqp

C

Dy

C

Bx

C

Az

DCzByAx

0

),(),(),(C

B

C

A

y

z

x

zqp

Surface Gradient

C

Dx

C

Az

DCzAx

0

)0,(),(C

Aqp

q

p

p1

p3 p2y

Relationship of Normal to Gradient

(p,q)1

0p

q

xy

x

p1p4

p5 Normal Vector

p1

p3

p2

y q

p

Polyhedron in Gradient Space

GH

F

ED

C

B

I

A＋

＋＋＋

＋

＋ ＋ ＋ ＋

＋＋

＋

＋

＋ ＋

＋－

－

－

－

－

－

－

－x

y

A’

D’C’

B’

I’H’

G’

F’

E’

p

q

Top view of polyhedronA x-y plane∥

Same order as left

Vector on a Surface

Suppose vector on surface with gradient

Under orthography, vector in scene projects to

is surface normal vector, so

),,( zyx ),( qpG

),( yxE

)1,,( qp

zyxqp

zyxqp

),(),(

0),,()1,,(

zEG

Vector on Two Surfaces

Suppose vector on boundary between two surfaces

Surfaces have gradients and

If , then

),,( zyx

),( 111 qpG ),( 222 qpG

2S

1S

E

),( yxE

0)( 21

21

EGG

zEGEG

EGG 21

p

q

G1

G2

Ordering of Points Along Gradient Line Perpendicular to Connect Edge

B1

B3

B2S

T

A B1’

B2’

B3’

A

p

q

If connect edge ST convex, then points on gradient spacemaintain same order (left-right) as A and Bi in image

If ST concave, then order switches

How does this gradient space stuff help us to label lines?

L is a “connect edge” (vector on two surface)Assume orthographyLine in gradient space connecting R1 and R2 must beperpendicular to line L

)1,,( 22 qp)1,,( 11 qp

2R1R

＋

),( 22 qp

),( 11 qp

p

q

LL

Line Labeling using Gradient Space1. Assign arbitrary gradient (0,0) to A

2. Consider B lines 1,2 may be connect edges or may be occluding edges

3. Suppose line 1 a connect edge

4. Suppose line 2 a connect edge, then (line A’B’) (line 2) impossible. So line 2 occluding.

BA

C

1

2

34

5

B’

A’ p

q

A’

B’

p

q

Line Labeling using Gradient Space5. Suppose lines 3 and 4 are connect edges

6. and so forth can get multiple interpretations

B’

A’ p

q

A’

B’p

qC’

C

＋－ － ＋

－＋

BA

C

1

2

34

5

Another Payoff: Detect Inconsistencies

R2

R1L2L1

L1 L2

),( 11 qp

),( 22 qp ),( 22 qp

Summary

Can use gradient space to

– represent surface orientation

– detect inconsistent line labels

– constraint labeled line drawings

– establish line labels without the vertex catalog

References

M.B. Clowes, “On seeing things,” Artificial Intelligence, Vol.2, pp.79-116, 1971

D.A. Huffman, “Impossible objects as nonsense sentences,” Machine Intelligence, Vol.6, pp.295-323, 1971

A.K.Mackworth, “On reading sketch maps,” 5th IJCAI, pp.598-606, 1977