EE 290A: Generalized Principal Component Analysis

Lecture 4: Generalized Principal Component Analysis

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 1

This lecture

GPCA: Problem Definition Segmentation of Multiple Hyperplanes

Reminder: HW 1 due on Feb. 8th.

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 2

Problem Definition

Define a mixture subspace model

Subspace Segmentation Problem:

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 3

Projectivization of Affine Subspaces Every affine subspace can be “lifted” to

a linear subspace by adding the homogeneous coordinates

Homogeneous representation

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 4

Conclusion: Projectivization does not lose information on data model and sample membership

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 5

Subspace Projection

High-dim data may lie in low-dim subspaces

When d << D, estimation is not efficient

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 6

Images of a subject under illumination lie on a 20-dim subspace

Subspace-Preserving Projections Subspaces in high-D space can be

projected onto a lower-D space while the membership of the samples is preserved

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 7

If the span of all subspaces is still a proper subspace of the ambient space

: use PCA If the span is the whole space, yet the

largest dimension is less than (D-1)

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 8

The approach for mixture-subspace segmentation

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 9

Choosing a SP-Projection

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 10

3.2 Introductory Cases

Segmenting points on a line

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 11

Determine the number of groups

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 12

Question: When j=K, is the null space of P always 1-D in this case?

Segmenting lines on a plane

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 13

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 14

Question 1: How to determine the number of lines?

Question 2: When k=K, is the null space of V always rank-1?

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 15

Segmenting point clusters on a line or segmenting lines on a plane is a special case of mixture hyperplanes.

Segmenting multiple hyperplanes

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 16

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 17

Find the vanishing polynomial from embedded data

Determine the number of hyperplanes by the rank of the embedded data matrix V.

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 18

Recover subspaces from vanishing polynomial

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 19

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 20

Sastry & Yang © Spring, 2011

EE 290A, University of California, Berkeley 21