A Graphical Model For A Graphical Model For Simultaneous Partitioning Simultaneous Partitioning

And LabelingAnd Labeling

Philip Cowans &Philip Cowans &

Martin Szummer Martin Szummer

AISTATS, Jan 2005AISTATS, Jan 2005

Cambridge

Motivation – Interpreting InkMotivation – Interpreting Ink

Hand-drawn diagramHand-drawn diagram

Machine Machine interpretationinterpretation

Graph ConstructionGraph ConstructionVertices are Vertices are grouped into grouped into partsparts..

Each part is Each part is assigned a assigned a

labellabelG

Edges, Edges, EE

Vertices, Vertices, VV

Labeled PartitionsLabeled Partitions

We assume:We assume:Parts are Parts are contiguous.contiguous.The graph is triangulated.The graph is triangulated.

We’re interested in We’re interested in probability distributionsprobability distributions over labeled partitions conditioned on over labeled partitions conditioned on observed data.observed data.

Conditional Random FieldsConditional Random Fields

CRFs (Lafferty CRFs (Lafferty et. al.et. al.) ) provide joint labeling of provide joint labeling of graph vertices.graph vertices.

Idea: define parts to be Idea: define parts to be contiguous regions with contiguous regions with same label.same label.

But…But… Large number of labels Large number of labels

needed.needed. Symmetry problems / Symmetry problems /

bias.bias.

+2+1

+2

-1 -1

+3 +3

A Better Approach…A Better Approach…

Extend the CRF framework to work Extend the CRF framework to work directlydirectly with labeled partitions. with labeled partitions.Complexity is improved – don’t need to deal Complexity is improved – don’t need to deal

with so many labels.with so many labels.No symmetry problem – we’re working directly No symmetry problem – we’re working directly

with the representation in which the problem with the representation in which the problem is posed.is posed.

ProjectionProjection

Projection maps labeled Projection maps labeled partitions onto smaller partitions onto smaller subgraphs.subgraphs.

If G If G µµ VV then, the then, the projection of projection of YY onto G is onto G is the unique labeled the unique labeled partition of G which is partition of G which is ‘consistent’ with ‘consistent’ with YY..

NotationNotation

YY Labeled partition of Labeled partition of GG

YY (A)(A) Labeled partition of the induced subgraph Labeled partition of the induced subgraph of A of A µµ VV

YYAA Projection of Projection of YY onto A onto A µµ VV

YYii Projection of Projection of YY onto vertex i. onto vertex i.

YYijij Projection of Projection of YY onto vertices i and onto vertices i and  j.j.

PotentialsPotentials

The ModelThe Model

- Unary:Unary:

- Pairwise:Pairwise:

The ModelThe Model

TrainingTraining

Train by finding MAP weights on example Train by finding MAP weights on example data with Gaussian prior (BFGS).data with Gaussian prior (BFGS).

We require the value and gradient of the We require the value and gradient of the log posterior:log posterior:

Normalization

Marginalization

PredictionPrediction

New data is processed by finding the most New data is processed by finding the most probable labeled partition.probable labeled partition.

This is the same as normalization with the This is the same as normalization with the summation replaced by a maximization.summation replaced by a maximization.

InferenceInference

These operations require summation or These operations require summation or maximization over all possible labeled maximization over all possible labeled partitions.partitions.

The number of terms grows super-The number of terms grows super-exponentially with the size of exponentially with the size of GG..

Efficient computation possible using Efficient computation possible using message passing as distribution factors.message passing as distribution factors.

Proof based on Shenoy & Shafer (1990).Proof based on Shenoy & Shafer (1990).

Message PassingMessage Passing

44

88

77

99

11

22 33

55

66

Message PassingMessage Passing

1,2,3,41,2,3,4

2,3,4,52,3,4,5

2,92,91,7,81,7,8

4,5,64,5,6

‘‘Upstream’Upstream’Message summarizes Message summarizes

contribution from contribution from ‘upstream’ to the ‘upstream’ to the sum for a given sum for a given configuration of the configuration of the separator.separator.

Junction tree constructed from cliques on original

graph.

Message PassingMessage Passing

PartitionPartition LabelsLabels ValueValue

(2)(3)(4)(2)(3)(4) +,+,-+,+,- 0.0120.012

(2)(3)(4)(2)(3)(4) +,-,-+,-,- 0.0430.043

(2,3)(4)(2,3)(4) +,++,+ 0.1340.134

(2,3,4)(2,3,4) -- 0.2350.235

…… …… ……

1,2,3,41,2,3,4

2,3,4,52,3,4,5

2,92,91,7,81,7,8

4,5,64,5,6

PartitionPartition LabelsLabels ValueValue

(2)(2) ++ 0.430.43

(2)(2) -- 0.720.72

PartitionPartition LabelsLabels ValueValue

(1)(1) ++ 0.230.23

(1)(1) -- 0.570.57

x22x22

Message Update RuleMessage Update Rule

Update messages (for summation) according toUpdate messages (for summation) according to

Marginals found usingMarginals found using

Z can be found explicitlyZ can be found explicitly

ComplexityComplexity

Clique SizeClique Size 22 33 44 55

CRFCRF 1616 216216 40964096 1.0 1.0 ££ 10 1055

Labeled PartitionsLabeled Partitions 66 2222 9494 454454

1

1000

1E+06

1E+09

1E+12

1E+15

1 3 5 7 9 11

Labeled Partitions

CRF

Experimental ResultsExperimental Results

We tested the algorithm on We tested the algorithm on hand drawn hand drawn inkink collected using a Tablet PC. collected using a Tablet PC.

The task is to partition the ink fragments The task is to partition the ink fragments into perceptual objects, and label them as into perceptual objects, and label them as containerscontainers or or connectorsconnectors..

Training data set was 40 diagrams, from Training data set was 40 diagrams, from 17 subjects with a total of 2157 fragments.17 subjects with a total of 2157 fragments.

3 random splits (20 training and 20 test 3 random splits (20 training and 20 test examples).examples).

Example 1Example 1

Example 1Example 1

Example 2Example 2

Example 2Example 2

Example 3Example 3

Example 3Example 3

Labeling ResultsLabeling ResultsModelModel Labeling Labeling

ErrorErrorGrouping Grouping

ErrorError

Independent LabelingIndependent Labeling 8.5%8.5% --

Joint LabelingJoint Labeling 4.5%4.5% --

Labeled PartitionsLabeled Partitions 2.6%2.6% 8.5%8.5%

• Labelling error: fraction of fragments Labelling error: fraction of fragments labeled incorrectly.labeled incorrectly.

• Grouping error: fraction of edges locally Grouping error: fraction of edges locally incorrect.incorrect.

ConclusionsConclusions

We have presented a conditional model definied We have presented a conditional model definied over labeled partitions of an undirected graph.over labeled partitions of an undirected graph.

Efficient exact inference is possible in our model Efficient exact inference is possible in our model using message passing.using message passing.

Labeling and grouping simultaneously can Labeling and grouping simultaneously can improve labeling performance.improve labeling performance.

Our model performs well when applied to the Our model performs well when applied to the task of parsing hand-drawn ink diagrams.task of parsing hand-drawn ink diagrams.

AcknowledgementsAcknowledgements

Thanks to:Thanks to:

Thomas Minka, Yuan Qi and Michel Gagnet Thomas Minka, Yuan Qi and Michel Gagnet for useful discussion and providing for useful discussion and providing

software. software.

Hannah Pepper for collecting our ink Hannah Pepper for collecting our ink database.database.