Graph Coloring: Comparing Cluster Graphs to Factor Graphs

Simon Streicher and Johan du PreezStellenbosch University

Graph Coloring: Comparing

Cluster Graphs to Factor Graphs

A B

C D EF

G

A B

C D EF

G

A,D E,DA,E,D

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G

A,E,DA,B,EA,C,D,F B,E,G E,D,G

A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

A B

C D EF

G

Graph Coloring

Practical example is a four coloring map problem:

You only need four colors to color-in a map with noneighboring countries having the same color.

Noted by Francis Guthrie in 1852Theorem proven by Appel and Haken in 1976

A B

C D EF

G

Graph Coloring




A B

C D EF

G

A B

C D EF

G

A B

C D EF

G

Graph Coloring




A B

C D EF

G

A B

C D EF

G

A B

C D EF

G

DF

A

C

B

G

E

A B

C D EF

G

Graph Coloring

DF

A

C

B

G

E

Undirected graph

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliques

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

A B

E

DF

A

C

B

G

E

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

A B

E

DF

A

C

B

G

EDF

A

C

B

G

E

B

G

E

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

A B

E

DF

A

C

B

G

EDF

A

C

B

G

E

B

G

EDF

A

C

B

G

E

D

G

E

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

A B

E

DF

A

C

B

G

EDF

A

C

B

G

E

B

G

EDF

A

C

B

G

E

D

G

E

DF

A

C

B

G

E

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

A B

E

DF

A

C

B

G

EDF

A

C

B

G

E

B

G

EDF

A

C

B

G

E

D

G

E

DF

A

C

B

G

EDF

A

C

B

G

E

DF

A

C

D

A

E

A B

EB

G

E

D

G

E

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

A B

E

DF

A

C

B

G

EDF

A

C

B

G

E

B

G

EDF

A

C

B

G

E

D

G

E

DF

A

C

B

G

EDF

A

C

B

G

E

DF

A

C

D

A

E

A B

EB

G

E

D

G

E

DF

A

C

B

G

E

Graph Coloring

DF

A

C

B

G

E

DF

A

C

Maximal cliquesD

A

E

DF

A

C

B

G

E

A B

E

DF

A

C

B

G

EDF

A

C

B

G

E

B

G

EDF

A

C

B

G

E

D

G

E

DF

A

C

B

G

EDF

A

C

B

G

E

DF

A

C

D

A

E

A B

EB

G

E

D

G

E

DF

A

C

B

G

E

A B

C D EF

G

Graph Coloring

Sudoku is also a graph coloring problem

Graph Coloring


4x4 Sudoku example:

A B C D

HGFE

I J K L

PONM

Graph Coloring


4x4 Sudoku example:

A B C D

HGFE

I J K L

PONM

A B C D

HGFE

I J K L

PONM

rows

Maximal cliques:

Graph Coloring


4x4 Sudoku example:

A B C D

HGFE

I J K L

PONM

A B C D

HGFE

I J K L

PONM

rows

Maximal cliques:

A B

FE

C D

HG

I J

NM

K L

PO

blocks

Graph Coloring


4x4 Sudoku example:

A B C D

HGFE

I J K L

PONM

A B C D

HGFE

I J K L

PONM

rows

Maximal cliques:

A B

FE

C D

HG

I J

NM

K L

PO

blocks

A

E

I

M

B

F

J

N

C

G

K

O

D

H

L

P

columns

Graph Coloring


4x4 Sudoku example:

A B C D

HGFE

I J K L

PONM

A B C D

HGFE

I J K L

PONM

rows

Maximal cliques:

A B

FE

C D

HG

I J

NM

K L

PO

blocks

A

E

I

M

B

F

J

N

C

G

K

O

D

H

L

P

columnsA B C D

HGFE

I J K L

PONM

C D

HG

A B

FE

K L

PO

I J

NM

D

H

L

P

C

G

K

O

B

F

J

N

A

E

I

M

Graph Coloring


4x4 Sudoku example:

A B C D

HGFE

I J K L

PONM

A B C D

HGFE

I J K L

PONM

rows

Maximal cliques:

A B

FE

C D

HG

I J

NM

K L

PO

blocks

A

E

I

M

B

F

J

N

C

G

K

O

D

H

L

P

columnsA B C D

HGFE

I J K L

PONM

C D

HG

A B

FE

K L

PO

I J

NM

D

H

L

P

C

G

K

O

B

F

J

N

A

E

I

M

1 2 3 4

3 4 1 2

2 3 4 1

4 1 2 3

1 2

3 4

3 4

1 2

2 3

4 1

4 1

2 3

1

3

2

4

2

4

3

1

3

1

4

2

4

2

1

3

Graph Coloring

1 2 3 4

3 4 1 2

2 3 4 1

4 1 2 3

A B C D

HGFE

I J K L

PONM

Probabilistic Graphical ModelsGraph Coloring

1 2 3 4

3 4 1 2

2 3 4 1

4 1 2 3

A B C D

HGFE

I J K L

PONM

1 2 3 4

2143

2 3 4 1

3214


1 2 3 4

3 4 1 2

2 3 4 1

4 1 2 3

A B C D

HGFE

I J K L

PONM

1 2 3 4

2143

2 3 4 1

3214

But how dowe take allconstraints

into account?


Probabilistic Graphical Models

PROBABILISTIC GRAPHICAL MODELSIn a general sense, a PGM of a system• clusters information into local sections, and• let the sections communicate about their combined outcome


In a probabilistic sense, these "local sections" are• prior distributions,• marginal distributions, and/or• conditional distributions;together, a compact representation of a larger space



A B

C D EF

G


A B

C D EF

G

DF

A

C D

A

E

A B

E

D

G

EB

G

E


A B

C D EF

G

DF

A

C D

A

E

A B

E

D

G

EB

G

E

P(A,B,C,D,E,F,G) = f (A,C,D,F) · f (A,D,E) · f (A,B,E) · f (B,E,G) · f (D,E,G)1 2 3 4 5


A B

C D EF

G

DF

A

C D

A

E

A B

E

D

G

EB

G

E


1 2 3 4 11 2 4 3 11 3 2 4 1

4 3 2 1 1non normalized...

P(A,B,C,D)

elsewhere 0

A,C,D,F



A B

C D EF

G

DF

A

C D

A

E

A B

E

D

G

EB

G

E


1 2 3 4 11 2 4 3 11 3 2 4 1


P(A,B,C,D)

elsewhere 0

A,C,D,F


1 2 3 11 2 4 11 3 2 1

4 3 2 1non norm...

P(B,E,G)

elsewhere 0

1 2 3 4 11 2 4 3 11 3 2 4 1


P(A,B,C,D)

elsewhere 0

B,E,G



A B

C D EF

G

DF

A

C D

A

E

A B

E

D

G

EB

G

E


1 2 3 4 11 2 4 3 11 3 2 4 1


P(A,B,C,D)

elsewhere 0

A,C,D,F


1 2 3 11 2 4 11 3 2 1

4 3 2 1non norm...

P(B,E,G)

elsewhere 0

1 2 3 4 11 2 4 3 11 3 2 4 1


P(A,B,C,D)

elsewhere 0

B,E,G


A,B,E D,E,GA,D,E


In a general sense, a PGM of a system• clusters information into local sections, and• let the sections communicate about their combined outcome

A B

C D EF

G

A,C,D,F B,E,GA,B,E D,E,GA,D,E

PROBABILISTIC GRAPHICAL MODELS


A B

C D EF

G


Factor/Bethé graph

A,D,EA,B,EA,C,D,F B,E,G E,D,G

A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE



A B

C D EF

G


Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G



A B

C D EF

G


Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G



A B

C D EF

G


Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G



A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G



A B

C D EF

G


Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G



A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G



A B

C D EF

G


Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G



A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G



A B

C D EF

G


Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G



A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G



A B

C D EF

G


Factor/Bethé graph


A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G



A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G


A B

C D EF

G


A B

C D EF

G


A B

C D EF



A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

A B

C D EF

G

A B

C D EF

G


A B

C D EF



A B F GC D E

A B

C

D

E

FGA

B

DE G

A

DE

A B

C D EF

G

A B

C D EF

G

Cluster graph

A,D E,DA,D,E

A,E

A,B,E B,E

A,C,D,F

B,E,G

E,D,G

G

A B

C D EF

G

A B

C D EF

G


CLUSTER GRAPHS

CLUSTER GRAPHSWe found that• Graph structure influence convergence speed and accuracy• Factor graphs are predominant in PGM literature• Cluster graphs outperform factor graphs


Why is cluster graphs the underdog?• Multiple solutions for the same clusters• Absence of a generic construction procedure


Why is cluster graphs the underdog?• Multiple solutions for the same clusters• Absence of a generic construction procedure

We propose the LTRIP procedure as a solution

B,C,D,E,FA,B,C,DA,B,G B,C,GB,E,F

Factor/Bethé graph:

A B C D E F G variables

clusters

CLUSTER GRAPHS




clusters

connection layer GA B C D E F

CLUSTER GRAPHS




clusters


*running intersection property

sepsets

A

A A

CLUSTER GRAPHS




clusters



sepsets

B

B BB B B

CLUSTER GRAPHS




clusters



sepsets C C C

C

CLUSTER GRAPHS




clusters



sepsets

A B C D E F G

G

AB

C

E FA

B

B

BB C CD

D EF G

CLUSTER GRAPHS


Cluster graph:


clusters



CLUSTER GRAPHS


Cluster graph:


clusters



CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

A

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,GA

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

B

BBA

B

B

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

C

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

B,C

BB,CA

B

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

D

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

B,C,D

BB,CA

B

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

E

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

B,C,D

BB,CA

B,E

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

F

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

B,C,D

BB,CA

B,E,F

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

G

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

B,C,D

BB,C

GA

B,E,F

CLUSTER GRAPHS


multivar. sepsets

LTRIP procedure:


clusters

B,E,F

B,C,D,E,F

A,B,C,D

A,B,GB,C,G

B,C,D

BB,C

GA

B,E,F

CLUSTER GRAPHS

A B C D

HGFE

I J K L

PONM

Sudoku cluster graph:

CLUSTER GRAPHS

A B C D

HGFE

I J K L

PONM


A B C D

HGFE

I J K L

PONM

A B

FE

C D

HG

I J

NM

K L

PO

A

E

I

M

B

F

J

N

C

G

K

O

D

H

L

P

CLUSTER GRAPHS

A B C D

HGFE

I J K L

PONM


A,B,C,D

E,F,G,H

A,E,I,M

B,F,J,N

A,B

E,F

A,E

B,F

C,D

G,H

I,M

J,N

A,B,E,F

C,D,G,H

I,J,M,N

C,G

D,H

I,J

M,N

K,L,O,P

K,O

L,P

K,L

O,P

I,J,K,L

C,G,K,O

D,H,L,P

M,N,O,P

CLUSTER GRAPHS

A

A,B,C,D E,F,G,H A,E,I,M B,F,J,N A,B,E,F C,D,G,H K,L,O,PI,J,K,L C,G,K,O D,H,L,PM,N,O,P I,J,M,N

A E I MB F J NC G K OD H L PE I

M

FJ

N

CG

K

OD

H

L

P

A

B

E

I

M

F

J

N

C G K

O

DH

L

PA

B

E

I

MF

J

N

CG

K

O

DH

LP

B

Sudoku factor/Bethé graph:

CLUSTER GRAPHS

CLUSTER GRAPHSRESULTS

A comparison of cluster graphs vs. factor graphs on PGMs build from Sudoku puzzles

Datasets used:• Project Euler @ projecteuler.net/problem=96• Sterten's 95 hardest Sudokus @ magictour.free.fr/top95

Tested with different cluster sizes by splitting-up clusters


A B C D E F G H I

9x9 Sudoku puzzle

...J K L M N O P Q R


A B C D E F G H I

9x9 Sudoku puzzle

...J K L M N O P Q R

A B C D E F G H I

...

J K L M N O P Q R


A B C D E F G H I

Split each 9 variable clique

C B A

D E A

F G A

H I A

H I B

F B G

D E B

C B D

F E C

H C G

D I C

D G H

D E F

F E G

H E I

F I H

F G H

H I G

into cliques of 3 variables


A B C D E F G H I

Split each 9 variable clique into cliques of 5 variables

E D B A C

F I G A H

F B I G H

E D F B C

D I G H C

E D F G H

E F I G H


A B C D E F G H I


E D F B G A C

E F B I G A H

E D F B G H C

E D F I G H C



A B C D E F G H I


A B C D E F G H I



A B C D E F G H I


A B C D E F G H I


A,B,C,D

E,F,G,H

A,E,I,M

B,F,J,N

A,B

E,F

A,E

B,F

C,D

G,H

I,M

J,N

A,B,E,F

C,D,G,H

I,J,M,N

C,G

D,H

I,J

M,N

K,L,O,P

K,O

L,P

K,L

O,P

I,J,K,L

C,G,K,O

D,H,L,P

M,N,O,P

A



M

FJ

N

CG

K

OD

H

L

P

A

B

E

I

M

F

J

N

C G K

O

DH

L

PA

B

E

I

MF

J

N

CG

K

O

DH

LP

B

Build these clusters into both a factor graph and a cluster graph


A B C D E F G H I


A B C D E F G H I


A,B,C,D

E,F,G,H

A,E,I,M

B,F,J,N

A,B

E,F

A,E

B,F

C,D

G,H

I,M

J,N

A,B,E,F

C,D,G,H

I,J,M,N

C,G

D,H

I,J

M,N

K,L,O,P

K,O

L,P

K,L

O,P

I,J,K,L

C,G,K,O

D,H,L,P

M,N,O,P

A



M

FJ

N

CG

K

OD

H

L

P

A

B

E

I

M

F

J

N

C G K

O

DH

L

PA

B

E

I

MF

J

N

CG

K

O

DH

LP

B

Build these clusters into both a factor graph and a cluster graph

Run message passing on both graphs(note, for a solved Suduko all clusters is reduced to a single entry)

P(A,B,C,D,E,F,G,H,I)A1

B3

C2

D6

E4

F5

G7

H9

I8

elsewhere1

0


Project Euler: solution count

36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3



36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3

0



36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3

0

cluster graph slower



36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3

0


25 0

232

25

25

2327

factor graphcl

uste

r gra

phSize 5



36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3

0


25 0

232

25

25

2327

factor graphcl

uste

r gra

phSize 5

0



36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3

0


25 0

232

25

25

2327

factor graphcl

uste

r gra

phSize 5

0




36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3

0


25 0

232

25

25

2327

factor graphcl

uste

r gra

phSize 5

0


5 0

414

5

45

419

0 0

464

0

50

464

cluster graph faster cluster graph faster

factor graph factor graph

clus

ter g

raph

clus

ter g

raph

Size 7 Size 9

0 0



36 0

131

36

14

1337

factor graph

clus

ter g

raph

Size 3

0


25 0

232

25

25

2327

factor graphcl

uste

r gra

phSize 5

0


5 0

414

5

45

419

0 0

464

0

50

464

cluster graph faster cluster graph faster

factor graph factor graph

clus

ter g

raph

clus

ter g

raph

Size 7 Size 9

0 0

Cluster graphs more successful than factor graphsNaive solver to test graph structures - can improve!


CONCLUSION• Main contribution is LTRIP for constructing cluster graphs• These cluster graphs show great promise over factor graphs• We hope LTRIP will enhance the popularity of cluster graphs

CONCLUSION• Main contribution is LTRIP for constructing cluster graphs• These cluster graphs show great promise over factor graphs• We hope LTRIP will enhance the popularity of cluster graphs

FUTURE WORK• Investigate more advance techniques for graph coloring PGMs• Mutual information approach for LTRIP's max spanning trees• Investigate cluster graphs on wider set of problems

Documents

Graph Coloring: Comparing Cluster Graphs to Factor Graphs