Structural pattern recognition - The Home of Romain Raveauxromain.raveaux.free.fr/document/presentation romain... · 2019-03-04 · Structural pattern recognition Romain Raveaux [email protected]

Structural pattern recognition

Romain [email protected]

Maître de conférences

Université de Tours

Laboratoire d’informatique (LIFAT)

Equipe RFAI

Optimisation combinatoire et apprentissage structurel pour

l'appariement et la classification de graphes

Contributions and perspectives

Contributions and perspectives on combinatorial optimization

and machine learning onto graph space

Application to : graph matching and graph classification

Presentation : Romain Raveaux

• CV

• Teaching

• Research

• Projects

• Administrative tasks

R. Raveaux 4

CV

• Diplomas and qualification

2010-2011

CNU Qualification : 27

2006-2010

Doctorat de l’Université de La Rochelle. Mention : Très honorable avec félicitations

Fouille et classification de graphes : Application à l’analyse d’images cadastrales couleurs.

2004-2006

Master Génie Informatique, Université de Rouen.

Option : Extraction et Indexation de l’information.

2004-2006 :

Master Génie Électrique et Informatique Industrielle,Université de Rouen.

Option : Réseaux et Télécoms.

R. Raveaux 5

CV

• Since 2012, teacher at Université de Tours• At Polytech’Tours (Engineering school)

• Computer Science department

• Since 2012, researcher at LIFAT• Computer science laboratory

• Team : Pattern recognition and image analysis

• 2011-2012 : Research engineers at SOOD company

• 2010-2011 : Attaché Temporaire d’Enseignement et de Recherche à l’IUT d’informatique de La Rochelle

R. Raveaux 6

Teaching

• Teaching : 241 h EqTD

• Projects : 30 h EqTD

• Responsability : 25 EqTD

R. Raveaux 7

Research

• Topic : Machine learning, discrete optimization, computer vision

• Supervisor of : 3 PhD students and 3 master students

• 15 journal papers and 25 conferences.

• Projects : members of 3 projets• ALPAGE, CARAMBA, VISIT

• Collaboration : National (GREYC, LITIS, LORIA). International (Campinas,Brazil, Griffith University, Australia)

• Scientific committee: GBR, past: GREC

R. Raveaux 8

Content

• Introduction• Title explanation• Data• Graph-based applications• Applicative Tools

• Combinatorial optimization for graph matching and graph classification• Graph matching• Graph classification

• Machine learning in graph space for matching and classification• Graph matching• Graph classification

• Conclusion• Summary• Perspective

R. Raveaux 9

Title explanation

• Graph matching in a single picture :

Introduction : Combinatorial Optimization : Machine Learning : ConclusionTitle explanation – Data- Graph-based applications - Applicative Tools

R. Raveaux 10

Title explanation

• Graph classification in a single picture :

1

2

3

G

Class 4Classifier


R. Raveaux 11

Title explanation

• Combinatorial optimization in a single picture :

A1 B2

3 C D

1-A 1-B 1-C

root

2-A 2-B

3-B

G2G1

1-D

3-C

3-C


R. Raveaux 12

Title explanation

• Machine learning in a single picture : Decision processData

Data set 1

Decision

Learningprocess

Model

Data set 2

Introduction : Combinatorial Optimization : Machine Learning : ConclusionTitle explanation – Data- Graph based applications - Applicative Tools

R. Raveaux 13

Title explanation

• Graph space in a single picture :

Introduction : Combinatorial Optimization : Machine Learning : ConclusionTitle explanation – Data - Graph-based applications - Applicative Tools

R. Raveaux 14

Data

• Data are crucial for computer scientists

• Data driven :• problems

• models

• solution methods (solving methods)


R. Raveaux 15

Type of Data

Taken from M. Bronstein. CVPR Tutorial 2017 16


R. Raveaux

Focus on structured Data

• String

• Tree

• Graph

17


R. Raveaux


String

Taken from: Marçal Rusiñol et al: Symbol spotting in vectorized technical drawingsthrough a lookup table of region strings. PAA 2010

18R. Raveaux


Tree

R. Raveaux et al:Structured representations in a content based image retrieval context. J. Visual Communication and Image Representation 24(8): 1252-1268 (2013)

19R. Raveaux

Graph

20

• Data are represented as graphs:• By nature (Social Network, Molecule, Protein interaction network)

• By construction (from images)


R. Raveaux

Graph

21


A graph can be represented by 4-tuple 𝐺 = (𝑉, 𝐸, 𝜇, 𝜉), with:

V the set of vertices,

𝐸 ⊆ 𝐸 ∩ 𝑉 × 𝑉 the set of edges,

𝜇: 𝑉 → 𝐿𝑉 the function that assigns attributes to vertices,

𝜉: 𝑉 → 𝐿𝐸 the function that assigns attributes to edges,

𝐿𝑉 the set of all possible attributes for vertices,

and 𝐿𝐸 the set of all possible attributes for edges.

R. Raveaux

Graph

22


R. Raveaux

Region Adjacency Graph

23


R. Raveaux

Region Adjacency Graph

Impact of noise on Graph-Based Representation 24


R. Raveaux

Neighborhood graph

25


R. Raveaux

Skeleton Graph

26


R. Raveaux

Graph of molecules : Chemoinformatics

27


R. Raveaux

Domain structure vs Data on a domain

Taken from [Brontein 2016, CoRR] 28


R. Raveaux

Fixed vs different domain

29

1D,2D, 3D shapes(Different graphs)


R. Raveaux

Structured data

• We will focus on graphs:• Graph as a generalization of Euclidean data : vector, matrix, tensors ….

• Graphs as a generalization of strings and trees.

• By nature, data are more likely to be graphs.

30


R. Raveaux

Graph-based applications

• Graph classification/clustering/regression

• Vertex classification/clustering/regression

• Graph matching

• Graph distance

• Graph-based search• Subgraph search• Subgraph spotting• Similarity search

• Graph prototypes• Median graphs/ Super graphs

Artificial Intelligence

is the set of tools to solve these problems : Modelisation, Machine learning, Optimization … …

31


R. Raveaux

Graph classification

32

1. cancerous or not cancerous molecules

2. determination of the boiling point

Molecular graph


R. Raveaux

Vertex classification



R. Raveaux

Vertex classification/clustering



R. Raveaux

Semi-supervised vertex classification

• Setting: • Some nodes are labeled (black circle)

• All other nodes are unlabeled

• Task: • Predict node label of unlabeled nodes


R. Raveaux 35

Graph matching

• When parts of the object must be tracked or compared.

• Detect and recognize at once


R. Raveaux 36

Graph matching (https://goo.gl/8dYCZb)

• Mettre vidéo graph matching

37


R. Raveaux

Graph matching (https://goo.gl/wi5m1E)

38


R. Raveaux

Graph distance/similarity

How similar are theses graphs ?

39


R. Raveaux

Graph prototype

d1d2

d3

40

G2

G3

G1

Gm


R. Raveaux

Graph-based search

q G2

G1

G3

G4

G5

G6

41


R. Raveaux

What do we need to solve these problems ?

• Modelisation, Machine learning, Optimization, …:• Graph comparison :

• Graph matching

• Graph distance

• Learning capability with graphs :• Learn to match

• Learn to classify

42

Introduction : Combinatorial Optimization : Machine Learning : ConclusionTitle explanation – Data- Graph-based applications - Tools

R. Raveaux

Two ways to solve the problems

• Graph/Node embedding : [Luqman et al., 2017, PR], [Kipf and Welling, 2017, ICLR].• The graphs/nodes are projected into a vector space

• Graph space : [Neuhaus and Bunke., 2007, MPAI], [Riesen, 2015, ACVPR].• The graphs are compared through graph matching methods.


R. Raveaux 43

Graph space vs graph embedding

Graph space Graph embedding

Graphcomparison

Pros • Manage structure and features at once

• Preserve the structure of graphs

• Fast algorithms available• Many learning techniques

Cons • Slow (Often) a combinatorialproblem

• Lack of learning techniques

• Does not capture the combinatorialnature of the problem

• Loss of topological information


R. Raveaux 44

Our positionning

• Many works have focus their interest on graph embeddings.• Review on graph embeddings : [Goyal and Ferrara, 2017, KBSyst][Cai et al,

2017, IEEE TKDE].

• We think it is unfortunate to reduce the graphs into vectors.

• We explore new avenues:• It is interesting to develop learning techniques in graph space.

• It is interesting to design optimization methods in graph space.


R. Raveaux 45

Content

• Combinatorial optimization for graph matching and graph classification• Graph matching

• Graph classification

• Machine learning in graph space for matching and classification• Graph matching

• Graph classification


R. Raveaux 46

Combinatorial optimization for graph matchingand graph classification• Learning free methods.

• Pure combinatorial problems

Introduction : Combinatorial Optimization : Machine Learning : ConclusionProblems – State of the art – Deadlocks – Contributions - Summary

R. Raveaux 47

Graph matching

• Problems

• State of the art

• Deadlocks

• Contributions

Introduction : Combinatorial Optimization : Machine Learning : ConclusionGraph Matching : Problems – State of the art – Deadlocks – Contributions - Summary

R. Raveaux 48

Graph matching : Problems

• Exact matching

• Error-Tolerent matching• Subgraph matching

• Error-correcting matching

• can be expressed as QAP, MAP-inference, GED problems.


R. Raveaux 49

Graph matching : the Graph Edit Distance (GED) Problem

2

1

G

3

4

a

c

b

G’

2 3

4

a 3

4

a b

4

a b

c

This solution costs:

𝑑 =

𝑜𝑝𝑖 ∈ 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠

𝐶(𝑜𝑝𝑖)

Deletion Substitution Substitution Both

Vertices operations

• OP1: 1 ϵ C1

• OP3: 2 a C2

• OP4: 3 b C3

• OP6: 4 c C4

Edges operations

• OP2: (1,2) ϵ C5

• OP5: (2,3) (a,b) C6

• OP7: (3,4) (b,c) C7

• OP8: (2,4) ϵ C8

The goal is to find 𝑑min the set of operations with the minimum cost.


Graph matching : the Qadratic AssignmentProblem

𝜙 𝑖 = 𝑘𝜙 𝑗 = 𝑙𝐷𝑖,𝑘,𝑗,𝑙 = 𝑐 𝑖𝑗, 𝑘𝑙𝐵𝑖,𝑘 = 𝑐 𝑖, 𝑘𝑁 = 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑛𝑜𝑑𝑒𝑠 = 2

i

j

k

l

G1 G2

ij kl


R. Raveaux 51

Graph matching : MAP-inference of a ConditionalRandom Field (CRF)Finding the most likely configuration of discrete CRF. CRF=G1

i

j

k

l

G1 G2

ij kl

𝑥𝑖 𝑎 𝑑𝑖𝑠𝑐𝑟𝑒𝑡𝑒 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒 𝑎𝑠𝑠𝑜𝑐𝑖𝑎𝑡𝑒𝑑 𝑡𝑜 𝑛𝑜𝑑𝑒 𝑖𝑥𝑖 𝑐𝑎𝑛 𝑡𝑎𝑘𝑒 𝑖𝑡𝑠 𝑣𝑎𝑙𝑢𝑒 ∈ 𝑘, 𝑙𝑥𝑖𝑗 𝑡𝑎𝑘𝑒𝑠 𝑖𝑡𝑠 𝑣𝑎𝑙𝑢𝑒 ∈ 𝑘, 𝑙 × {𝑘, 𝑙}

𝑉 = 𝑖, 𝑗 𝑎𝑛𝑑 𝐸 = 𝑖𝑗𝜃 𝑥𝑖 = 𝑐𝑜𝑠𝑡 𝑡𝑜 𝑎𝑠𝑠𝑜𝑐𝑖𝑎𝑡𝑒 𝑖 𝑤𝑖𝑡ℎ 𝑡ℎ𝑒 𝑙𝑎𝑏𝑒𝑙 𝑥𝑖𝜃 𝑥𝑖𝑗 = 𝑐𝑜𝑠𝑡 𝑡𝑜 𝑎𝑠𝑠𝑜𝑐𝑖𝑎𝑡𝑒 𝑖𝑗 𝑤𝑖𝑡ℎ 𝑡ℎ𝑒 𝑙𝑎𝑏𝑒𝑙 𝑥𝑖𝑗


R. Raveaux 52

Graph matching : Related Problems


Errot-tolerant graph matching : NP-hard problem. [Zeng et al. 2009, PVLDB]: Combinatorial problem

R. Raveaux 53

Graph matching models

• Graph matching can be expressed by : • Integer Quadratic Program

• Integer Linear Program


R. Raveaux 54

i

j

k

l

G1 G2

ij kl

Graph matching models : Integer QuadraticProgram


R. Raveaux 55

Graph matching models : Integer Linear Program

• Linear constraints

• Linear objective function


R. Raveaux 56

Graph matching : State of the art

Subgraph GM problem Methods

Models

Exact Heuristique

ILP #papers : 0 #papers : 0

IQP #papers : 0 #papers : 7IPFP[Leordeanuet al., 2009]


R. Raveaux 57

Graph matching : State of the art

Error-correcting GM problem

Methods

Models

Exact Heuristique

ILP #papers : 0 #papers : 1ILP [Justice andHero, 2006]

IQP #papers : 0 #papers : 3mIPFP[Bougleuxet al., 2017b]


R. Raveaux 58

Graph matching : State of the artMethods Error-correcting GM Subgraph GM

Constrained quadraticprogramming

#papers : 1 mIPFP [Bougleuxet al., 2017b]

#papers : 4 – IPFP [Leordeanuet al., 2009]

Spectral #papers : 0 #papers : 2 SM[Leordeanuand Hebert, 2005]

Branch and bound #papers : 2 A* [Riesenet al., 2007]

#papers : 0

Bio-inspired #papers : 1 GEDEVO [Ibragimovet al., 2013]

#papers : 2 AG [Cross et al., 1997]

Hungarian method #papers : 3 SFBP [Serratosa,2015]

#papers : 0

Probabilistic #papers : 1 BayesianGED [Myers et al.,2000]

#papers : 1 [Zass andShashua, 2008]


R. Raveaux 59

Deadlocks :

• Facts:• Exact methods are rarely study

• Few works have paid attention to ILP models.

• Deadlocks :• To study exact methods

• To derive heuristics from exact methods

• There is a need to study heuristics according to speed and effectiveness criteria.


R. Raveaux 60

Exact methods for error-correcting matching

• Integer Linear Program


R. Raveaux 61

• Notations

i

j

k

l

G1 G2

ij lk

x

x

y

Error-correcting matching : Integer Linear Program

62


[Lerouge et al., 2017, PR]R. Raveaux

• Objective function:

Vertex substitutions Edge substitutions vertex deletions

vertex insertions edge insertionsedge deletions


63


R. Raveaux

substitutionsdeletion

Insertion

Insertion

deletionsubstitutions

Vertices

mapping

constraints

Edges

mapping

constraints


64


R. Raveaux

• Topological constraints

i

j

k

l

G1 G2

ij lk

x

x

y


65


R. Raveaux

Heuristic : LocBra

• A Matheuristic is derive from F1.

• Δ = 𝐻𝑎𝑚𝑚𝑖𝑛𝑔 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒

• Adding branching constraints


R. Raveaux 66

Heuristic : Anytime

• Export all the list of dUB

• Interruptability: At each iteration, we can

stop the algorithm if it exceeds CT


[Abu-Aisheh et al., 2016, PRL]

R. Raveaux 67

Antyime

A A’B B’

C’C D

A-A’ A-B’ A-C’ A-ϵ

root

B-A’ B-C’ B-ϵ

C-C’ C-ϵ

D--ϵ

B-A’ B-B’ B-ϵ

g=1

h=4

f=5

g=2

h=5

f=7

g=3

h=2

f=5

g=1

h=5

f=6

g=4

h=2

f=6

g=4

h=3

f=7

g=4

h=3

f=7

g=1

h=5

f=6

g=1.5

h=4

f=5.5

g=3

h=3

f=6B-B’

g=4

h=3

f=7

g=1.5

h=4

f=5.5

g=2

h=3.8

f=5.8

g: current cost

h: estimated cost

f : total cost

G1 G2


R. Raveaux 68

https://goo.gl/xJn5nq


R. Raveaux 69

https://goo.gl/CebSg2


R. Raveaux 70

Benchmarking


R. Raveaux 71

Conference paper: Z. Abu-Aisheh, R. Raveaux and J-Y Ramel: A Graph Database Repository and

Performance Evaluation Metrics for Graph Edit Distance. GbRPR 2015 : 138-147.

Database # graphs avg. (max) # nodes

Decomposition Overview Purpose

GREC 50 12.5 (20) MIX, 5, 10, 15 and 20 Classification

CMU 111 30 (30) 30 Matching

MUTA 80 40 (70) MIX ,10, 20, 30, … and

70

Classification

PAH 94 20.7 (28) - Classification

Acyclic 185 8.2 (11) - Classification

Alkane 150 8.9 (10) - Classification

Datasets


[Abu-Aisheh et al.,2015a, GBR]

R. Raveaux 72

ICRP 2016 Contest



R. Raveaux 73

Summary

• Error tolerent graph matching :• New ILP models : F1, F2, F3 :

• New exact methods : Branch-bound, mathematical solver : • Fast convergence, evaluation with heuristics

• New heuristics : Matheuristic and Anytime• Accurate and flexible

• New benchmarks

• A contest


R. Raveaux 74


• Problems


• Deadlocks

• Contributions

Introduction : Combinatorial Optimization : Machine Learning : ConclusionGraph Classification : Problems – State of the art – Deadlocks – Contributions - Summary

R. Raveaux 75


• Learning-free


R. Raveaux 76

Problem


R. Raveaux 77

State of the art : speeding up the kNN problem

Generic (dissimilarity space) Graph oriented methods

Dissimilarity functionMetric property [Uhlmann, 1991, IPL] Fast GED methods [Riesen,

2015, ACVPR ]

Dissimilarity spaceSpace partition, prototypes, hashtable, proximity graph, line search [Malkov and Yashunin, 2016, PAMI]

Graph prototypes [Musmanno and Ribeiro, 2016, EJOR]

• Structuring the dissimilarity space lies at the art of machine learning (out of scope here)

• Graph distance is a combinatorial problem.• Many methods have focused on : fast GED methods.


R. Raveaux 78

Deadlocks

• Is there a way to specialize a line search method to operate on graph space instead of the generic dissimilarity space ?


R. Raveaux 79

Is there a way to specialize a line search method to operate on graph space instead of the generic dissimilarity space ?

• Merging the GED problem and the kNN problem in a single problem:

Γ set of all possible matchings between 𝐺 ∈ 𝑇𝑒𝑆 and 𝐺𝑗 ∈ 𝑇𝑟𝑆

Merging

The set of all possible matchings between 𝐺 and all 𝐺𝑗 ∈ 𝑇𝑟𝑆


R. Raveaux 80

Is there a way to specialize a line search method to operate on graph space instead of the generic dissimilarity space ?

• Merging the GED problem and the kNN problem in a single problem:

Merging



R. Raveaux 81

Solving the MGED problem

• A Branch and bound method



R. Raveaux 82

Data sets


R. Raveaux 83

Methods

• Fast GED methods : BP, FBP, DF, BS-1

• Our proposal : one-tree (with different initialisation)


R. Raveaux 84

Classification test


R. Raveaux 85

Summary

• « Learning free » methods for classification

• Merging GED problem and kNN in a single problem (MGED)

• New algorithm for this problem

• Efficient when the data size increases

• Can be combined with other learning based-methods:• Learning the distance

• Learning prototypes


R. Raveaux 86

Machine learning in graph space for matching and classification

Introduction : Combinatorial Optimization : Machine Learning : ConclusionProblems – State of the art – Deadlocks – Contributions - Summary

R. Raveaux 87

Graph matching

• Problem


• Deadlocks

• Contributions


R. Raveaux 88

Learning graph mathching problem

A possible loss :


R. Raveaux 89

State of the art


R. Raveaux 90

Deadlocks

• How to deal with insertion and deletion costs ?

• Can an heuristic output solutions closer to optimality thanks to machine learning?


R. Raveaux 91

Parametrized GED


[Raveaux et al., 2017, GBR].

R. Raveaux 92

Parametrized GED

Φ = 𝑑 ∗, 1 ; 𝑑 ∗, 2 ; 𝑑 ∗, ε ; 𝑑 ∗∗, 12 ; 𝑑 ∗∗, εε𝛽 = 𝛽1 ; 𝛽2 ; 𝛽ε ; 𝛽4 ; 𝛽εε


R. Raveaux 93

Learning with parametrized GED algorithm

• The learning problem reformulated :


R. Raveaux 94

Learning algorithm : gradient descent


R. Raveaux 95

Data set

CMU-House

|TrS| 50

|TeS| 50

|V| 30

|E| 70

Attributes xy distance

Node matching cost 1

Edge matching cost L1 norm


R. Raveaux 96

Methods

• GED solver : BP

• BP without learning

• BP with our learning scheme


R. Raveaux 97

Learning graph matching testhttps://youtu.be/JrMR2-5mjA4


R. Raveaux 98

Learning Graph matching test


R. Raveaux 99

Summary

• Parametrized error correcting graph matching

• Learning scheme for graph matching

• Independent to the graph matching methods


[Raveaux et al., 2017, GBR].

R. Raveaux 100


• Problem


• Deadlocks

• Contributions


R. Raveaux 101

Problem


R. Raveaux 102

State of the art


R. Raveaux 103

State of the art


R. Raveaux 104

State of the art


R. Raveaux 105

State of the art


R. Raveaux 106

Deadlocks

• Learning graph distance for classification with local parameters for nodes and edges.

• Learning graph matching and graph prototypes in a hierarchical manner.


R. Raveaux 107

Learning graph distance for classification with local parameters for nodes and edges.

Learning rule


[Martineau et al., 2018, PRL].

R. Raveaux 108

Learning graph distance : Learning algorithm


[Martineau et al., 2018, PRL].R. Raveaux 109

Benchmark


R. Raveaux 110

Data sets


R. Raveaux 111

Methods

• Learning schemes for global parameters :• Grid search [Riesen et al, 2009, IVC] (R-1NN)

• Constraint quadratic programming [Cortes et al, 2015, PRL] (C-1NN)

• 1NN methods

• 1NN based on median graphs

R. Raveaux 112

Results


R. Raveaux 113

Learning graph matching and graph prototypes in a hierarchical manner.


R. Raveaux 114

The basics of artificial neural networks

115


R. Raveaux

The basics of graph neural networks

116


R. Raveaux

Adjacency matrix

117


R. Raveaux

Degree matrix

118


R. Raveaux

Key idea and Intuition [Kipf and Welling, 2016]

• The key idea is to generate node embeddings based on local neighborhoods.

• The intuition is to aggregate node information from their neighbors using neural networks.

• Nodes have embeddings at each layer and the neural network can be arbitrary depth. “layer-0” embedding of node u is its input feature, i.e. Fu.

119


R. Raveaux

GNN: a pictorial model

120


R. Raveaux

Simple example of f

121


R. Raveaux

2 issues of this simple example

• Issue 1:• for every node, f sums up all the feature vectors of all neighboring nodes but

not the node itself.

• Fix: simply add the identity matrix to A

• Issue 2:• A is typically not normalized and therefore the multiplication with A will

completely change the scale of the feature vectors.

• Fix: Normalizing A such that all rows sum to one:

122


R. Raveaux

Altogether: [Kipf and Welling, 2016]

123

• The two patched mentioned before +

• A better (symmetric) normalization of the adjacency matrix


R. Raveaux

Our idea : graph matching based neural network


R. Raveaux 124

Graph convolution layer :

• A layer is composed of many filter graphs (GF)

• A layer outputs : A graph with node features and edge features


R. Raveaux 125

GNN for graph classification

• A GNN outputs : node embeddings

• Adding a global average pooling layer

• Average pooling


R. Raveaux 126

Benchmarking


R. Raveaux 127

Data set : MNIST


R. Raveaux 128

Data set

MNIST

# classes 2

|TrS| 1 000 graphs

|TeS| 5 000 graphs

|V| 196 and 75 nodes

Node Attributes Pixel intensity (gray level)

Edge Attributes none


R. Raveaux 129

Methods

• Lenet 5 [Lecun et al., 1998, IEEE]

• MoNet [Monti et al., 2016, CVPR]

• Our


R. Raveaux 130

Results


R. Raveaux 131

Summary

• Learning scheme for error-correcting graph matching in a classification context.

• Hierarchical leaning of filter graphs and graph similarity.

• Deep learning paradigm can be extended to graph space


R. Raveaux 132

Conclusion

• Summary• Perspective

Introduction : Combinatorial Optimization : Machine Learning : ConclusionSummary - Perspective

R. Raveaux 133

Summary

• Remember the cons:• Slow methods (Often) a combinatorial problem• Lack of learning techniques

• Our asnwsers:• Faster methods to compare/classify graphs

• Thanks to discrete optimization techniques.

• Learning algorithm operating in graph space to compare/classify graphs• Parametrized graph matching• Learning scheme merging combinatorial problems and gradient descent

• Deep learning paradigm can be extended to graph space


R. Raveaux 134

Perspectives

• Combinatorial optimization• Application of optimization tools for pattern recognition problems

• Learning prototypes

• Machine learning• Theoretical understanding of what is learned • Common benchmark datasets

• Merging Machine learning and Combinatorial optimization• Integration of machine learning into combinatorial optimization methods

(branch and bounds, matheuristics)• Integration of combinatorial optimization into machine learning mehods

• Combinatorial layers in neural networks


R. Raveaux 135

Thank you for your attention

• Any question ?


Colleagues from the lab

R. Raveaux 136

Our references :

• [Martineau et al., 2018, PRL] : Maxime Martineau, Romain Raveaux, Donatello Conte, Gilles Venturini. Learning error-correcting graph matching with a multiclass neural network. Pattern Recognition Letters, Elsevier, 2018.

• [Raveaux et al., 2017, GBR] : Romain Raveaux, Maxime Martineau, Donatello Conte, Gilles Venturini: Learning Graph Matching with a Graph-Based Perceptron in a Classification Context. GbRPR 2017: 49-58

• [Abu-Aisheh et al., 2017, PRL] : Zeina Abu-Aisheh, Benoit Gaüzère, Sébastien Bougleux, Jean-Yves Ramel, Luc Brun, Romain Raveaux, Pierre Héroux, Sébastien Adam: Graph edit distance contest: Results and future challenges. Pattern Recognition Letters 100: 96-103 (2017)

• [Abu-Aisheh et al.,2015a, GBR] : Zeina Abu-Aisheh, Romain Raveaux, Jean-Yves Ramel: A Graph DatabaseRepository and Performance Evaluation Metrics for Graph Edit Distance. GbRPR 2015: 138-147

• [Abu-Aisheh et al., 2016, PRL] : Zeina Abu-Aisheh, Romain Raveaux, Jean-Yves Ramel: Anytime graph matching. Pattern Recognition Letters 84: 215-224 (2016)

• [Lerouge et al., 2017, PR] : Julien Lerouge, Zeina Abu-Aisheh, Romain Raveaux, Pierre Héroux, Sébastien Adam: New binary linear programming formulation to compute the graph edit distance. Pattern Recognition 72: 254-265 (2017)

R. Raveaux 137

References

• [Brontein 2016, CoRR] Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: goingbeyond euclidean data. CoRR, abs/1611.08097, 2016.

• [Luqman et al., 2017, PR] Ramzi Chaieb, Karim Kalti, Muhammad MuzzamilLuqman, Mickaël Coustaty, Jean-Marc Ogier, and Najoua Essoukri Ben Amara. Fuzzy generalized median graphs computation: Application to content-based document retrieval. Pattern Recognition, 72:266 – 284, 2017.

• [Kipf and Welling, 2017, ICLR] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. CoRR, abs/1609.02907, 2016.

R. Raveaux 138

References

• [Neuhaus and Bunke., 2007, MPAI] M. Neuhaus and H. Bunke. Bridging the gap between graph edit distance and kernel machines. Machine Perception and Artificial Intelligence., 68:17–61, 2007.

• [Riesen, 2015, ACVPR] Kaspar Riesen. Structural Pattern Recognition withGraph Edit Distance - Approximation Algorithms and Applications. Advances in Computer Vision and Pattern Recognition. Springer, 2015. ISBN 978-3-319-27251-1. doi:10.1007/978-3-319-27252-8. URL https://doi.org/10. 1007/978-3-319-27252-8.

• [Goyal and Ferrara, 2017, KBSyst] Palash Goyal and Emilio Ferrara. Graph embedding techniques, applications, and performance: A survey. CoRR, abs/1705.02801, 2017. URL http://arxiv.org/abs/1705.02801

R. Raveaux 139

References

• [Cai et al, 2017, IEEE TKDE] HongYun Cai, Vincent W. Zheng, and Kevin Chen-Chuan Chang. A comprehensive survey of graph embedding: Problems, techniques and applications. CoRR, abs/1709.07604, 2017. URL http://arxiv.org/abs/1709.07604.

• [Zeng et al. 2009, PVLDB] Zhiping Zeng, Anthony K. H. Tung, Jianyong Wang, Jianhua Feng, and Lizhu Zhou. Comparing stars: On approximating graph edit distance. PVLDB, 2(1):25–36, 2009. doi:10.14778/1687627. 1687631. URL http://www.vldb.org/pvldb/2/vldb09-568.pdf.

• [Uhlmann, 1991, IPL] Jeffrey K. Uhlmann. Satisfying general proximity / similarity queries with metric trees. Information Processing Letters, 40(4):175 – 179, 1991. ISSN 0020-0190. doi:https://doi.org/10. 1016/0020-0190(91)90074-R. URL http://www.sciencedirect.com/science/article/pii/ 002001909190074R.

R. Raveaux 140

References

• [Malkov and Yashunin, 2016, PAMI] Yury A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. CoRR, abs/1603.09320, 2016. URL http: //arxiv.org/abs/1603.09320

• [Musmanno and Ribeiro, 2016, EJOR] Leonardo M. Musmanno and Celso C. Ribeiro. Heuristics for the generalized median graph problem. European Journal of Operational Research, 254(2):371 – 384, 2016. ISSN 0377-2217. doi:https://doi.org/10.1016/j.ejor.2016.03.048. URL http://www.sciencedirect.com/science/article/pii/S0377221716301941.

• [Lecun et al., 1998, IEEE] Yann Lecun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, pages 2278–2324, 1998.

R. Raveaux 141

References

• [Monti et al., 2016, CVPR] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. CoRR, abs/1611.08402, 2016.

R. Raveaux 142

Documents

Structural pattern recognition - The Home of Romain Raveauxromain.raveaux.free.fr/document/presentation romain... · 2019-03-04 · Structural pattern recognition Romain Raveaux [email protected]