Morphological Multi-scale Decomposition and efficient representations with Auto-Encoders

April 23th - September 28th

SupervisorsMVA supervisor:

Bastien PONCHON Internship Defense - september 21th 2018

Agenda01. Introduction

02. Part-based Representation using Non-Negative Matrix Factorization

03. Part-based Representation using Auto-Encoders

04. Using a Deeper Architecture

05. Conclusion

2

01 - Introduction

3

Representation Learning and Part-Based representation

○

○

○atom images,

○4

A few recaps on flat mathematical morphology

Dilation by a structuring element SE:

commutes with supremum.5

SE

Erosion by a structuring element SE:

A few recaps on flat mathematical morphology

6

Max-Approximation to Morphological Operators

7

Motivation for Non-Negative and Sparse representation

8

Objectives and Motivations of the Internship

○○

○

○ Universal approximator theorem:

9

Evaluation and Data of the Proposed Models

○ Approximation error of the representation

○ Max-approximation error to the dilation

○ Sparsity of the encoding○ Classification Accuracy

10

02 - Non-Negative Matrix Factorization

11

General Presentation02 -

12

○ Matrix factorization algorithm:

data matrixdictionary matrixencoding matrix

○ separable factorial articulation family:●

●●

Addition of sparsity constraints (Hoyer 2004)02 -

○

Sparsity measure of vector :

After each update of and in the NMF algorithm, the encodings and atoms are projected on the space verifying:

13

Results - Sh = 0.602 -

14

Original images and reconstruction - Reconstruction error: 0.0109

Histogram of the encodings - Sparsity metric: 0.650

15

Atom images of the representation

Results - Max-Approximation to dilation02 -

16

Dilation of the original images by a disk of radius 1

Max-approximation to the dilation by a disk of radius 1

03 - Part-Based Representation using Auto-Encoders

17

Auto-encoder loss function, minimized during training:

Shallow Auto-Encoders 03 -

18

ReconstructionInput image Encoder Latent representation

Max-approximation

Decoder

The rows of are the atom images of the learned representation !

“Dilated” Decoder

Enforcing the Sparsity of the Encoding03 -

Regularization of the auto-encoder:

Various choices for the sparsity-regularization function:

19

expected activation of each hidden unit fixed level

Enforcing Non-Negativity of the Atoms of the Dictionary03 -

Two common approaches:○

○●●

20

Stronger decay of the negative weights

Results - Reconstructions03 -

21

Original images

p=0.05, beta=0.001

p=0.01, beta=0.005

No Constraint

p=0.2, beta=0.001

p=0.1, beta=0.01

Results - Encodings03 -

22

Original images

p=0.05, beta=0.001

p=0.01, beta=0.005

No Constraint

p=0.2, beta=0.001

p=0.1, beta=0.01

Results - Atoms03 -

23

p=0.01, beta=0.005

No Constraint

p=0.1, beta=0.01

Results - Max-approximations to dilation03 -

24

Original images

p=0.05, beta=0.001

p=0.01, beta=0.005

No Constraint

p=0.2, beta=0.001

p=0.01, beta=0.01

25

04 - Using a Deeper Architecture

26

An Asymmetric Auto-Encoder04 -

Motivations:○○○○

27

ReconstructionInput image infoGANLatent

representation

Max-approximation

Decoder

“Dilated” Decoder“InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets”, Chen et al. 2016

○○

Results - Reconstructions04 -

28

No Constraint

p=0.05, beta=0.005

p=0.01, beta=0.01

Results - Encodings04 -

29

No Constraint

p=0.05, beta=0.005

p=0.01, beta=0.01

Results - Atoms04 -

30

p=0.01, beta=0.01

No Constraint

p=0.05, beta=0.005

Results - Max-Approximations to dilation04 -

31

No Constraint

p=0.05, beta=0.005

p=0.01, beta=0.01

06 - Conclusion and Future Works

32

Conclusion - Reconstructions06 -

33

NMF with Sparsity Constraint Sh=0.6

Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001

Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005

Original Images

Conclusion - Encodings06 -

34

NMF with Sparsity Constraint Sh=0.6

Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001

Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005 -

Conclusion - Atoms06 -

35

NMF with Sparsity Constraint Sh=0.6

Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005

Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001

Conclusion - Max-Approximations to dilation06 -

36

NMF with Sparsity Constraint Sh=0.6

Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001

Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005 -

Dilation of Original Images

Conclusion and possible improvements06 -

○

○

○

37

05 - Multi-Scales Morphological Decompositions

38

Additive Morphological Decomposition05 -

39

One of the considered Morphological Decomposition:○

○

○

○○

40

Positive Additive Decomposition Using Openings by Reconstruction05 -

41

05 -

42

Results05 -

43

○

○

○atom images,

○

Representation (latent features)

Representation Learning and Part-Based representation

44

Image