Toward an uncertainty principle for weighted...

EUSIPCO 2015

Toward an uncertainty principlefor weighted graphs

02/09/2015

Bastien Pasdeloup*, Réda Alami*, Vincent Gripon*, Michael Rabbat**

* name.surname@telecom-bretagne.eu** name.surname@mcgill.ca

Signal processing on graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

1Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Generalization of classical Fourier analysis to more complex domains

Normalized Laplacian:

Signal processing on graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

1Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Generalization of classical Fourier analysis to more complex domains

Normalized Laplacian:

Assumption: normalized signals

Graph Fourier transform

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

2Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Diagonalization of

( real and symmetric)

Graph Fourier transform

Inverse graph Fourier transform

Graph domain Spectral domain

A portage of tools

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

3Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Filtering

Convolution

Translation

Modulation

Dilatation

…

Reference paper: Shuman et. al – The emerging field on signal processing on graphs – 2013

The tool that interests us here

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

4Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Heisenberg principle A signal cannot be both fully localized in thetime and frequency domains

On graphs A signal cannot be both fully localized in thegraph and spectral domains

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Graph spread: – Around a particular node – Use of the geodesic (i.e shortest path) distance

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

x

0 diffusion steps

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

x

1 diffusion steps

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

x

2 diffusion steps

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

x

3 diffusion steps

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

x

k diffusion steps

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

x

Convergence

The uncertainty principle applied to graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

5Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Reference paper: Agaskar & Lu – A spectral graph uncertainty principle – 2013

Spectral spread: – Around the smallest eigenvalue (=0) – This choice is often discussed, but corresponds to the steady state

x

Convergence

Coincides with the observation thatdiffusion concentrates the signal in

the frequency domain, but smoothes itin the graph domain (i.e decreases thegraph Laplacian quadratic form: )

Current limitations of their work

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

6Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

– Currently addresses unweighted graphs only

– Arbitrarily uses the geodesic distance

– Does not give any hint on the choice of the reference node

Current limitations of their work

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

6Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

– Currently addresses unweighted graphs only

– Arbitrarily uses the geodesic distance

– Does not give any hint on the choice of the reference node

Addressed in this paper

A naive extension to weighted graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

7Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Idea #1: Let's do the same with a weighted adjacency matrix!

Geodesic distance is also defined for weighted graphs

The normalized Laplacian is still real and symmetric

A naive extension to weighted graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

7Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Idea #1: Let's do the same with a weighted adjacency matrix!

Geodesic distance is also defined for weighted graphs

The normalized Laplacian is still real and symmetric

This leads to a discontinuity of the graph spreadwith respect to the graph structure

Illustration of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

7Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Illustration of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

7Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Illustration of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

7Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Illustration of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

7Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Illustration of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

7Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Explanation of the discontinuity

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

8Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

?

Source of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

9Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Source of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

9Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Distance function(since we want the spread to

increase as the signal is far away)

Source of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

9Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Distance function(since we want the spread to

increase as the signal is far away)

Recall that this definition of spectral spread is acceptable

because it is minimized for completely diffused signals (that are smooth in the graph domain)

Source of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

9Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Distance function(since we want the spread to

increase as the signal is far away)

Recall that this definition of spectral spread is acceptable

because it is minimized for completely diffused signals (that are smooth in the graph domain)

Laplacian quadratic form:

Source of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

9Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Distance function(since we want the spread to

increase as the signal is far away)

Recall that this definition of spectral spread is acceptable

because it is minimized for completely diffused signals (that are smooth in the graph domain)

Laplacian quadratic form:

To be minimized (i.e localized in the spectral domain),high values in should be associated to very similar nodes

Source of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

9Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Distance function(since we want the spread to

increase as the signal is far away)

Recall that this definition of spectral spread is acceptable

because it is minimized for completely diffused signals (that are smooth in the graph domain)

To be minimized (i.e localized in the spectral domain),high values in should be associated to very similar nodes

Laplacian quadratic form:

Source of the problem

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

9Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Distance function(since we want the spread to

increase as the signal is far away)

Recall that this definition of spectral spread is acceptable

because it is minimized for completely diffused signals (that are smooth in the graph domain)

To be minimized (i.e localized in the spectral domain),high values in should be associated to very similar nodes

Laplacian quadratic form:

From now on, will be regarded as a similarity matrix

A change of distance function

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

10Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

New definition of graph spread:

– Similar to the previous definition

– We need to change the distance function so that we do not use as distances

Proposed requirements on the distance function :

1°)

2°)

3°) is continuous, and if we increase for a single edge , then does not increase

A change of distance function

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

10Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

New definition of graph spread:

– Similar to the previous definition

– We need to change the distance function so that we do not use as distances

Proposed requirements on the distance function :

1°)

2°)

3°) is continuous, and if we increase for a single edge , then does not increase

The spread should be a positive quantity

A change of distance function

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

10Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

New definition of graph spread:

– Similar to the previous definition

– We need to change the distance function so that we do not use as distances

Proposed requirements on the distance function :

1°)

2°)

3°) is continuous, and if we increase for a single edge , then does not increase

To have for and only for a signallocalized on

The spread should be a positive quantity

A change of distance function

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

10Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

New definition of graph spread:

– Similar to the previous definition

– We need to change the distance function so that we do not use as distances

Proposed requirements on the distance function :

1°)

2°)

3°) is continuous, and if we increase for a single edge , then does not increase

Similar graphs shouldhave similar spreads

The spread should be a positive quantity

To have for and only for a signallocalized on

A change of distance function

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

10Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

New definition of graph spread:

– Similar to the previous definition

– We need to change the distance function so that we do not use as distances

Proposed requirements on the distance function :

1°)

2°)

3°) is continuous, and if we increase for a single edge , then does not increase

Similar graphs shouldhave similar spreads

is considered as a matrix of similarities

The spread should be a positive quantity

To have for and only for a signallocalized on

Some compliant functions

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

11Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Inverse similarity matrix:

– Use of the squared geodesic distance with instead of :

– Simple correction of the discontinuity

– Basically, can be replaced by any positive non-increasing function

(eg. Gaussian kernel)

Diffusion distance:

– is a signal fully localized on node

Associated uncertainty curves

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

12Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Mean uncertainty curves for various graphs, for 100 random weights

Associated uncertainty curves

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

12Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Mean uncertainty curves for various graphs, for 100 random weights

softer than

Associated uncertainty curves

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

12Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Mean uncertainty curves for various graphs, for 100 random weights

softer than

Same curves order

Application to semi-localized graphs

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

13Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Conclusion

Introduction & motivation – Uncertainty on graphs – Extension to weighted graphs – Conclusion

14Bastien Pasdeloup – Toward an uncertainty principle for weighted graphs – EUSIPCO 2015 – 02/09/2015

Summary of the paper:

– Extension of the uncertainty principle to weighted graphs

– Highlight of the confusion when using as a distances matrix

– Statement of requirements on the distance functions used in

Future work:

– Study of the impact of the choice for the distance function

– Obtention of the analytical expressions of the curves

– Study of the impact of the choice of the central node (eg. to compare graphs)

– Use the uncertainty principle to guide a graph reconstruction process

EUSIPCO 2015

Toward an uncertainty principlefor weighted graphs

02/09/2015

Bastien Pasdeloup*, Réda Alami*, Vincent Gripon*, Michael Rabbat**

* name.surname@telecom-bretagne.eu** name.surname@mcgill.ca