Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Graph Signal Processing (GSP)
(or how I started seeing graphs everywhere)
Antonio Ortega
Signal and Image Processing InstituteDepartment of Electrical Engineering
University of Southern CaliforniaLos Angeles, California
Apr 14, 2017
Outline
Graphs Everywhere
GSP basics
Contributions
Discussion
Graphs everywhere ...
Graphs provide a flexible model to represent many datasets:
I Examples in Euclidean domains
(a) (b)
11
.5
(c)
(a) Computer graphics1 (b) Wireless sensor networks 2 (c) image - graphs
... and then some
I Examples in non-Euclidean settings
(a)
Combined ARQ-Queue
0
1
2
3
0 1 2 3
ARQ
Que
ue(b)
(a) Social Networks 3, (b) Finite State Machines(FSM)
Graph Signal Processing
I Given a graph (fixed or learned from data)
I and given signals on the graph (set of scalars associated to vertices)
I define frequency, sampling, transforms, etc
I in order to solve problems such as compression, denoising, interpolation,etc
I Overview papers:[Shuman, Narang, Frossard, Ortega, Vandergheysnt, SPM’2013][Sandryhaila and Moura 2013]
Examples
11
.5
I Sensor network
I Relative positions of sensors(kNN), temperature
I Does temperature varysmoothly?
I Social network
I Friendship relationship, ageI Are friends of similar age?
I Images
I Pixel positions and similarity,pixel values
I Discontinuities and smoothness
Outline
Graphs Everywhere
GSP basics
Contributions
Discussion
Graphs 101
I Graph : vertices (nodes) connected via some links (edges)
I Graph Signal: set of scalar/vector values defined on the vertices.
1
2
3
4
5
6
7
8
9
10
1112
13
14
15
16
17
18
19
20
21 22 23
24 25 26
27
Graph-signal
Graph G = (V,E ,w)
Vertex Set V = {v1, v2, ...}Edge Set E = {(v1, v2), (v1, v3), ...}Weighted edges w , sets of weights aij
Graph Signal x = {x1, x2, ...}Neighborhood, h-hopNh(i) = {j ∈ V : hop dist(i , j) ≤ h}
Multiple algebraic representations
1
2
3
4
5
6
7
8
9
10
1112
13
14
15
16
17
18
19
20
21 22 23
24 25 26
27
I Graph G = (V,E ,w).
I Adjacency A, aij , aji = weights oflinks between i and j (could bedifferent if graph is directed.)
I Degree D = diag{di}, in case ofundirected graph.
I Various algebraic representations
I Normalized adjacency 1|λmax |A
I Laplacian matrix L = D− A.
I Symmetric normalizedLaplacian L = D−1/2LD−1/2
I Discussion:
1. Undirected graphs easier to work with2. Some applications require directed graphs3. Graphs with self loops are useful
Graph Fourier Transform (GFT)
I Different results/insights for different choices of operator
I Laplacian L = D− A = UΛU′
I Eigenvectors of L : U = {uk}k=1:N
I Eigenvalues of L : diag{Λ} = λ1 ≤ λ2 ≤ ... ≤ λN
I Eigen-pair system {(λk , uk)} provides Fourier-like interpretation —Graph Fourier Transform (GFT)
Eigenvectors of graph Laplacian
(a) λ = 0.00 (b) λ = 0.04 (c) λ = 0.20
(d) λ = 0.40 (e) λ = 1.20 (f) λ = 1.49
I Basic idea: increased variation on the graph, e.g., ftLf, as frequencyincreases
Outline
Graphs Everywhere
GSP basics
Contributions
Discussion
Graph Filterbank Designs
I Formulation of critically sampled graph filterbank design problem
I Design filters using spectral techniques [Hammond et al. 2009].
I Orthogonal (not compactly supported) [Narang and O. TSP’12]
I Bi-Orthogonal (compactly supported) [Narang and O. TSP’13]
analysis side synthesis side
filter downsample upsample filter
- -
Example
−1 0 1
−0.1 0 0.1
−1 0 1
−0.05 0 0.05
LL Channel LH Channel
HL Channel HH Channel
Reconstructed graph-signals for each channel.
Graph Sampling?
I Measure a few nodes to estimate information throughout the graph
I Reconstruct signal in whole graph
Questions:
I What properties enable recovery? Need to define frequency
I How to sample? No obvious regular sampling
I How to reconstruct? Filtering is needed
Graph Sampling?
I Measure a few nodes to estimate information throughout the graph
I Reconstruct signal in whole graph
Questions:
I What properties enable recovery? Need to define frequency
I How to sample? No obvious regular sampling
I How to reconstruct? Filtering is needed
Results: Real Datasets
1 2 3 4 5 6 7 8 9 100.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Percentage of labeled data
Accu
racy
RandomLLGC BoundMETISLLRProposed
I USPS: handwritten digits
I xi = 16× 16 image
I number of classes = 10
I K -NN graph with K = 10
I wij = exp
(−‖xi−xj‖
2
2σ2
)
2 3 4 5 6 7 8 9 100.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Percentage of labeled dataA
ccu
racy
I ISOLET: spoken letters
I xi ∈ R617 speech features.
I number of classes = 26
I K -NN graph with K = 10
I wij = exp
(−‖xi−xj‖
2
2σ2
)
1 2 3 4 5 6 7 8 9 100.1
0.2
0.3
0.4
0.5
0.6
0.7
Percentage of labeled data
Accu
racy
I Newsgroups: documents
I xi ∈ R3000 tf-idf of words
I number of classes = 10
I K -NN graph with K = 10
I wij =x>i xj‖xi‖‖xj‖
Assumptions
I Given data matrix X = [x1, · · · , xN ] ∈ Rn×N .
I The k-th row of X (k-th variable) is attached to k-th vertex of the graph.
I Each xi is a graph signal in an unknown graph.
I Sensor network: each vertex is a sensor, signal is a measurement/timeseries
I Data model: attractive Gaussian Markov Random Field (a-GMRF) ⇔Gaussian with a Generalized Laplacian (GL) for precision matrix.
I Q = P + L with P diagonal (self loop matrix) and L a combinatorialLaplacian.
I Q = (qij) and qij ≤ 0 for all i 6= j
I Graph estimation: Max. Likelihood under aGMRF model.
Use block coordinate descent to solve
minQ is GL
− log det(Q) + tr(QS),
with S = 1N
XXT .
Experiment: Texture graphWe consider wood textures from Brodatz dataset, with 0 and with 60 degreerotation. For each texture of Brodatz dataset, take 8× 8 blocks, compute theircovariance matrix S and solve GL estimation rho = 0.
2.7539e−05 1
wood060
0.053489 1
Texture graphs using our GL estimation (only off diagonal elements of Q). The graphshave |E | = 130 and |E | = 117 edges respectively
.
Outline
Graphs Everywhere
GSP basics
Contributions
Discussion
Discussion
I Research Question (Twitter Edition):
I GSP 4 CPS + IOT @USC-CCI ?
I Systems are large and irregular in space/time
I Sampling and interpolation (sensors)I Variable topology communication networksI Data reduction to reduce complexity; multiresolution representationsI Control and optimization of large scale systems
Discussion
I Research Question (Twitter Edition):
I GSP 4 CPS + IOT @USC-CCI ?
I Systems are large and irregular in space/time
I Sampling and interpolation (sensors)I Variable topology communication networksI Data reduction to reduce complexity; multiresolution representationsI Control and optimization of large scale systems
Discussion
I Research Question (Twitter Edition):
I GSP 4 CPS + IOT @USC-CCI ?
I Systems are large and irregular in space/time
I Sampling and interpolation (sensors)
I Variable topology communication networksI Data reduction to reduce complexity; multiresolution representationsI Control and optimization of large scale systems
Discussion
I Research Question (Twitter Edition):
I GSP 4 CPS + IOT @USC-CCI ?
I Systems are large and irregular in space/time
I Sampling and interpolation (sensors)I Variable topology communication networks
I Data reduction to reduce complexity; multiresolution representationsI Control and optimization of large scale systems
Discussion
I Research Question (Twitter Edition):
I GSP 4 CPS + IOT @USC-CCI ?
I Systems are large and irregular in space/time
I Sampling and interpolation (sensors)I Variable topology communication networksI Data reduction to reduce complexity; multiresolution representations
I Control and optimization of large scale systems
Discussion
I Research Question (Twitter Edition):
I GSP 4 CPS + IOT @USC-CCI ?
I Systems are large and irregular in space/time
I Sampling and interpolation (sensors)I Variable topology communication networksI Data reduction to reduce complexity; multiresolution representationsI Control and optimization of large scale systems
References I
D. Shuman, S. K. Narang, P. Frossard, A. Ortega, P. Vandergheynst,
“Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Data Domains”Signal Processing Magazine, May 2013
A. Sandryhaila and J. Moura,
“Discrete Signal Processing on Graphs”IEEE Transactions on Signal Processing, 2013