Embed Size (px)
Citation preview
29.10.2014
1
1/20
Department of Electrical EngineeringUniversity of Isfahan
Introduction to
Compressive
SensingSection I: Sparsity
2
References
Y. C. Eldar and G. Kutyniok, Compressed Sensing: Theory
and Application, Cambridge University Press, 2012.
Fundamental of Image and Video Processing Online course
29.10.2014
2
3
Outlines
4
Introduction
Nyquist theory Vs. Compressive Sensing
DSP Reconstruction filter
Analog Sampling Kernels
DSP
Analog Interpolation
Kernels
29.10.2014
3
5
Introduction
Nyquist theory Vs. Compressive Sensing
DSP Reconstruction filter
Analog Sampling Kernels
DSP
Analog Interpolation
Kernels
Nyquist Theory CS Theory
infinite-length, continuous-time Sig
Sampling Measuring
Simple recovery Special and nonlinear recovery
6
Introduction
Nyquist
sampling:
Compressive
Sampling:
Many beautiful papers covering theory, algorithms, and applications
b A x
b = Ax
b A x
=
=
29.10.2014
4
7
Outlines
8
Review of Vector Spaces
𝒙 =𝟑𝟎𝟒
𝒙 2 = 3 2 + 4 2 = 25 = 5
𝒙 1 = 3 + 4 = 7
𝒙 0 = 2
𝒒𝒖𝒂𝒔𝒊𝒏𝒐𝒓𝒎
29.10.2014
5
9
Review of Vector Spaces
𝒒𝒖𝒂𝒔𝒊𝒏𝒐𝒓𝒎
10
Review of Vector Spaces
29.10.2014
6
11
Outlines
12
Signal Modeling
Low Dimensional Signal Models
Sparse ModelsLow-Rank Matrix Models
Parametric Models
Finite Union of Subspaces
Sparsity…
29.10.2014
7
13
Outlines
14
What is Sparsity?
A vector is said to be sparse if it only has “a few” non-zero components.
The vector can represent a signal, witch my be sparse in its native domain (e.g., image of sky at night) or can be made sparse in another domain (e.g., natural images in the DCT domain)
A sparse vector may originate in numerous applications
29.10.2014
8
15
What is Sparsity?
A vector is said to be sparse if it only has “a few” non-zero components.
The vector can represent a signal, witch my be sparse in its native domain (e.g., image of sky at night) or can be made sparse in another domain (e.g., natural images in the DCT domain)
A sparse vector may originate in numerous applications
=
16
Outlines
29.10.2014
9
17
Applications
Compressive Sensing
Image and Video Processing
Machine learning
Statistics
Genetics
Econometrics
Neuroscience
…
18
Econometrics
sparse
29.10.2014
10
19
Robust Regression
Least Square Method:
20
Robust Regression
Least Square Method:
29.10.2014
11
21
Recommender Systems
Matrix Completion problem
Rank Minimization Problem
Low Rank Matrix Many of Singular Values are zero
22
Image Denoising
DCT Domain Bases Sparse
Smooth
• Image Inpating• Super Resolution Image• Face Rrecognition• Video Surveillance• …
29.10.2014
12
23
Compressive Sensing
24
Outlines
29.10.2014
13
25
Linear Inverse Problems
26
Linear Inverse Problems
29.10.2014
14
27
Linear Inverse Problems
It depends on the application
Regularization Principle: Adding priori knowledge to problem
Sparse
28
Unit Sphere
29.10.2014
15
29
30
It’s not a sparse solution
Derivation of closed form solution
29.10.2014
16
31
Solutions are sparse
32
Solution is sparse
Basis Pursuit Problem
29.10.2014
17
33
Solution is sparse
34
On Convexity
Non-convex functionConvex function
Convex set Non-convex set
29.10.2014
18
35
Convex Optimization Problem
Convex function Convex set
36
A
29.10.2014
19
37
Small solutionClosed form
Sparse solutionNon- convex
Sparse solutionconvex Sparse solution
NP-Hard
Greedy approaches (Matching Pursuit) approximate the solution
Can be solved via convex optimization algorithms
38
Outlines
29.10.2014
20
39
Problem Reformulation
Noise in Observation
Swapping the Constraint and Objective
40
Problem Reformulation
Noise in Observation
Swapping the Constraint and Objective
convex convex
29.10.2014
21
41
Problem Reformulation
Bring constraint to objective
LASSO Problem (least absolute shrinkage and selection operator)
