Signal Reconstruction using Least Absolute Errors

Genesis J. Islas

School of Mathematical and Statistical Sciences, Arizona State University

November 21, 2016

Overview

Inverse Problem Introduction

Regularization ModelsCharacteristics of different normsProblem

Results

Gaussian MatricesFourier MatricesGaussian Noise

2 / 24

Inverse Problem:

Suppose f is an unknown function in Rn. We would like to recover fgiven A ∈ Rm×n and b ∈ Rm that satisfy the relationship

Af + e = b.

Here e ∈ Rm is a vector of errors.

The signal f can be approximated by solving the minimization problem

minf∈Rn

||Af − b||.

3 / 24

Tikhonov Regularization

The characteristics of f determines which models can successfully solvethe problem.

Example:

If f is smooth then Tikhonov regularization can be used

||Af − b||22 + λ||Tf ||2.

A common choice for T is

T =

−1 1. . .

. . .

−1 1

∈ Rn−1×n.

T is the finite difference.

This model penalizes solutions with discontinuities or sharpchanges.

4 / 24

TV Regularization

The characteristics of f determines which models can successfully solvethe problem.

Example:

If f is known to be sparse then TV regularization can be used.

||Af − b||22 + λ||Tf ||1

This model is able to recover functions with discontinuities.

5 / 24

Norms

0-norm: The number of non-zero values

||x||0 = |{k|xk 6= 0}|1-norm:

||x||1 = |x1|+ |x2|+ ...+ |xn|2-norm:

||x||2 = (x21 + x22 + ...+ x2n)12

Ax = b Ax = b

x̂x̂

`1 `2

6 / 24

Problem

Goal: Recover f given A ∈ Rm×n and b ∈ Rm that satisfy therelationship

Af + e = b.

Assume ||e||0 is small. Then the signal only has a few corruptions butthe magnitudes can be large.

Perhaps we would like to solve the minimization problem

minf∈Rn

||Af − b||0.

However, this is not a convex problem and is not easy to solve.

7 / 24

Reconstruction using `1 vs `2

We have seen that there is a relationship between `0 and `1.

Under certain conditions we have the following equivalence

f∗ = arg minf∈Rn

||Af − b||0 ⇐⇒ f∗ = arg minf∈Rn

||Af − b||1.

We are interested in comparing the accuracy of the solutions to thefollowing minimization problem using `1 vs `2:

f∗ = arg minf∈Rn

||Af − b||

8 / 24

Single Case

Let A be an 80× 50 matrix with values drawn from the standardnormal distribution.

The signal, f ∈ R50, has the form

f = 5.27 sin(.352π

50x) + 5.08 cos(.94

2π

50x).

9 / 24

Single Case

Let ||e||0 = 5 with ei ∈ [0, 100].

10 / 24

Single Case

11 / 24

Gaussian Standard Normal Matrix

A ∈ Rm×50 with values drawn from a standard normal distribution

m = 50, 52, ..., 100

The signal is given by

f = A sin(k2π

50x) +B cos(j

2π

50x)

for x = 1, 2, ..., 50

||e||0 = 0, 1, ..., 20

Each case is performed 20 times for `1 and `2.

12 / 24

Gaussian Standard Normal Matrix

Success - 1 (Yellow), Failure - 0 (Blue)

13 / 24

Gaussian Standard Normal Matrix

Relative Error :∣∣∣∣∣∣x−xapprox

x

∣∣∣∣∣∣2

14 / 24

Fourier Matrix

A ∈ Rm×50 is a Fourier Matrix with values determined by

Akj =1√m

exp

(2πi kj

m

)m = 50, 52, ..., 100

The signal stays the same,

f = A sin(k2π

50x) +B cos(j

2π

50x)

||e||0 = 0, 1, ..., 20

The probability of recovery is calculated from 20 iterations.

15 / 24

Fourier Matrix

Success - 1 (Yellow), Failure - 0 (Blue)

16 / 24

Fourier Matrix

17 / 24

Gaussian Noise

Now consider the case where e consists of a few large corruptions aswell as small corruptions everywhere.

18 / 24

Gaussian Noise

A ∈ Rm×50 with m = 50, 52, ..., 100.

The signal stays the same,

f = A sin(k2π

50x) +B cos(j

2π

50x).

Now e = e1 + e2||e1||0 = 0, 1, ..., 20e2 has values drawn from a normal distribution

Each case is performed 20 times for `1 and `2.

19 / 24

Gaussian Noise

20 / 24

Gaussian Noise

Residual : ||x− xapprox||2

21 / 24

Gaussian Noise

22 / 24

Acknowledgments

Dr. Rodrigo Platte

Dr. Toby Sanders

Research Training Group Program

School of Mathematical and Statistical Sciences, ASU

23 / 24

Thank You.

Questions?

24 / 24