# Oral PhD Exam Alon Kipnis - Stanford University kipnisal/Slides/ آ  /32 Analog-to-Digital

• View
0

0

Embed Size (px)

### Text of Oral PhD Exam Alon Kipnis - Stanford University kipnisal/Slides/ آ  /32 Analog-to-Digital

• /32

Analog-to-Digital Compression Oral PhD Exam

Alon Kipnis

Fundamental performance limits of

1

• /32

Outline

Motivation — Factors affecting analog-to-digital conversion

Main problem — Combined problem sampling and lossy compression

Corollary — Optimal sampling under compression constraints

Summary — Toward a unified spectral theory of analog signal processing and lossy compression

2

sampling analog

quantization (lossy compression)

010010011001001000 0100101010010001…

digital

• /32

Motivation

010010011001 001000010000 1000100111…

information loss

A/D conversion

Challenges: 1) measure 2) minimize

The analog-to-digital (A/D) conversion problem:

3

• /32

Motivation: measuring information loss d i s t o r t i o n

A/D parameters

Minimal distortion in A/D:

4

analog

010010011001001 000010010101001

digital reconstruction

analog

quantization (lossy compression)sampling

• /32

Background: Lossy Compression

5

sampling quantization (lossy compression) digitalanalog

...

0 . . . 00

0 . . . 01

1 . . . 11

RT

...T0

X(t) R bitrate:

[bits/sec]

The Source Coding Theorem [Shannon ‘48]:

D(R) Shannon’s

distortion-rate function

=

Theoretic lower bound for distortion in A/D Ignores effect of sampling

= optimization over probability distributions

reconstruction analog

Enc

• /32

fs

The Sampling Theorem [Whittaker, Kotelinkov, Shannon]:

d i s t o r t i o n

sampling rate fs

Background: The Sampling Theorem

t X(t)

fs > fNyq , 2fB

t sinc(t)

⇤=

6

fNyq = 2fB

Y [n] = X(t/fs)

Ignores effect of quantization

sampling quantization (lossy compression) digitalanalog

Shannon’s distortion-rate

function D(R)

• /32

Combined sampling and lossy compression

7

sampling optimal lossy compression digitalanalog

D (f s , R) ?=

Minimal distortion under sampling and lossy compression

d i s t o r t i o n

sampling rate fs

unlim ited

bitrate Shannon’s distortion-rate

function D(R)

unlimited sampling rate

fNyq

• /32

Sampling under Bitrate Constraints

Can we attain D(R) by sampling below Nyquist ?

The Sampling Theorem [Whittaker, Kotelinkov, Shannon]

t X(t) Y [n] = X(t/fs)

fs > fNyq , 2fB

t sinc(t)

⇤=

8

“we are not interested in exact transmission when we have a continuous [amplitude] source, but only in transmission to within a given tolerance” [Shannon ’48]

D (f s , R) ?=d

i s t o r t i o n

sampling rate fs

D(R)

fNyq = 2fB

• /32

Motivation — Summary

2) Can we attain D(R) by sampling below Nyquist ?

9

d i s t o r t i o n

sampling rate fs

fNyq = 2fB

D(R)

D (f s , R) ?=

?

1) What is the minimal distortion in sampling and lossy compression?

unlim ited

bitrate

unlimited sampling rate

• /32

Combined Sampling and Source Coding

, inf enc�dec,T

1

T

Z T

0 E ⇣ X(t)� bX(t)

⌘2 dtD(fs, R)

Assumptions:

is zero mean Gaussian stationary with PSD X(t) SX(f) SX(f)

f is unimodal SX(f)

Pointwise uniform sampling Y [n] = X(n/fs) 10

sampling lossy compression reconstruction

Enc Dec fs Y [·]

X(t) bX(t) R

• /32

Special case I: Gaussian Distortion Rate Function

Enc Dec Y [·]

fs > fNyq

fs

) = D(R)D(fs, R)

[Pinsker ’54] D✓(R) =

Z 1

�1 min {SX(f), ✓} df

R✓ = 1

2

Z 1

�1 log

+ [SX(f)/✓] df

SX(f)

f

SX(f)

f

R

D(R)

, WF(SX)

(

(water-filling)

X(t) bX(t)

11

R

• /32

Special case II: MMSE in sub-Nyquist Sampling

Y [·] Enc

Rfs

R ! 1 ) mmse(X|Y )=D(fs, R) mmse(fs)= MMSE in sub-Nyquist sampling [Chan & Donaldson ‘71, Matthews ’00]

X

k2Z SX(f � fsk) eSX|Y (f) =

P k S

2 X(f � fsk)P

k SX(f � fsk)

fs f

SX(f)

SX (f � f s )S

X (f + f s )

bX(t)DecX(t)

12

• /32

Combined Sampling and Source Coding

eSX|Y (f)

f

fs

Distortion due to sampling

Distortion due to bitrate constraint

Theorem*[K., Goldsmith, Eldar, Weissman ‘13]

D(fs, R) mmse(fs)= + WF ⇣ eSX|Y

(*) A. Kipnis, A. J. Goldsmith, T. Weissman and Y. C. Eldar, ‘Rate-distortion function of sub- Nyquist sampled Gaussian sources corrupted by noise’, Allerton 2013 13

Enc Dec fs Y [·] R

X(t) bX(t)

• /32

Example: Uniform PSD

D (f s , R)

f

SX(f)

fB

d i s t o r t i o n

fs

D(R)

D(fs, R) vs fs (R = 1)

m m se(f

s ) fNyq = 2fB

14

• /32

Achievability Scheme Enc Dec

fs Y [·]

D(fs, R) = +

Y [·] estimator E [X(t)|Y [·]]

eX(·)

Enc

Enc

Enc

mmse(fs) WF ⇣ eSX|Y

orthogonalizing transformation

eX�2 [·]

eX�k [·]

eX�1 [·]

...*

15

(*) A. Kipnis, A. J. Goldsmith and Y. C. Eldar, ‘The distortion rate function of cyclostationary Gaussian processes’, (under review) 2016

R X(t) bX(t)

X

i

Ri  R

R1

R2

Rk

• /32

Pre-Sampling Operation Enc Dec

fs H(f)

eSX|Y (f)

fs ✓

without pre-sampling filter

Linear pre-processing can reduce distortion

fs

with pre-sampling filter

eSX|Y (f)

fs

D(R)

d i s t o r t i o n

16

bX(t)X(t) R

H (f) ⌘

1

H (f)

• /32

Optimal pre-Sampling Filter Theorem* [K., Goldsmith, Eldar, Weissman ’14] The optimal pre-sampling filter is (i) anti-aliasing (ii) maximizes passband energy

(*) A. Kipnis, A. J. Goldsmith, Y. C. Eldar and T. Weissman, ‘Distortion-Rate function of sub-Nyquist sampled Gaussian sources’, IEEE Trans. on Information Theory, January, 2016

SX(f)

fs

H?(f) S X (f)

fs

H?(f)

no aliasing

D?(fs, R) = mmse ?(fs) + WF

⇣ |H?|2 SX

17

• /32

Optimal pre-Sampling Filter

(*) A. Kipnis, A. J. Goldsmith, Y. C. Eldar and T. Weissman, ‘Distortion-Rate function of sub-Nyquist sampled Gaussian sources’, IEEE Trans. on Information Theory, January, 2016

Theorem* [K., Goldsmith, Eldar, Weissman ’14] The optimal pre-sampling filter is (i) anti-aliasing (ii) maximizes passband energy

fs

f low-pass is optimal

fsfsfs

f

maximal aliasing-free set is optimal

18

• /32

Why anti-aliasing is optimal ? X1 ⇠ N

� 0,�21

� X2 ⇠ N

� 0,�22

X1 X2+=Y h1 h2

fsfsfs

f

1(�1 > �2)

1(�1 < �2)

h1 h2

=

=

19

h1 h2 argmin ?=Question: {mmse(X1|Y ) +mmse(X2|Y )}

mmse(Xi|Y ) = E (Xi � E[Xi|Y ])2 fs

• /32

Critical Sub-Nyquist Sampling Rate D?(R, fs) vs fs

D(R)

fNyqfR

mmse(fs)

✓ fs

fs

fs

fs

d i s t o r t i o n

(R is fixed)

Sub-Nyquist sampling achieves optimal distortion-rate performance

D?(fs, R) = D(R) fs � fR

SX(f)

20

• /32

Critical Sub-Nyquist Sampling Rate Theorem* [K., Goldsmith, Eldar ’15]

D?(fs, R) = D(R) fs � fR

✓ fR

(+) A. Kipnis, A. J. Goldsmith and Y. C. Eldar, ‘Gaussian distortion-rate function under sub- Nyquist nonuniform sampling’, Allerton 2014

Extends Kotelnikov-Whittaker-Shannon sampling theorem:

Incorporates lossy compression

Valid when input signal is not band limited

Alignment of degrees of freedom

Holds under non-uniform sampling +

(*) A. Kipnis, A. J. Goldsmith and Y. C. Eldar, ‘Sub-Nyquist sampling achieves optimal rate-distortion’, Information Theory Workshop (ITW), 2015

21

• /32 R

1

fR fNyq

Critical Sub-Nyquist Sampling Rate

critical sub-sampling ratio vs R

*

22

• /32

Summary Transforming analog signals to bits involves sampling and lossy compression

Parts of the signal removed due to lossy compression can be removed at the sampling stage

23

Recommended

Documents
Documents
##### Alon hillel tuch
Economy & Finance
Documents
Documents
Documents
Documents
Documents