7/30/2019 Max Lloyd Quantization

1/5

11

MaxMax--LloydLloyd

QuantizerQuantizer

Quantization ConceptQuantization Concept

z L-level Quantization

Minimize errors for this lossy process

What L values to use?

Map what range of continuous values to each of L values?

tmin tmax

z

Uniform partition Maximum errors = ( tmax - tmin ) / 2L = 2V / 2L

over a dynamic range of 2V

Best solution?

X Consider minimizing maximum absolute error (min-max) vs. MSE

X what if the value between [a, b] is more likely than other intervals?

tmin tmax

tk tk+1

(tmaxtmax)/2L

quantization

errorUMCPENEE631Slides(createdbyM.Wu2001/2004)

7/30/2019 Max Lloyd Quantization

2/5

22

Consider the Probability DistributionConsider the Probability Distribution

z Minimize error in a probability sense

MMSE (minimum mean square error)X assign high penalty to large error and to likely occurring values

X squared error gives convenience in math.: differential, etc.

z An optimization problem

What {tk} and {rk} to use?

Necessary conditions: by setting partial differentials to zero

t1 tL+1

p.d.f pu(x)

r1 rL

z Allocate more reconstruct. values in more probable ranges

UMCPENEE631

Slides(createdbyM.Wu2001)

Derivation of MMSE (MaxDerivation of MMSE (Max--Lloyd)Lloyd) QuantizerQuantizer

z MSE of L-level quantizer

z Optimal (MMSE) quantizer necessary conditions

7/30/2019 Max Lloyd Quantization

3/5

33

MMSEMMSE QuantizerQuantizer(Lloyd(Lloyd--Max)Max)z Reconstruction and decision levels need to satisfy

z Solve iteratively

Choose initial values of {tk}(0)

, compute {rk}(0)

Compute new values {tk}(1), and {rk}

(1)

z For large number of quantization levels

Approx. constant pdf within t[tk, tk+1), i.e. p(t) = p(tk) for tk=(tk+tk+1)/2

Reference: S.P. Lloyd: Least Squares Quantization in PCM, IEEE Trans. Info.

Theory, vol.IT-28, March 1982, pp.129-137

UMCPENEE631Slides(createdbyM.Wu2001/2004)

Observations on the MMSEObservations on the MMSE QuanitizerQuanitizer

7/30/2019 Max Lloyd Quantization

4/5

44

Example: Uniform DensityExample: Uniform Density

Uniform quantizer is the optimal mean square quantizer for uniform density.

MMSEMMSE QuantizerQuantizerfor Uniform Distributionfor Uniform Distribution

z Uniform quantizer

Is optimal for uniformly distributed r.v. in MMSE sense

MSE = 2 / 12 with = 2V / L

z SNR of uniform quantizer

SNR = 6n dB with L = 2n

X 1 bit is worth 6 dB.t1 tL+1

2V

1/(2V)

p.d.f. of uniformdistribution

t1 tL+1

UMCPENEE631S

lides(createdbyM.Wu2001/2004)

7/30/2019 Max Lloyd Quantization

5/5

55

z Note: Quantization is a Lossy Stepin SourceCoding

z Quantization is a lossy way to data compression

Lloyd-Max quantizer minimizes MSE distortion with a

number of output bits n (i.e. a give L=2n)

UMCPENEE739MSlides(createdbyM.Wu2002)