Encoding Stereo Images

Christopher Li, Idoia Ochoa and Nima Soltani

Outline

• System overview• Detailed encoder description• Demonstration• Results• Extensions• Conclusions

System Overview (Encoder)

R

L DWT Quant Arith Enc

DWT Quant

Motion Estimation

DCT Re-order

Arith Enc

Arith Enc

Huff Enc

residuals

shift vectors

use ME

Left Image

• Daubechies-4 wavelet decomposition – 5 levels for luminance, 4 for chrominance

• Uniform quantization with adaptive levels– Each component meets its own fraction of MSE

• Arithmetic coding on the quantized residuals– Frequency tables are sent for each arithmetic

coder

Left Quantization

• Decomposed PSNR constraint

• Allocated fractions of MSE to each color component• Met PSNR constraints by finding maximum uniform

quantization levels that meet assigned MSEs

Left QuantizationMotion Estimation Enable Signal

• Heuristically choose differential vs. separate encoding of right image

Quantize with

Calculate MSE ¿𝑀𝑆𝐸𝑚𝑎𝑥?

Y wavelet coeffs

Encode differentially

Encode separately

Yes

No

Right ImageMotion Estimation Block

• Partition into 30x30 blocks• Find shift vectors that minimize the MSE• Search an area from [-64,64] in the direction

and [-6,6] in the direction for minimum distortion

Right ImageResidual coding

• Impose residuals of Cb and Cr to be 0– Use remaining fraction of MSE for Y component

• Compute DCT of block– Reshape using zig-zag ordering– Replace remaining zeros in block with end of block

character• Perform arithmetic coding

Right ImageShift vector coding

• Offline– Find joint statistics of the shift vectors over the

training set– Construct Huffman table

• During run-time, encode shift vectors using this Huffman table

Right ImageSeparately coded

• Same method as left image– D4 wavelet, with 5 levels for Y, 4 for Cb, Cr– Uniform quantization with variable step– Arithmetic coding with frequencies sent

Writing to File

• Unique quantization values encoded in header bits• Arithmetic coders– Encode frequencies, output length of sequence and

sequence itself• Huffman encoders – Length of sequence and sequence itself– Tables stored offline

Decoder

• Perform all the steps of the encoder in reverse1. Decode left image using inverse DWT2. Read motion estimation flag for right image• If enabled, decode shift vectors and residuals• Else, decode using inverse DWT

Demonstration

Results

Image Bits/pixel

1 1.7324

2 0.7566

3 0.1807

4 0.7164

5 0.9304

6 1.2390

7 1.8104

Image Bits/pixel

8 1.7188

9 0.6453

10 1.7766

11 0.9341

12 0.5879

13 2.1404

14 2.3837

Block size

Extensions

• Use intra-block coding for right image• Explore using different wavelets• Implement embedded zero trees in C• Explore run-length coding further• Apply uniform deadzone quantizers

Conclusions

• Important trade-off between bits allocated to shift data and residual data

• Arithmetic coding outperforms Huffman• Reshaping the DCT blocks allows us to use

information, such as its size, to our advantage• Uniform quantizer is faster, simpler and has

less overhead than Lloyd-max quantizers• MEX files reduce runtime significantly!

Thank you

Questions?