Generalized Lifting for Sparse Image Representation and Coding

Generalized Lifting for SparseImage Representation and Coding

Julio C. Rolón1,2, Philippe Salembier1

1Technical University of Catalonia (UPC), SpainDept. of Signal Theory and Telecommunications

2National Polytechnic Institute (IPN), MexicoCITEDI Research Center

Outline

Introduction• Motivation• Previous work

Lifting structures• Classical lifting• Generalized lifting

Coding scheme

Experimental results• Energy and entropy• Rate-distortion• Perceptual quality

Conclusions

Outline

Coding scheme

Conclusions

Introduction: Motivation

Limitations of wavelets in image representation• Excellent for decorrelation of impulsional-like events

• Limitations on images because wavelets are not able to fully decorrelate edges

LH subband zoom

Introduction: Motivation

Limitations of wavelets in image representation• Excellent for decorrelation of impulsional-like events

• Limitations on images because wavelets are not able to fully decorrelate edges

LH subband zoom

Need of additional decorrelation for coding applications

Outline

Coding scheme

Conclusions

Introduction: Previous work

Main approaches• Go beyond wavelets trying to overcome their limitations• Edges become 1-dimensional localizable singularities• Increased sparsity of the coefficients

Curvelets[Candès,Donoho 1999; Candès 2006]

Contourlets[Do,Vetterli 2005]

Frequency-domain methodsBandelets

[LePennec,Mallat 2005; Peyré,Mallat 2005]

Adaptive directional lifting[Chappelier et al 2006; Ding et al 2007;

Heijmans et al 2005; Mehrseresht,Taubman 2006; Piella et al 2002,2006; Wang et al 2006;

Zhang et al 2005 ]

Spatial-domain methods

Generalized liftingMay be highly non-linear,

preserving perfect reconstruction

Outline

Coding scheme

Conclusions

Lifting structures: Classical lifting

Approximation signal: x'[n] = x[n] + U(y'[n])

y'[n] = y[n] – P(x[n])Detail signal:

[Sweldens 1996]

LAZY WAVELET TRANSFORM

P(x[n])

y'[n]Details are the prediction error

Lifting structures: Classical lifting

Approximation signal: x'[n] = x[n] + U(y'[n])

[Sweldens 1996]

Invertibility key:+/- operations

Outline

Coding scheme

Conclusions

Lifting structures: Generalized lifting

Approximation signal: x'[n] = U(x[n],y'[n])

y'[n] = P(y[n],x[n – i])|i∈CDetail signal:

[Solé,Salembier 2004]

LAZY WAVELET TRANSFORM

0 1 2 3 4 5 6 7 8 9

Py'[n]

n0 1 2 3 4 5 6 7 8 9

0 1 2 3 4 5 6 7 8 9

Details produced by P operator

y[n] → y'[n]P

Py'[n]

n0 1 2 3 4 5 6 7 8 9

0 1 2 3 4 5 6 7 8 9

Details produced by P operator

y[n] → y'[n]P

Py'[n]

n0 1 2 3 4 5 6 7 8 9

y'x y P(y,x)

Approximation signal: x'[n] = U(x[n],y'[n])

[Solé,Salembier 2004]

y[n] y[n]

s[n] s[n]

x[n] x[n]

BIJECTIVITY OF P, U IS MANDATORYTO ACHIEVE PERFECT RECONSTRUCTION

Previous work has been done in: [Solé,Salembier 2004,2007]

• Lossless coding• 1-dimensional separable context• Applied directly in the lmage domain

Previous work has been done in: [Solé,Salembier 2004,2007]

• Lossless coding• 1-dimensional separable context• Applied directly in the lmage domain

Our main goal is to evaluate the potential of the GL method for lossy coding• Quantization• Comparison between 1-dimensional context and 2-dimensional

non-separable context• Applied in the wavelet domain (as in bandelets)• Assumption: the pdf of the signal is known in the decoder

(as well as in the coder)

Outline

Coding scheme

Conclusions

Coding scheme: The encoder

A 011101…

A 10001…

ORIGINAL

A 011101…

A 10001…

ORIGINAL

2-D WAVELETTRANSFORMED

A 011101…

A 10001…

ORIGINAL

SCALARQUANTIZED

A 011101…

A 10001…

ORIGINAL

LWT 1-D row-wisedownsampling

and 2-D quincunxdownsampling

SCALARQUANTIZED

A 011101…

A 10001…

ORIGINAL

SCALARQUANTIZED

A 011101…

A 10001…

ORIGINAL

SCALARQUANTIZED

GL – MAPPEDDETAIL

Coding scheme: The decoder

DWT-1Q-1

A-1011101…

A-110001…

-4 -3 -2 -1 0 1 2 3 4

Coding scheme: Generalized predict design

-4 -3 -2 -1 0 1 2 3 4

Input signal pdf mapped pdf

Criterion: Energy minimization of the mapped details

][ny ][' ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 ][ny

][' ny

][' ny][ny

ASSUMPTION: Mapping is known by the decoder, i.e. the decoder knows the pdf of the signal

Outline

Coding scheme

Conclusions

1 2 3 4 5

100% reference: DWT DWT + GL, 1-D context DWT + GL, 2-D context

energy entropy% %

DWTscale

Experimental results: Energy and entropy

1 2 3 4 5

Outline

Coding scheme

Conclusions

Experimental results: Rate-distortion

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.530

Bitrate (bpp)

Wavelet + GLBandeletsWavelets

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 130

Bitrate (bpp)

Wavelet + GLBandeletsWavelets

Generalized liftingPotential gain is ~1-3 db, for 2-D context

Experimental results: Rate-distortion

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 124

Bitrate (bpp)

Wavelet + GLWavelets

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 124

Bitrate (bpp)

Wavelet + GLWavelets

Additional comparisons against wavelets

Outline

Coding scheme

Conclusions

Experimental results: Perceptual quality @ 0.2bpp

original wavelets32.03 dB

Bandelets31.24 dB

GL33.33 dB

Experimental results: Perceptual quality @ 0.11bpp

original wavelets29.47 dB

Bandelets28.52 dB

GL30.63 dB

Outline

Coding scheme

Conclusions

The potential of GL has been evaluated for lossy coding

Conclusions

The use of GL approach proposed here follows the ideas of spatial-domain methods(bandelets and adaptive directional lifting)

Conclusions

The GL method differentiates from these methods in thatis a non-linear framework

Conclusions

Our predict design is based on the pdf of the image

Conclusions

Our predict design is based on the pdf of the image

Generalized lifting approach gives promising results atreducing the energy while improving the R-D performance in 1-3 dB

Conclusions: Future work

Remove the assumption that the decoder knows the pdf of the signal• This work: pdf is assumed to be known in coder and decoder• Future: pdf estimation with fixed as well as adaptive strategies

Multiscale GL approach• This work: one scale of GL applied for 1-D and 2-D contexts• Future: multiscale GL in 2-D context

Quantization• This work: scalar pre-quantization• Future: include Q in the GL mapping, post-quantization

Efficient entropy coding• This work: arithmetic encoding• Future: embedded-block coders like EBCOT, SPECK, etc.

Questions

Julio C. Rolón, Philippe SalembierTechnical University of Catalonia (UPC), SpainDept. of Signal Theory and Telecommunications

National Polytechnic Institute (IPN), MexicoCITEDI Research Center

{jcrolon,philippe}@gps.tsc.upc.edu

Thank you !!!

Julio C. Rolón, Philippe SalembierTechnical University of Catalonia (UPC), SpainDept. of Signal Theory and Telecommunications

National Polytechnic Institute (IPN), MexicoCITEDI Research Center

{jcrolon,philippe}@gps.tsc.upc.edu

Generalized Lifting for Sparse Image Representation and Coding

Documents

Generalized Power Method for Sparse Principal Component ...jmlr.csail.mit.edu/papers/volume11/journee10a/journee10a.pdf · Generalized Power Method for Sparse Principal Component

Generalized power method for sparse principal component ... · CORE DISCUSSION PAPER 2008/70 Generalized power method for sparse principal component analysis M. JOURNEE¶ ⁄, Yu

Bayesian image modeling by generalized sparse … · 1 Bayesian image modeling by generalized sparse Markov random fields and loopy belief propagation Kazuyuki Tanaka Graduate School

Generalized Low-rank plus Sparse Tensor Estimation by Fast

GENERALIZED LIFTING MODULES - Hindawi

Sparse Optimization - Lecture: Basic Sparse Optimization ...wotaoyin/summer2013/...Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online

The RGCCA package for Regularized/Sparse Generalized ... · The ﬁrst one relates to the generalization of two-block methods - including Canonical Correlation Analysis (Hotelling

IEEE-CIC ICCC2018 Tutorial-templateiccc2018.ieee-iccc.org/.../Tutorial-6-Generalized-Sparse-and-Low-Ran… · IEEE/CIC ICCC2018 Generalized Sparse and Low-Rank Optimization for Ultra-Dense

JTC Series Lifting Screw Jack,Lifting Screw Drive Spindle,Lifting Jack Gear,Screw Type Lifting Jacks,Jack Screw Assembly for Lifting,Lifting With Acme Jack Screws,Mechanical Lifting

IEEE TRANSACTIONS ON AUTOMATIC CONTROL 1 Sparse … · 2 IEEE TRANSACTIONS ON AUTOMATIC CONTROL Information Criterion (AIC) or Generalized Cross-Validation (GCV). For example, such

Modelling sparse generalized longitudinal observations

Factors of Sparse Polynomials are Sparse

Sparse Generalized Principal Components Analysis with

Dual Extrapolation for Sparse Generalized Linear Models · = 1) and Sparse Logistic regression (f i(t) = log(1 + exp( y it)), = 4). Note that we could use a more complex regularizer

The RGCCA package for Regularized/Sparse Generalized ... · The RGCCA package for Regularized/Sparse Generalized Canonical Correlation Analysis 2017-05-10 Contents 1 Introduction

Generalized Sparse Signal Mixing Model and Application to Noisy Blind Source Separation

The Spectral Transformation Lanczos Method for the ... · OCTOBER 1980, PAGES 1251-1268 The Spectral Transformation Lanczos Method for the Numerical Solution of Large Sparse Generalized

TO FLUTTER ANALYSIS OF THIN LIFTING SURFACES · TO FLUTTER ANALYSIS OF THIN LIFTING SURFACES ... In several places numerical values of generalized masses are given with units of slug-ft2

Understanding the Sparse Vector Technique for Differential ... › pvldb › vol10 › p637-lyu.pdf · were modeled as a generalized private threshold testing algorithm (GPTT), and

A SHIFTED BLOCK LANCZOS ALGORITHM FOR SOLVING SPARSE SYMMETRIC GENERALIZED EIGENPROBLEMS