Rate-Distortion Optimization for Geometry Compression of Triangular Meshes Frédéric Payan I3S laboratory - CReATIVe Research Group Université de Nice -

Rate-Distortion Optimization for Geometry Rate-Distortion Optimization for Geometry Compression of Triangular MeshesCompression of Triangular Meshes

Frédéric PayanFrédéric Payan

I3S laboratory - CReATIVe Research GroupUniversité de Nice - Sophia Antipolis

Sophia Antipolis - FRANCE

PhD Thesis

Supervisor : Marc Antonini

MotivationsMotivations

Goal : Goal :

propose an efficient compression algorithm for highly propose an efficient compression algorithm for highly detailed triangular meshesdetailed triangular meshes

Objectives : Objectives : HighHigh compression ratio compression ratio Rate-Quality OptimizationRate-Quality Optimization Multiresolution Multiresolution approachapproach FastFast algorithm algorithm

SummarySummary

BackgroundBackground

Distortion criterion for multiresolution meshesDistortion criterion for multiresolution meshes

Optimization of the Rate-Distorsion trade-offOptimization of the Rate-Distorsion trade-off

Experimental resultsExperimental results

Conclusions and perpectivesConclusions and perpectives

SummarySummary


Triangular MeshesTriangular Meshes

RemeshingRemeshing

Multiresolution analysis Multiresolution analysis

CompressionCompression

Bit allocationBit allocation

I. Background


3D modeling3D modeling

Applications :Applications : Medecine Medecine CADCAD Map modelingMap modeling GamesGames CinemaCinema Etc.Etc.

I. Background

Irregular meshesIrregular meshesvalence different of 6valence different of 6

=> 2 informations :=> 2 informations : GeometryGeometry (vertices) (vertices) ConnectivityConnectivity (edges) (edges)

5 neighbors

9 neighbors

4 neighbors

I. Background

ExamplesExamples

99,732 triangles=> + 1.1 Mb

More than 380 millions of More than 380 millions of triangles triangles =>=> several Gigabytes several Gigabytes ((Michelangelo Project, Michelangelo Project, 1999)1999)

40,000 triangles=> + 0.45 Mb

MultiresolutionMultiresolution AnalysisAnalysis : :

Without connectivity modification => Without connectivity modification => wavelet wavelet transform for irregular meshes (transform for irregular meshes (S.Valette et R.Prost, S.Valette et R.Prost, 20042004))

A mesh is only one instance of the surface A mesh is only one instance of the surface geometry geometry => => RemeshingRemeshing

goal : goal : regular regular and and uniformuniform geometry sampling geometry sampling

=> Considered solution :=> Considered solution :Semi-regular remeshingSemi-regular remeshing

Irregular meshes (2)Irregular meshes (2)

I. Background

SummarySummary



RemeshingRemeshing




I. Background

Semi-regular remeshingSemi-regular remeshing

Original meshCoarse meshSubdivised mesh (1)Finest semi-regular version

Simplification

Irregularmesh

Coarsemesh

Subdivision

Semi-regularmesh

I. Background

Semi-regular remesherSemi-regular remesher

MAPSMAPS ((A. Lee et al. A. Lee et al. , 1998, 1998) ) Coarse mesh (geometry+connectivity)Coarse mesh (geometry+connectivity) N sets of 3D details (geometry) => N sets of 3D details (geometry) => 3 floating numbers3 floating numbers

Normal MeshesNormal Meshes ((I. Guskov et al.I. Guskov et al., 2000, 2000)) Coarse mesh (geometry+connectivity)Coarse mesh (geometry+connectivity) N’ sets of 3D details (geometry) => N’ sets of 3D details (geometry) => 1 floating number1 floating number

I. Background

Normal MeshesNormal Meshes

Known direction: Known direction: normal at the surfacenormal at the surface

0v 1v

2v 1M

01,d

3v

4v02 ,d

12 ,d

2M

0M

Surface to remeshSurface to remesh

=>=> More compact representationMore compact representation

I. Background

SummarySummary



RemeshingRemeshing




I. Background

Multiresolution analysisMultiresolution analysis

MultiresolutionMultiresolution RepresentationRepresentation:: Low frequencyLow frequency (LF) mesh (LF) mesh

(geometry + topology)(geometry + topology)

N sets of wavelet coefficients (N sets of wavelet coefficients (3D vectors3D vectors) ) (geometry)(geometry)

…

Details Details Details Details

I. Background

SummarySummary



RemeshingRemeshing




I. Background


Objective : Objective : reduce the information quantity reduce the information quantity useful for useful for

representing numerical datarepresenting numerical data

2 approachs 2 approachs : : Lossy or lossless compressionLossy or lossless compression

High compression ratiiHigh compression ratii=> Lossy compression=> Lossy compression

I. Background

Compression schemeCompression scheme

Q Entropy Coding

Transform 1010…Remeshing

Bit Allocatio

n

Target bitrate

or distortion

Preprocessing

Semi-regular Wavelet coefficients

Optimize the Rate-Distortion (RD)

tradeoff

I. Background

SummarySummary



RemeshingRemeshing




I. Background

Bit allocation: goalBit allocation: goal

Optimization of the tradeoff between bitstream size Optimization of the tradeoff between bitstream size and reconstruction quality:and reconstruction quality:

minimize D(R)minimize D(R)

oror minimize R(D)minimize R(D)

R

D

I. Background

Bit allocation and meshesBit allocation and meshes

Related Works (geometry compression):Related Works (geometry compression):

Zerotree Zerotree codingcoding PGCPGC : :

Progressive Geometry Compression Progressive Geometry Compression (A. (A. Khodakovsky et al.,Khodakovsky et al., 2OOO) 2OOO) NMCNMC : :

Normal Mesh Compression Normal Mesh Compression (( A. A. Khodakovsky et I. GuskovKhodakovsky et I. Guskov, , 2002). 2002).

=> => Stop coding when bitstream given size is reached.Stop coding when bitstream given size is reached.

Estimation-quantization (EQ) Estimation-quantization (EQ) codingcoding MSECMSEC : :

Geometry Compression of Normal Meshes Using Rate-Distortion Geometry Compression of Normal Meshes Using Rate-Distortion AlgorithmsAlgorithms (S. (S. Lavu et al.Lavu et al., 2003), 2003)

=> => Local Local RD optimization.RD optimization.

I. Background

Proposed bit allocationProposed bit allocation

1.1. Low computational complexityLow computational complexity

2.2. Improve the quantization processImprove the quantization process

3.3. Maximize the quality of the reconstructed meshMaximize the quality of the reconstructed meshaccording to a given target bitrateaccording to a given target bitrate

=> Which distortion criterion for evaluating the losses?

I. Background

SummarySummary






Coding/DecodingCoding/Decoding

Q

Entropy Decoding

Transform

Bit Allocatio

n

1010…

Target bitrate or

distortion

Remeshing

Preprocessing

Q*Inverse

transform

Entropy coding

Quantized semi-regular

Semi-regular

II. Distortion criterion for multiresolution meshes

Considered distorsion criterionConsidered distorsion criterionMSE due to quantization of the semi-regular meshMSE due to quantization of the semi-regular mesh

1#

0

2

2ˆ

#

1 SR

jjjSRT vv

SRMSED

semi-regular verticessemi-regular vertices

quantized semi-regular verticesquantized semi-regular vertices

Number of verticesNumber of vertices

Wavelet => ? iSR MSEfunctionMSE

MSE for one subbandMSE for one subband


Related worksRelated worksK.ParkK.Park and and R.HaddadR.Haddad (1995) (1995) M-channel schemeM-channel scheme quantization model : “noise plus gain”quantization model : “noise plus gain”

B.UsevitchB.Usevitch (1996) (1996) quantization model : “additive noise”quantization model : “additive noise” N decomposition levelsN decomposition levels Sampled on square gridsSampled on square grids

Filter bankFilter bank

Problem : - non adapted for lifting scheme ! - usable for any sampling grid ?


Lifting scheme for meshesLifting scheme for meshes

3 3 prédiction operators Pprédiction operators P

=> wavelet coefficients=> wavelet coefficients

3 3 update operators U update operators U

=> LF mesh=> LF mesh

Triangular grid => Triangular grid => 4 channels4 channels


Triangulaire samplingTriangulaire sampling

1 triangular grid => 4 cosets1 triangular grid => 4 cosets

0 0 0

00

0

LF subband (0)

n1

n2

HF subband 1

1 1

1

HF subband 22

2

2

HF subband 3

3

3

3


4-channel lifting scheme: analysis4-channel lifting scheme: analysis

LF

HF 1

HF 2

HF 3

-P1

+-P2

+-P3

+

U1

+

U2

+

U3

+

split

Semi-regular mesh


4-channel lifting scheme: synthesis4-channel lifting scheme: synthesis

P

+

P

+

P

+

Merge

LF

HF 1

HF 2

HF 3

Semi-regular mesh

-U

+

-U

+

-U

+

=> Derivation of the MSE of the quantized meshaccording to the quantization error of each 4 subband


Input signal :Input signal :

Quantization error model : Quantization error model : « additive noise »« additive noise »

S is one realization of a stationar and ergodic S is one realization of a stationar and ergodic random process => random process => deterministic quantitydeterministic quantity

=> MSE of the input signal=> MSE of the input signal

Proposed MethodProposed Method

KRss kk /)(

0RSREQM SR #

1


ss ˆ

Proposed MethodProposed Method: : HypothesisHypothesis

Uncorrelated error Uncorrelated error in each subbandin each subband

Subband errorsSubband errors mutually uncorrelated mutually uncorrelated

Synthesis filter energyQuantization error

energy

1

0#

1 M

igSR ii

RRSR

MSE 00


Proposed MethodProposed Method: principle: principle

Synthesis filter energySynthesis filter energy

Polyphase components Polyphase components of the filtersof the filters Cauchy theoremCauchy theorem

Quantization error energyQuantization error energy

Uncorrelated error Uncorrelated error in each subbandin each subband

0ir

0igr


Proposed MethodProposed Method:: solution solution

withwith

1

0

M

jiiSR MSEwMSE

1

0

2,#

# M

jji

ii

d

GSR

SRw

Zk

k

For 1 decomposition level

Polyphase component

MSE of the subband i


Weights relative to the non-orthogonal filters

polyphase representationpolyphase representationLifting scheme: Lifting scheme:

=> Polyphase components => Polyphase components depend on only the depend on only the prediction and update prediction and update opératorsopérators

1121111

1222122

1121111

121

1

1

1

1

MMMMM

M

M

M

pupupuu

pupupuu

pupupuu

ppp

G

New formulation : => can be applied easily to lifting scheme


1,12,11,10,1

1,22,21,20,2

1,12,11,10,1

1,02,01,00,0

MMMMM

M

M

M

GGGG

GGGG

GGGG

GGGG

G

Proposed MethodProposed Method : : solution solution

avecavec

1

0

2,

M

jjii

d

GwZk

k

For N decomposition levels

1

0

1

1,,0,10,1

N

i

M

lliliNN MSEWMSEWMSE

lili

li wwSR

SRW 0

,, #

#

etet


OutlineOutline

This foThis formulation can be applied to lifting schemermulation can be applied to lifting scheme

Global formulation of the weights for any :Global formulation of the weights for any :

Grid and related subsamplingGrid and related subsampling

number of channels M number of channels M

Number of decomposition levels NNumber of decomposition levels N



Experimental ResultsExperimental Results

=> PSNR Gain : up to 3.5 dB=> PSNR Gain : up to 3.5 dB


Visual impactVisual impact

OriginalWithout the weights With the weights

Coding/DecodingCoding/Decoding

Q

Entropy Decoding

Transform

Bit Allocation

1010…

Target bitrate or

distortion

Remeshing

Preprocessing

Q*Inverse

transform

Entropy coding

Quantized semi-regular

Semi-regular


MSE and irregular meshMSE and irregular mesh

Quality of the reconstructed mesh :Quality of the reconstructed mesh :

Reference : Reference : irregular meshirregular mesh

Used metric:Used metric:

geometrical distance between two surfaces:geometrical distance between two surfaces:

the «surface-to-surfacethe «surface-to-surface distance distance ((s2ss2s) ») »


=> Is the MSE suitable to control the quality?

Forward distance:Forward distance:

distance between one point and one surface:distance between one point and one surface:

Quality of the reconstructed meshQuality of the reconstructed mesh

Input mesh Input mesh (irregular)(irregular)

Quantized meshQuantized mesh

(semi-regular)(semi-regular)

12

21( ) ( )

p Md M M d p M dM

M

2M'2'')(Proj'min)',( ppppMpd

Mp

SRqIRdIRSRqdIRSRqss ,,,max,2


Simplifying approximationsSimplifying approximations

Normal meshesNormal meshes: :

=> => infinitesimal remeshing errorinfinitesimal remeshing error

=>=> uniform and regularuniform and regular geometry sampling geometry sampling

Highly detailed meshes:Highly detailed meshes:

=> densely sampled geometry=> densely sampled geometry

1#

0

2,ˆ#

1,

SR

jj SRvd

SRIRSRqdss

Relation with the quantization error?


Hypothesis: asymptotical caseHypothesis: asymptotical casePreservation of the LF subbandsPreservation of the LF subbands

=>=> normal orientations slightly modified normal orientations slightly modified=> errors lie in the normal direction (=> errors lie in the normal direction (normal normal

meshesmeshes) )


n

0v

1v

2v

n’

0v̂

1v̂

2v̂θ

ε(v2

)

nn’

0v

0v̂1v

1v̂

2v̂

2v

θ

ε(v2

)

=>=> 2

ˆ,ˆ jjj vvSRvd

1#

0

2

2ˆ

#

1,

SR

jjjSS vv

SRIRSRqd

SREQM

=> MSE : suitable criterion to controlthe quality of the reconstructed mesh

Proposed heuristicProposed heuristic

1#

0

2,ˆ#

1,2

SR

jj SRvd

SRIRSRqssApproximating formulation:

2

ˆ,ˆ jjj vvSRvd

Asymptotical case+ normal meshes


SummarySummary






Optimization of the Rate-Distorsion Optimization of the Rate-Distorsion trade-offtrade-off

Objective : Objective :

find the quantization steps that find the quantization steps that maximize the quality of maximize the quality of the reconstructed meshthe reconstructed mesh

Scalar quantization Scalar quantization (less complex than VQ)(less complex than VQ)

3D Coefficients => 3D Coefficients => data structuring?data structuring?

targetconstraintwith

minimize

RR

MSE

T

SR

III.Optimization of the Rate-Distorsion trade-offIII.Optimization of the Rate-Distorsion trade-off

NormalNormal at the surface: at the surface: z-axis of the local framez-axis of the local frame

=> Coefficient : => Coefficient : Tangential Tangential components (components (x x andand y- y-coordinatescoordinates)) Normal Normal components (components (z-z-coordinatescoordinates))

Local framesLocal frames

zz

x

Global frame

xx

z

Local frame

z

x


Histogram of the polar angle


Local frame: Local frame:

0° 90° 180°

=> => Most of coefficients have only normal Most of coefficients have only normal componentscomponents

θ

xy

z

=> Components treated separately (2 scalar subbands)

MSE of one subband MSE of one subband ii

2,1,, iSRiSRJj

jii MSEMSEMSEMSEi

MSE relative to the tangential components

MSE relative to the normal components


How solving the problem?How solving the problem?

Find the Find the quantization steps and lambdaquantization steps and lambda that minimize the following lagrangian criterion:that minimize the following lagrangian criterion:

Method:Method:=> => first order conditionsfirst order conditions

cible,

0,,,,

0, RqRaqMSEWqJ ji

N

i Jjjiji

JjjijiSR

N

iiji

ii

Distortion Constraint relative to the bitrate


SolutionSolution Need to solveNeed to solve

(2N + 4) equations with (2N + 4) unknowns(2N + 4) equations with (2N + 4) unknowns

target0

,,,

,

,,

,,

RqRa

W

a

qR

qMSE

N

i Jjjijiji

i

ji

jiji

jijiSR

i


jiq ,

Tji Rq ,

PDF of the component sets:Generalized Gaussian Distribution (GGD)

=> model-based algorithm (C. Parisot, 2003)

Model-based algorithmModel-based algorithm

compute the variance and compute the variance and αα for each subband for each subband

compute the bitratescompute the bitratesfor each subbandfor each subbandλλ

Target bitrateTarget bitratereached?reached?new new λλ

compute the quantizationcompute the quantizationstep of each subbandstep of each subband

jiR ,

jiq ,

Complexity : 12 operations / semi-regularExample : 0.4 second (PIII 512 Mb Ram)

=> Fast process.


Look-up tables

SummarySummary






Compression schemeCompression scheme

SQ 3D-CbAC*

UnliftedButterflyNormal

meshes

BitAllocatio

n

Target BitratePreprocessing

1010…Connectivity coding*

(coarse mesh connectivity)MPX

IV. Experimental resultsExperimental results

* Touma-Gotsman coder* Touma-Gotsman coder

* Context-based Bitplane Arithmetic * Context-based Bitplane Arithmetic Coder (EBCOT-like)Coder (EBCOT-like)

Visual resultsVisual results

Input mesh(irregular)

CR = 832.2 bits/iv

CR = 226 0.82 bits/iv

CR = 9000.2 bits/iv

Compression ratio:

sizebitstream

IRTrIRIRCR

#log3#323# 2


ComparisonComparison

Quality criterion : Quality criterion :

State-of-the-art methods:State-of-the-art methods: NMCNMC ( (Normal meshesNormal meshes + Butterfly NL + zerotree) + Butterfly NL + zerotree) EQMCEQMC ( (Normal meshesNormal meshes + Butterfly NL + EQ) + Butterfly NL + EQ) PGCPGC (MAPS + Loop) (MAPS + Loop)

s2s between the irregular input s2s between the irregular input mesh and the quantized semi-mesh and the quantized semi-regular oneregular one

Bounding box Bounding box diagonaldiagonal

SRqIRss

bbPSNR

,2log20 10


PSNR-bitrate curve: PSNR-bitrate curve: RabbitRabbit


PSNR-bitrate curve: PSNR-bitrate curve: FelineFeline

=> PSNR Gain: up to 7.5 dB=> PSNR Gain: up to 7.5 dB


PSNR-bitrate curve: PSNR-bitrate curve: HorseHorse


Geometrical comparisonGeometrical comparison

NMC NMC (62.86 dB)(62.86 dB)

Proposed algorithmProposed algorithm ( (65.35 dB65.35 dB))

Bitrate = 0.71 bits/iv


SummarySummary






ConclusionsConclusions New shape compression method:New shape compression method:

Contributions :Contributions :1.1. Weighted MSE : suitable distortion criterionWeighted MSE : suitable distortion criterion

2.2. Original formulation of the weights Original formulation of the weights (suitable in case of lifting scheme)(suitable in case of lifting scheme)

3.3. Bit alllocation of low computational complexity that optimizes Bit alllocation of low computational complexity that optimizes the the quality of a quantized mesh.quality of a quantized mesh.

4.4. An original Context-based Bitplane Arithmetic CoderAn original Context-based Bitplane Arithmetic Coder

V. Conclusions and perspectives

=> Better results than=> Better results thanthe state-of-the-art methods. the state-of-the-art methods.

PerspectivesPerspectives

Take into account some Take into account some visual properties the visual properties the human eye appreciates human eye appreciates (local curvature, volume, (local curvature, volume, smoothness…) smoothness…)

Reference : Z.Reference : Z.Karni and C.GotsmanKarni and C.Gotsman, 2000, 2000

Algorithm for huge meshes: « on the flow » Algorithm for huge meshes: « on the flow » compressioncompression

Reference : Reference : A. Elkefi et al.A. Elkefi et al., 2004, 2004

V. Conclusions and perspectives

EndEnd

Documents

Rate-Distortion Optimization for Geometry Compression of Triangular Meshes Frédéric Payan I3S laboratory - CReATIVe Research Group Université de Nice -