Transcript Diapositive 1

Rate-Distortion Optimization for Geometry
Compression of Triangular Meshes
Frédéric Payan
PhD Thesis
Supervisor : Marc Antonini
I3S laboratory - CReATIVe Research Group
Université de Nice - Sophia Antipolis
Sophia Antipolis - FRANCE
Motivations
Goal :
propose an efficient compression algorithm for highly
detailed triangular meshes
Objectives :




High compression ratio
Rate-Quality Optimization
Multiresolution approach
Fast algorithm
Summary
Background
Distortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-off
Experimental results
Conclusions and perpectives
I. Background
Summary
Background
Triangular Meshes
Remeshing
Multiresolution analysis
Compression
Bit allocation
I. Background
Triangular Meshes
3D modeling
Applications :






Medecine
CAD
Map modeling
Games
Cinema
Etc.
I. Background
Irregular meshes
valence different of 6
=> 2 informations :


Geometry (vertices)
Connectivity (edges)
4 neighbors
5 neighbors
9 neighbors
Examples
40,000 triangles
=> + 0.45 Mb
99,732 triangles
=> + 1.1 Mb
More than 380 millions of
triangles => several Gigabytes
(Michelangelo Project, 1999)
I. Background
Irregular meshes (2)
Multiresolution Analysis :

Without connectivity modification => wavelet
transform for irregular meshes (S.Valette et R.Prost,
2004)

A mesh is only one instance of the surface
geometry => Remeshing
=> Considered solution :
goal : regular and uniform geometry sampling
Semi-regular remeshing
I. Background
Summary
Background
Triangular Meshes
Remeshing
Multiresolution analysis
Compression
Bit allocation
I. Background
Semi-regular remeshing
Irregular
mesh
Simplification
Coarse
mesh
Subdivision
Semi-regular
mesh
Coarse
mesh
Finest
Subdivised
Original
semi-regular
mesh
mesh
version
(1)
I. Background
Semi-regular remesher
MAPS (A. Lee et al. , 1998)

Coarse mesh (geometry+connectivity)

N sets of 3D details (geometry) => 3 floating numbers
Normal Meshes (I. Guskov et al., 2000)

Coarse mesh (geometry+connectivity)

N’ sets of 3D details (geometry) => 1 floating number
I. Background
Normal Meshes
Known direction: normal at the surface
Surface to remesh
v2
v3
d 2 ,0
M1 M 2
d 2,1
v4
d1,0
=> More compact representation
v0
M
v1
I. Background
Summary
Background
Triangular Meshes
Remeshing
Multiresolution analysis
Compression
Bit allocation
I. Background
Multiresolution analysis
…
Details
Details
Details
Multiresolution Representation:

Low frequency (LF) mesh
(geometry + topology)

N sets of wavelet coefficients (3D vectors)
(geometry)
Details
I. Background
Summary
Background
Triangular Meshes
Remeshing
Multiresolution analysis
Compression
Bit allocation
I. Background
Compression
Objective :
reduce the information quantity useful for
representing numerical data
2 approachs :
Lossy or lossless compression
High compression ratii
=> Lossy compression
I. Background
Compression scheme
Semi-regular
Wavelet coefficients
Transform
Remeshing
Q
Entropy
Coding
1010…
Bit
Allocation
Preprocessing
Target bitrate
or distortion
Optimize the
Rate-Distortion (RD)
tradeoff
I. Background
Summary
Background
Triangular Meshes
Remeshing
Multiresolution analysis
Compression
Bit allocation
I. Background
Bit allocation: goal
Optimization of the tradeoff between bitstream size
and reconstruction quality:


minimize D(R)
or
minimize R(D)
D
R
I. Background
Bit allocation and meshes
Related Works (geometry compression):
Zerotree coding

PGC :
Progressive Geometry Compression (A. Khodakovsky et al., 2OOO)

NMC :
Normal Mesh Compression (A. Khodakovsky et I. Guskov, 2002).
=> Stop coding when bitstream given size is reached.
Estimation-quantization (EQ) coding

MSEC :
Geometry Compression of Normal Meshes Using Rate-Distortion
Algorithms (S. Lavu et al., 2003)
=> Local RD optimization.
I. Background
Proposed bit allocation
1. Low computational complexity
2. Improve the quantization process
3. Maximize the quality of the reconstructed mesh
according to a given target bitrate
=> Which distortion criterion
for evaluating the losses?
Summary
Background
Distortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-off
Experimental results
Conclusions and perpectives
II. Distortion criterion for multiresolution meshes
Semi-regular
Coding/Decoding
Entropy
coding
Q
Transform
Remeshing
Bit
Allocation
Preprocessing
Target bitrate or
distortion
Inverse
transform
Quantized
semi-regular
Q*
Entropy
Decoding
1010…
II. Distortion criterion for multiresolution meshes
Considered distorsion criterion
MSE due to quantization of the semi-regular mesh
DT  MSESR 
1
# SR
# SR1

v j  vˆ j
j 0
2
2
Number of vertices
semi-regular vertices
quantized semi-regular vertices
MSE for one subband
Wavelet =>
MSESR  functionMSEi ?
II. Distortion criterion for multiresolution meshes
Related works
K.Park and R.Haddad (1995)


M-channel scheme
quantization model : “noise plus gain”
Filter bank
B.Usevitch (1996)



quantization model : “additive noise”
N decomposition levels
Sampled on square grids
Problem : - non adapted for lifting scheme !
- usable for any sampling grid ?
II. Distortion criterion for multiresolution meshes
Lifting scheme for meshes
3 prédiction operators P
=> wavelet coefficients
3 update operators U
=> LF mesh
Triangular grid => 4 channels
II. Distortion criterion for multiresolution meshes
Triangulaire sampling
1 triangular grid => 4 cosets
0
2
0
2
0
3
1
3
1
n2
LF subband (0)
0
2
0
HF subband 1
HF subband 2
3
1
0
n1
HF subband 3
II. Distortion criterion for multiresolution meshes
4-channel lifting scheme: analysis
+
-P1
+
+
U1
HF 1
+
split
-P2
U2
+
Semiregular
mesh
LF
HF 2
-P3
+
U3
HF 3
II. Distortion criterion for multiresolution meshes
4-channel lifting scheme: synthesis
+
LF
+
+
P
-U
+
HF 1
P
-U
+
HF 2
-U
HF 3
Merge
P
+
Semiregular
mesh
=> Derivation of the MSE of the quantized mesh
according to the quantization error of each 4 subband
II. Distortion criterion for multiresolution meshes
Proposed Method
Input signal :
s  s(k )  R / k  K 
Quantization error model : « additive noise »
  sˆ  s 
S is one realization of a stationar and ergodic
random process =>
deterministic quantity

=> MSE of the input signal
EQM SR 
1
# SR
R 0
II. Distortion criterion for multiresolution meshes
Proposed Method: Hypothesis
Uncorrelated error in each subband
Subband errors mutually uncorrelated
MSE SR 
1
M 1
R 0 R 0

# SR
i 0
Synthesis filter energy
gi
i
Quantization error
energy
II. Distortion criterion for multiresolution meshes
Proposed Method: principle
Synthesis filter energy rg i 0 


Polyphase components of the filters
Cauchy theorem
Quantization error energy r i 0 

Uncorrelated error in each subband
II. Distortion criterion for multiresolution meshes
Proposed Method: solution
For 1 decomposition level
M 1
MSE SR 
 wi MSEi
MSE of the subband i
j 0
with
wi 
# SRi
# SR
M 1
  G k 
2
i, j
Weights relative to the
non-orthogonal filters
j  0 kZ d
Polyphase
component
II. Distortion criterion for multiresolution meshes
polyphase representation
Lifting scheme:
1 0,0
  G
 
u11, 0
  G
  G
G
G
u22 , 0
 

 
 
u MM1 1, 0
  G
G
p1 0 ,1
G
p20, 2


1 G
u11p,1
G
u1 1p,22


 uG
p
2 21,1
1 G
u22p, 22





G
u MM1p11,1
G
u MM1p1,22


Gp0M, M1 1 



G
u11 ,pMM11 


G
u22 ,pMM11 







1G
 uMM11,pMM11 

New formulation :
=>=>
Polyphase
depend
on only
the
can be components
applied easily
to lifting
scheme
prediction and update opérators
II. Distortion criterion for multiresolution meshes
Proposed Method : solution
For N decomposition levels
N 1 M 1
MSE  WN 1, 0  MSE N 1, 0   Wi ,l  MSEi ,l
i  0 l 1
avec Wi ,l 
# SRi ,l
# SR
M 1
et
wi 

w0   wl
i
 Gi , j k 
j  0 kZ d
2
II. Distortion criterion for multiresolution meshes
Outline
This formulation can be applied to lifting scheme
Global formulation of the weights for any :

Grid and related subsampling

number of channels M

Number of decomposition levels N
II. Distortion criterion for multiresolution meshes
Experimental Results
=> PSNR Gain : up to 3.5 dB
II. Distortion criterion for multiresolution meshes
Visual impact
Without the weights
Original
With the weights
II. Distortion criterion for multiresolution meshes
Semi-regular
Coding/Decoding
Entropy
coding
Q
Transform
Remeshing
Bit
Allocation
Preprocessing
Target bitrate or
distortion
Inverse
transform
Quantized
semi-regular
Q*
Entropy
Decoding
1010…
II. Distortion criterion for multiresolution meshes
MSE and irregular mesh
Quality of the reconstructed mesh :

Reference : irregular mesh

Used metric:
geometrical distance between two surfaces:
the «surface-to-surface distance (s2s) »
=> Is the MSE suitable to control the quality?
II. Distortion criterion for multiresolution meshes
Quality of the reconstructed mesh


s 2sSRq , IR   max d SRq , IR , d IR, SRq 
Quantized mesh
(semi-regular)
Input mesh
(irregular)
Forward distance:
1
2
 1

2
d ( M  M )  
pM d ( p M ) dM  
 M 

distance between one point and one surface:
d ( p, M ' )  min p  p' 2  p  ProjM' ( p)
p ' M '
2
II. Distortion criterion for multiresolution meshes
Simplifying approximations
Normal meshes:
=> infinitesimal remeshing error
=> uniform and regular geometry sampling
Relation with the
quantization error?
Highly detailed meshes:
=> densely sampled geometry
dss SRq , IR  
1
# SR
# SR1
 d vˆ , SR 
2
j
j 0
II. Distortion criterion for multiresolution meshes
Hypothesis: asymptotical case
Preservation of the LF subbands
vˆ j , SR  slightly
vˆ j modified
vj
=> dorientations
=> normal
2
=> errors lie in the normal direction (normal meshes)
n’ θ n
n’ n
θ
v2
vˆ2
ε(v2 )
v2
vˆ1
v0
vˆ0
ε(v2 )
vˆ1
v1
v0
v1
vˆ2
vˆ0
II. Distortion criterion for multiresolution meshes
Proposed heuristic
Approximating formulation: s 2s SRq , IR  
# SR 1
1
 d vˆ j , SR 
2
# SR
j 0
Asymptotical case
+ normal meshes
d vˆ j , SR   vˆ j  v j
2
1
# SR1
ˆ j control
d SS SRq
, IR  
v j  vto
=> MSE
: suitable
criterion

2
# SR j 0
the quality of the reconstructed mesh
EQM SR
2
Summary
Background
Distortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-off
Experimental results
Conclusions and perpectives
III.Optimization of the Rate-Distorsion trade-off
Optimization of the Rate-Distorsion
trade-off
Objective :
find the quantization steps that maximize the quality of
the reconstructed mesh
MSE SR
minimize

RT  Rtarget
with constraint
Scalar quantization (less complex than VQ)
3D Coefficients => data structuring?
III.Optimization of the Rate-Distorsion trade-off
Local frames
Normal at the surface: z-axis of the local frame
z
z
x
x
Local
frame
Global
frame
=> Coefficient :
 Tangential components (x and y-coordinates)
 Normal components (z-coordinates)
III.Optimization of the Rate-Distorsion trade-off
Histogram of the polar angle
Local frame:
z
θ
y
x
0°
90°
180°
=>=>
Most
of coefficients
haveseparately
only normal
Components
treated
components
(2 scalar subbands)
III.Optimization of the Rate-Distorsion trade-off
MSE of one subband i
MSEi   MSEi , j  MSESRi ,1  MSESRi , 2
jJ i
MSE relative to the
tangential components
MSE relative to the
normal components
III.Optimization of the Rate-Distorsion trade-off
How solving the problem?
Find the quantization steps and lambda
that minimize the following lagrangian criterion:
 N

J  qi , j    Wi  MSE SR i , j qi , j        ai , j Ri , j qi , j   Rcible 


i 0
jJ i
i

0
j

J
i


N
Distortion
Method:
=> first order conditions
Constraint relative
to the bitrate
III.Optimization of the Rate-Distorsion trade-off
Solution
Need to solve
(2N + 4) equations with (2N + 4) unknowns
ai , j
 MSE SR i , j qi , j 
 

Wi
 Ri , j qi , j 
 N

ai , j Ri , j qi , j   Rtarget


 i 0 jJ i

  qi , j 
q  R
i, j
PDF of the component sets:
Generalized Gaussian Distribution (GGD)
=> model-based algorithm (C. Parisot, 2003)
T
III.Optimization of the Rate-Distorsion trade-off
Model-based algorithm
compute the variance and α for each subband
λ
  Ri , j
compute the bitrates
for each subband
Complexity : 12 operations / semi-regular
newExample
λ
:
Target bitrate
0.4 second
reached? (PIII
Look-up tables
512 Mb Ram)
=> Fast
process.
compute
the quantization
step of each subband
  qi , j
Summary
Background
Distortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-off
Experimental results
Conclusions and perpectives
IV. Experimental results
Compression scheme
Connectivity coding*
(coarse mesh connectivity)
Normal
meshes
Unlifted
Butterfly
SQ
MPX
1010…
3D-CbAC*
Bit
Allocation
Preprocessing
Target Bitrate
* Context-based Bitplane Arithmetic
Coder (EBCOT-like)
* Touma-Gotsman coder
IV. Experimental results
Visual results
Input mesh
(irregular)
CR = 226
0.82 bits/iv
CR = 83
2.2 bits/iv
Compression ratio:
CR 
CR = 900
0.2 bits/iv
# IR  3  32  # TrIR  3  log 2 # IR 
bitstream size
IV. Experimental results
Comparison
Quality criterion :


bb

PSNR  20 log 10 
 s 2sIR, SRq  
Bounding box
diagonal
s2s between the irregular input
mesh and the quantized semiregular one
State-of-the-art methods:



NMC (Normal meshes + Butterfly NL + zerotree)
EQMC (Normal meshes + Butterfly NL + EQ)
PGC (MAPS + Loop)
IV. Experimental results
PSNR-bitrate curve: Rabbit
IV. Experimental results
PSNR-bitrate curve: Feline
=> PSNR Gain: up to 7.5 dB
IV. Experimental results
PSNR-bitrate curve: Horse
IV. Experimental results
Geometrical comparison
NMC
(62.86 dB)
Proposed algorithm
(65.35 dB)
Bitrate = 0.71 bits/iv
Summary
Background
Distortion criterion for multiresolution meshes
Optimization of the Rate-Distorsion trade-off
Experimental results
Conclusions and perpectives
V. Conclusions and perspectives
Conclusions

New shape compression method:
Contributions :
1.
Weighted MSE : suitable distortion criterion
2.
Original formulation of the weights (suitable in case of lifting scheme)
3.
4.
Bit alllocation of low computational complexity that optimizes the
quality of a quantized mesh.
An original Context-based Bitplane Arithmetic Coder
=> Better results than
the state-of-the-art methods.
V. Conclusions and perspectives
Perspectives
Take into account some visual properties the
human eye appreciates (local curvature, volume,
smoothness…)
Reference : Z.Karni and C.Gotsman, 2000
Algorithm for huge meshes: « on the flow »
compression
Reference : A. Elkefi et al., 2004
End