Transcript notes

Wavelets
Fast Multiresolution Image
Querying
Jacobs et.al. SIGGRAPH95
Outline

Overview / Background
 Wavelets

2D Image matching
 L1,
L2 metrics
 Wavelet metric
 Evaluation

Use in 3D
Image matching
2D analogue of 3D shape matching
 Raster instead of XYZ

What are we trying to match?

Looking for different images of the same things?
 Different
projections
 Different colors

Looking for images that look the same?
 Look

for similar shapes and colors
Metric to discern like human eye
Wavelets

Decompose a signal into component parts
 Fourier
analysis: a signal can be represented
as a (possibly infinite) sum of sine and cosine
functions
Signal becomes a set of wavelet
coefficients
 Coefficients represent features of signal

Wavelets II
Signal can be completely reconstructed
from all the coefficients
 Signal can be partially reconstructed from
some coefficients

Wavelets for 2D Images
Each color plane in image is signal
 Coefficients will represent visual features
in the image
 Store as many coefficients as needed

 Image

compression (see next slide)
c.f. Statistical shape descriptors
Wavelet Reconstruction
SIGGRAPH 96 Course Notes: Wavelets in Computer Graphics
Comparing Images
Develop a metric that describes how
closely two images match
 Smaller difference in metric = images
more similar

Image metrics


Comparing images Q and T, with dimensions i,j
L1-Norm
Q, T 1   Q[i, j ]  T [i, j ]
i, j


For each pixel in Q, calculate the difference
between Q[i,j] and T[i,j]
Add absolute value of differences of all i,j to form
metric
Image metrics II

L2-Norm
1/ 2
Q, T



2

2
   Q[i, j ]  T [i, j ] 
 i, j

For each pixel in Q, calculate the square of the
difference between Q[i,j] and T[i,j]
Add for all i,j, and take square root
Better than L1?
Image metrics III

Problems with L1 and L2
 Expensive
to compute / compare: O(i*j)
 Not discriminating in cases with
Color Shift
 Misregistration
 Noise / Dithering


In general, not good descriptors
 c.f.
D1, D2 in 3D
Wavelets as image metrics
Capture features of images e.g. edges in
coefficients
 Use Haar wavelets

 Square
basis functions
 Easy to implement and compute

Calculate coefficients, truncate, quantize
Truncation
128x128 image has 1282 coefficients
 Truncation = only storing largest ‘n’
coefficients
 ‘n’ ~ 40-60 depending on exact use
 Discarding smaller coefficients discards
high frequency information i.e. detail

 Loss
of that information is desirable
Quantization

Reduce precision of wavelet magnitude
+ve  +1
 Large –ve  -1
 Else  0
 Large

Turns out this works well for matching
Wavelet metric
~
~
Q, T  w0,0 Q[0,0]  T [0,0]   wi , j Q[i, j ]  T [i, j ]
i, j
Q,T are query and target image
coefficients
 w is weighting function

Weighting function




Weighting function applied to give particular
pairs of coefficients different significance in
comparison
Gives ability to statistically tune the metric
Determined experimentally from dataset
(Appendix A)
Weights expensive to calculate
 Compute
fewer
 Bins to map range of i,j onto a weight
Wavelet metric II
Q, T  w0 Q[0,0]  T [0,0] 

~
~
wbin(i , j ) Q[i, j ]  T [i, j ]
~
i , j:Q[ i , j ] 0
For i=0, j=0 value in Q and T is
proportional to the average overall color
 From quantization, use ≠

Calculating coefficients
Standard two-dimensional Haar wavelet
decomposition
 Decompose each row, then decompose
each column of the result
 Trivial to implement

Wavelet metric III

Final metric is
 T[0,0]
 Sign,
i and j of n largest coefficients in T
Faster Matching

To speed up matching, use 6 arrays
 One
for each combination of R,G,B,+, DR+, DR-, …
 Each i,j in Dx is a list of all images with a
metric coefficient in that color range, with that
sign
Evaluation





Better than L1 and L2
Matches to ~1% of database
Compact
Fast to compare: similar complexity to an 8x8 pixel
image L1/L2 for any resolution
More robust




Misregistration
Color shifting
Dithering
Different resolutions
Evaluation II
Evaluation III

Limits
 Scaling
~1.5 times
 Rotation ~20 degrees
 Translation ~ 15% of width
Matching in 3D
Can this be used in 3D as well
 Compares image rather than geometry
 Render 3D into voxels?
 Projection of 3D object into 2D?


In general, other 3D specific methods
probably much better