Transcript Slide 1

ON BETWEEN-COEFFICIENT
CONTRAST MASKING OF DCT BASIS
FUNCTIONS
1
Nikolay Ponomarenko (*), Flavia Silvestri(**),
Karen Egiazarian (***), Marco Carli (**),
Jaakko Astola (***) and Vladimir Lukin (*)
(*) National Aerospace University, Kharkov, Ukraine
(**) University of Rome "Roma TRE", Rome, Italy
(***) Tampere University of Technology, Tampere, Finland
Marco Carli
VPQM 2006
26/01/2007
2
Outline
Outline
1.
2.
3.
4.
5.
Introduction
Proposed model of between-coefficient contrast masking of DCT basis functions
Modification of PSNR using a new masking model
MATLAB implementation of the proposed measure
A set of test images for comparative analysis for taking into account the masking
effect in quality metrics
6. Subjective experiment to test quality measures
7. Results of the experiment
8. Examples of quality assessment of test images
9. Example of use of the proposed model to masking noise on a real image
10.Summary and Conclusion
Marco Carli
VPQM 2006
26/01/2007
3
Introduction
Human visual sensitivity
varies as a function of several key
image properties, such as:
Light level
Spatial frequency
Color
Masking model can be used in :
Image and video compression
Image filtering
Digital watermarking
Validation of effectiveness of image processing
methods
Local image contrast
Eccentricity
Temporal frequency
Goal of the research:
Efficient accounting for local image
contrast using a model of betweencoefficient contrast masking of
DCT basis functions
Marco Carli
Requirements to the model:
Images compressed (filtered or processed) with
accounting the model can be visualized in
unknown illumination conditions, monitor
brightness, distance to the monitor, viewing angle,
etc. Thus such model should operate by only
some averaged parameters of image visualization
VPQM 2006
26/01/2007
4
Proposed model of between-coefficient
contrast masking of DCT basis functions
Let us denote a weighted energy of DCT coefficients of an image block 8x8 as Ew(X):
7
7
E w (X )   X ij C ij
2
i 0
(1)
j 0
where Xij is a DCT coefficient with indices i,j, Cij is a correcting factor determined by the CSF.
The DCT coefficients X and Y are visually undistinguished if Ew(X-Y) < max(Ew(X)/16,
Ew(Y)/16), where Ew(X)/16 is a masking effect Em of DCT coefficients X (normalizing factor 16
has been selected experimentally).
Reducing of the masking effect due to an edge presence in the
analyzed image block: we propose to reduce a masking effect
for a block D proportionally to the local variances V(.) in blocks
D1, D2, D3, D4 in comparison to the entire block:
Em(D) = Ew(D)δ(D)/16,
(2)
where δ(D) = (V(D1)+V(D2)+V(D3)+V(D4))/4V(D), V(D) is the
variance of the pixel values in block D.
Marco Carli
VPQM 2006
26/01/2007
5
Proposed model of between-coefficient
contrast masking of DCT basis functions
Values of Cij have been obtained using the quantization table for the color component Y of
JPEG (the values of quantization table JPEG have been normalized by 10 and squared)
JPEG Quantization table of Y
component
16
11
10
16
24
40
51
Values of Cij
i\j
0
1
2
3
4
5
6
7
0
0
0.8264
1.0000
0.3906
0.1736
0.0625
0.0384
0.0269
1
0.6944 0.6944
0.5102
0.2770
0.1479
0.0297
0.0278
0.0331
2
0.5102 0.5917
0.3906
0.1736
0.0625
0.0308
0.0210
0.0319
3
0.5102 0.3460
0.2066
0.1189
0.0384
0.0132
0.0156
0.0260
4
0.3086 0.2066
0.0730
0.0319
0.0216
0.0084
0.0094
0.0169
61
12
12
14
19
26
58
60
55
14
13
16
24
40
57
69
56
14
17
22
29
51
87
80
62
18
22
37
56
68 109 103 77
24
35
55
64
81 104 113 92
49
64
78
87 103 121 120 101
5
0.1736 0.0816
0.0331
0.0244
0.0152
0.0092
0.0078
0.0118
72
92
95
98 112 100 103 99
6
0.0416 0.0244
0.0164
0.0132
0.0094
0.0068
0.0069
0.0098
7
0.0193 0.0118
0.0111
0.0104
0.0080
0.0100
0.0094
0.0102
Marco Carli
VPQM 2006
26/01/2007
6
Modification of PSNR using a new
masking model
A basis of the proposed metric is a PSNR-HVS (Egiazarian K., Astola J., Ponomarenko N., Lukin V.,
Battisti F., Carli M. “New full-reference quality metrics based on HVS”, CD-ROM Proceedings of the Second
International Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p).
Block 8x8
of original
image
Block 8x8
of distorted
image
DCT of
difference
between
pixel
values
Reduction
by value of
contrast
masking
MSEH
calculation
of the block
Flow-chart of PSNR-HVS-M calculation
Reduction by value of contrast masking in
accordance to the proposed model is carried out in
the following manner. First, the maximal masking
effect Emax is calculated as max(Em(Xe), Em(Xd))
where Xe and Xd are the DCT coefficients of a
original image block and a distorted image block,
respectively. Then, the visible difference between Xe
and Xd is determined as:
Marco Carli
X eij  X dij , i  0, j  0

0, X eij  X dij  E norm / C ij
X∆ij = 
X eij  X dij  E norm / C ij , X eij  X dij  E norm / C ij
X  X  E / C , otherwise
dij
norm
ij
 eij
where Enorm is
E max / 64 .
VPQM 2006
26/01/2007
7
MATLAB implementation of the
proposed measure
The MATLAB implementation of PSNR-HVS-M is available on www.cs.tut.fi/~ponom/psnrhvsm.htm
Marco Carli
VPQM 2006
26/01/2007
A set of test images for comparative
analysis for taking into account the
masking effect in quality metrics
8
While creating an image test set we took into
consideration the following:
Such set should contain images with both spatially
uncorrelated and correlated noise (the latter one is
typical for images formed by digital cameras and is
more visible for humans);
The set should contain images with noise distributed
spatially uniformly and with noise which is masked or
unmasked (concentrated in regions with maximal and
minimal masking properties, respectively);
The set is to be maximally simple for visual
comparison by humans (because of this in our set we
used only three values of noise variance σ2 and a total
number of distorted test images was 2x3x3 = 18
images).
Original test images having a lot of different type
regions with high masking effect
Marco Carli
VPQM 2006
26/01/2007
9
Subjective experiment to test quality
measures
Result of the experiment: the test image set ordered according to subjective visual quality.
Number of observers: 155 (45 from Finland, 43 from Italy, 67 from Ukraine).
Number of comparisons of visual appearance of test images: 8192 (on average 53 for each observer).
17” or 19” Monitor Resolution: 1152x864 pixels.
Number of experiments carried out using CRT monitors: 128.
Number of experiments carried out using LCD monitors: 27.
Cross correlation factors
Group of observers
Spearman correlation
Kendall correlation
Finland – Italy
0.996
0.895
Finland – Ukraine
0.996
0.935
Italy - Ukraine
0.997
0.961
CRT - LCD
0.998
0.922
Marco Carli
VPQM 2006
26/01/2007
10
Results of the experiment
Spearman Kendall
correlation correlation
Measure
Reference
PSNR-HVS-M
This paper
0.984
0.948
PSNR-HVS
Egiazarian K., Astola J., Ponomarenko N., Lukin V., Battisti F., Carli M. “New fullreference quality metrics based on HVS”, CD-ROM Proceedings of the Second Intern.
Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p
0.895
0.712
NQM
Damera-Venkata N., Kite T., Geisler W., Evans B. and Bovik A. "Image Quality
Assessment Based on a Degradation Model", IEEE Trans. on Image Processing, Vol.
9, 2000, pp. 636-650
0.857
0.673
Solomon J. A., Watson A. B., and Ahumada A. “Visibility of DCT basis functions:
Effects of contrast masking”. Proc. of Data Compression Conf., 1994, pp. 361-370
http://vision.arc.nasa.gov/dctune/ - DCTune 2.0 page
0.829
0.712
Wang Z., Bovik A. “A universal image quality index”, IEEE Signal Processing Letters,
vol. 9, March, 2002, pp. 81–84
0.550
0.438
0.537
0.359
DCTune
UQI
PSNR
Peak Signal to Noise Ratio
VQM
Xiao F. “DCT-based Video Quality Evaluation”, Final Project for EE392J, 2000
0.441
0.281
SSIM
Wang Z., Bovik A., Sheikh H., Simoncelli E. “Image quality assessment: from error
visibility to structural similarity”, IEEE Trans. on Image Proc., vol.13, 2004, pp.600-612
0.406
0.358
VIF
Sheikh H. R. and Bovik A. C., "Image Information and Visual Quality", IEEE
Transactions on Image Processing, vol. 15, February, 2006, pp. 430-444
0.377
0.255
PQS
Miyahara, M., Kotani, K., Algazi, V.R. ”Objective picture quality scale (PQS) for image
coding”, IEEE Transactions on Communications, vol. 46, issue 9, 1998, pp. 1215-1226
0.302
0.242
Marco Carli
VPQM 2006
26/01/2007
11
Examples of quality assessment of test
images
DCTune = 24.9, PSNR-HVS-M = 33.20 dB
PSNR-HVS-M says: “This is better!”
Marco Carli
DCTune = 24.5, PSNR-HVS-M = 29.31 dB
DCTune says: “This is better!”
VPQM 2006
26/01/2007
12
Examples of quality assessment of test
images
SSIM = 0.80, PSNR-HVS-M = 25.50 dB
SSIM says: “This is better!”
Marco Carli
SSIM = 0.79, PSNR-HVS-M = 31.29 dB
PSNR-HVS-M says: “This is better!”
VPQM 2006
26/01/2007
13
Example of use of the proposed model
to masking noise on a real image
Original test image Baboon
Marco Carli
The image with masked noise,
PSNR=26.18 dB, MSE=158,
PSNR-HVS=34.43 dB, PSNR-HVS-M=51.67 dB
VPQM 2006
26/01/2007
14
Summary and Conclusion
Summary
A simple and efficient model of between-coefficient contrast masking of DCT basis functions
is proposed;
A modification of PSNR that takes into account this masking model is proposed;
Subjective experiments on comparison of known quality metrics are carried out;
Conclusions
The proposed measure based on the designed masking model has demonstrated the best
correspondence to the results of the subjective experiments. However for providing more
reliable conclusions on efficiency of the proposed model it is necessary to carry out additional
more extensive experiments and research.
The proposed test set has allowed to demonstrate drawbacks of many well known metrics
that do not fully or even badly correspond to human visual perception.
Marco Carli
VPQM 2006
26/01/2007