Transcript View synthesis software and assessment of its performance

M15672:
View synthesis software and
assessment of its performance
Mateusz Gotfryd
Krzysztof Wegner
Marek Domański
Chair of Multimedia Telecommunications and Microelectronics
Poznań University of Technology, Poland
July, 21th 2008
1
Outline
Algorithm description
Test sequences
Used depth maps
Experiment description








Objective video quality measure
Subjective video quality measure
Results
Summary
2
Algorithm description



Virtual camera can be synthesized from
a real camera (the real reference view)
and the respective depth map
Unfortunately such an approach suffers
from occlusion
Virtual camera can be synthesized more
correctly if two reference views are
used
3
Block diagram
View synthesis
algorithm
4
View synthesis algorithm




Proposed algorithm is composed of two
identical, separate paths
Each virtual view is synthesized from
one reference view
In the end both images of virtual view
are merged into one
Futher we describe single path in detail
(right one)
5
Virtual view depth map


Coordinates of each point in a
reference view are transformed into
coordinates of a point in the virtual
view and virtual view depth map is
created
In the case when two points from
reference view are transformed into
the virtual view and have the same
coordinates, always the points closer
to the camera location are chosen
Visibility
problem
Proper
depth values
6
Virtual view depth map




Resultant virtual view depth map has
many small black holes on surfaces
which have been rotated during the
transformation from reference view
into a virtual view
Median filter is used on virtual view
depth map
Some regions in view have been
uncovered
Black regions have unknown depth
Uncovered
regions
7
Virtual view synthesis


Color information is copied from reference view based on
inverse homography matrix
Calculated coordinates of corresponding point in reference
view are not on pixel gird therefore interpolation is used
8
Merging of two
virtual view images


Images from two paths are merged into one
Unknown regions from first path are filled
with information from second path
9
Merging of two
virtual view images


Images from two paths are merged into one
Unknown regions from first path are filled
with information from second path
10
Filling holes

Missing areas are interpolated from
neighboring pixels
11
Contour correction


Aliasing and blurring on the edges of the
object are main reasons of „ghosting” effect
Unknown regions are outlined
by 1 pixel-width
12
Test sequences

Sequences provided by Fraunhofer HHI




Book Arrival,
Leaving Laptop,
Alt Moabit.
Only 3 views (2nd, 3rd and 4th) have been
used in the experiments
13
Depth maps

Required depth maps were calculated with
software provided by Nagoya University
14
Experiment description


Video objective quality measured by PSNR
Video subjective quality measured by
Mean Opinion Score (MOS)



15 human subjects
Rating range from 1 to 10
Collected opinions have been averaged
15
Results
16
Single frame
subjective quality
17
View synthesis quality
18
View synthesis quality
19
Average synthesis quality
20
Summary



We have presented a new view synthesis software
Subjective experiments have been carried out with
view-synthesis software provided to MPEG
Our proposal received good results
 in criteria of objective quality measure (PSNR)
 and in case of subjective quality measure (MOS)
21