x - Hanyang University
Download
Report
Transcript x - Hanyang University
Computer Vision –
Sampling
Hanyang University
Jong-Il Park
Introduction
Topics
Television Standards (NTSC, SECAM, PAL)
Multi-dimensional Sampling Theory
Practical Limitations in Sampling and Reconstruction
Image Re-Sampling
Department of Computer Science and Engineering, Hanyang University
Department of Computer Science and Engineering, Hanyang University
Television Standards
Frame
525 lines/frame
(or 625 lines/frame)
frame rate : 30 frames/s
(or 25 frames/s)
Field
even field, odd field
262.5 lines/field
(or 312.5 lines/field)
Department of Computer Science and Engineering, Hanyang University
NTSC
NTSC
525 scan lines/frame, 30 frames/s
line frequency : 15,750 Hz ( = 30 x 525 Hz )
2:1 line interlacing
color video composite signal - (Y,I,Q)
Bandwidth : Y - 4.2MHz,
I - 1.3MHz, Q - 0.5MHz
color sub-carrier : 3.583125 MHz
( = 30 x 525 x 455/2 Hz)
phase change of 180 : between lines, between
frames
Korea, North America, Japan etc.
Never Twice Same Color!
Department of Computer Science and Engineering, Hanyang University
Other Standards
SECAM(Sequential Couleur a Memoire)
Idea
avoid the quadrature demodulation and corresponding
chrominance shift due to phase detection errors in
NTSC
France, Eastern Europe.
625 lines/frame, 25 frames/s with 2:1 line interlace.
color video composite signal - (Y,U,V)
color sub-carrier : 4.25 MHz (for U) and 4.41 MHz (for
V)
Something Essentially Contradictory to American
Method!
Department of Computer Science and Engineering, Hanyang University
Other Standards (cont.)
PAL (Phase Alternating Line)
Idea
changes by 180 degree between successive line in
the same field
cross talk can be suppressed
• Germany, UK, South America
• 625 lines at 25frames/s with 2:1 line interlace.
• color video composite signal - (Y,U,V)
(Bandwidth) Y - 4.2MHz, U - 1.3MHz, V - 1.3MHz
Peace At Last!
Department of Computer Science and Engineering, Hanyang University
Sampling Theory
For One-Dimensional Signal
g (t )
g S (t )
Periodic
Sampling
s(t )
g S (t ) g (t ) s(t )
where s(t ) (t mt )
m
S g S ( f ) S g ( f ) SS ( f )
(Fourier T ransform)
Department of Computer Science and Engineering, Hanyang University
Sampling Theory (cont.)
For One-Dimensional Signal (cont.)
Ss(f)
Sg(f)
...
...
-B
B
f
-2fs
-fs
0
fs
2fs
f
Nyquist Sampling Rate
fs > 2B
fs =1/ t
reconstruction filter
Sgs(f)
...
...
-2fs
-fs -B
B fs
2fs
3fs
4fs
f
Department of Computer Science and Engineering, Hanyang University
Sampling Theory(cont.)
For Two-Dimensional Signal
Band-limited Image
Fourier Transform of
a bandlimited function
Its region of support
Department of Computer Science and Engineering, Hanyang University
Sampling Theory (cont.)
For Two-Dimensional Signal (cont.)
Structure
Orthogonal Structure (Rectangular Tesselation)
Field Quincunx Structure (Triangular Tesselation)
g ( x, y )
g S ( x, y)
Periodic
Sampling
s(t )
g S ( x, y) g ( x, y) s( x, y)
SgS (u, v) Sg (u, v) SS (u, v)
(Fourier T ransform)
Department of Computer Science and Engineering, Hanyang University
Sampling Theory(cont.)
For Two-Dimensional Signal(cont.)
Structure(cont.)
Department of Computer Science and Engineering, Hanyang University
On a rectangular samplinggrid,
theNyquist samplinginterval; Δx Δy Δ1 1
On a triangular samplinggrid (quincunx structureor triangular
structure), with theintersample distanceof Δ2 , the
spectrumwill repeat with spaceing Δ12
if Δ2 2 , then ther
e will be no aliasing
T hus,if theimage does not contain the high frequencies
in bot h dimension,thesamplingratecan be reduced
by a factorof 2
Department of Computer Science and Engineering, Hanyang University
2D sampling
For Two-Dimensional Signal (cont.)
Orthogonal Structure (Rectangular Tesselation)
SgS (u, v) Sg (u, v) SS (u, v)
g S ( x, y) g ( x, y) s( x, y)
(Fourier T ransform)
Sampling Function
s ( x, y ) ( x mx, y ny )
n
m
S s (u , v) uv (u ku , v lv)
k
where u
l
1
1
, v
x
y
Department of Computer Science and Engineering, Hanyang University
2D sampling - Spectrum
For Two-Dimensional Signal (cont.)
Orthogonal Structure (cont.)
Spectrum of sampled signals
By
v
y
y
x
x
(a) sampling function
v
u
(b) spectrum of sampling
function and signal
Bx
u
v
v
u
u
(c) spectrum of sampled signal
Department of Computer Science and Engineering, Hanyang University
2D sampling - Reconstruction
For Two-Dimensional Signal(cont.)
Orthogonal Structure(cont.)
Reconstruction of the original image from its samples
If 2 Bx u and 2 B y v, then
1
xy
, (u , v)
H (u , v)
uv
, ot herwise
0
g ( x, y)
m
sin(xu m) sin( yv n)
g
(
m
,
n
)
S
n
( xu m) ( yv n)
Nyquist Sampling Rate(or Frequency) and Nyquist
Interval
u Nyquist 2 Bx , vNyquist 2 By ,
1
1
1
1
xNyquist
, y Nyquist
,
u Nyquist 2 Bx
vNyquist 2 By
Department of Computer Science and Engineering, Hanyang University
Reconstruction Filter
sin x x sin y y
hr ( x, y ) K
x x y y
hc ( x, y )
20 J1 0 x 2 y 2
x2 y2
where J1 () is a first - order Bessel function
Department of Computer Science and Engineering, Hanyang University
Aliasing effect
For Two-Dimensional Signal(cont.)
Orthogonal Structure(cont.)
Aliasing Effect
If u uNyquist or v vNyquist , then
the original image cannot be reconstructed from its samples.
Bx
u
nu
(n 1)u
Department of Computer Science and Engineering, Hanyang University
Eg. Aliasing
For Two-Dimensional Signal (cont.)
Orthogonal Structure (cont.)
Aliasing Effect (cont.)
Zone Plate image ( = 1)
Aliasing ( = 2)
Department of Computer Science and Engineering, Hanyang University
Eg. Aliasing
Examples
Little aliasing due to an effective antialiasing filter
Noticeable aliasing
Department of Computer Science and Engineering, Hanyang University
Practical limitations in sampling
Practical Limitations
Real-world images are not band-limited.
aliasing errors
can be reduced by LPF before sampling
LPF attenuate higher spatial frequencies
Resolution loss
blurring
No ideal LPF at reconstruction stage.
Department of Computer Science and Engineering, Hanyang University
Sampling aperture
Finite aperture (finite duration pulse)
H 0 e
j T 2
2 sin T 2
Department of Computer Science and Engineering, Hanyang University
Sampling aperture - Spectrum
Practical Limitations (cont.)
Sampling Aperture/ LPF operation
Department of Computer Science and Engineering, Hanyang University
Reconstruction
Reconstruction of a signal from its sample using
interpolation
Department of Computer Science and Engineering, Hanyang University
Linear interpolation
linear interpolation
Department of Computer Science and Engineering, Hanyang University
Department of Computer Science and Engineering, Hanyang University
Sampling Theory(cont.)
Department of Computer Science and Engineering, Hanyang University
Geometrical Image Resampling
Bilinear interpolation
F ( p, q) (1 a)[(1 b) F ( p, q) bF( p, q 1)]
a[(1 b) F ( p 1, q) bF( p 1, q 1)]
Department of Computer Science and Engineering, Hanyang University
Geometrical Image Resampling(Cont.)
Bicubic interpolation
F ( p, q)
2
2
F ( p m, q n) R [m a]R [(n b)]
m 1 n 1
c
c
1
Rc ( x) [(x 2)3 4( x 1)3 6( x)3 4( x 1)3 ] ,
6
where ( z ) m z m for z 0, 0 for z 0
Department of Computer Science and Engineering, Hanyang University
Resampling by Convolution
Convolution methods : integer zoom(ex. 2 : 1)
Zero Interleaving
Convolution
Pyramid : 1 1
Peg :
Cubic B-spline :
1 1
1 1
2 1
2 4 2
4
1 2 1
1
4
1
6
64
4
1
Bell :
1
1 3
16 3
1
3
9
9
3
3
9
9
3
1
3
3
1
4 6 4 1
16 24 16 1
24 36 24 6
16 24 16 1
4 6 4 1
Department of Computer Science and Engineering, Hanyang University
Eg: Resampling by Convolution
Original
Zero interleaving
Department of Computer Science and Engineering, Hanyang University
Eg: Resampling by Convolution(Cont.)
peg
Bell
Pyramid
Cubic B-spline
Department of Computer Science and Engineering, Hanyang University
Image Warping
image filtering: change range of image
g(x) = h(f(x))
f
f
h
x
x
image warping: change domain of image
g(x) = f(h(x))
f
f
h
x
x
Department of Computer Science and Engineering, Hanyang University
Image Warping
image filtering: change range of image
g(x) = h(f(x))
f
g
h
image warping: change domain of image
g(x) = f(h(x))
f
g
h
Department of Computer Science and Engineering, Hanyang University
Parametric (global) warping
Examples of parametric warps:
translation
affine
rotation
perspective
aspect
cylindrical
Department of Computer Science and Engineering, Hanyang University
2D Coordinate Transformations
x’ = x + t
x = (x,y)
rotation:
x’ = R x + t
similarity:
x’ = s R x + t
affine:
x’ = A x + t
perspective:
x’ H x
x = (x,y,1)
(x is a homogeneous coordinate)
translation:
These all form a nested group
Department of Computer Science and Engineering, Hanyang University
Image Warping
Given a coordinate transform x’ = h(x) and a source
image f(x), how do we compute a transformed image
g(x’) = f(h(x))?
h(x)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Forward Warping
Send each pixel f(x) to its corresponding location x’ =
h(x) in g(x’)
• What if pixel lands “between” two pixels?
h(x)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Forward Warping
Send each pixel f(x) to its corresponding location x’ =
h(x) in g(x’)
• What if pixel lands “between” two pixels?
• Answer: add “contribution” to several pixels,
normalize later (splatting)
h(x)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Inverse Warping
Get each pixel g(x’) from its corresponding location x =
h-1(x’) in f(x)
• What if pixel comes from “between” two pixels?
h-1(x’)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Inverse Warping
Get each pixel g(x’) from its corresponding location x =
h-1(x’) in f(x)
• What if pixel comes from “between” two pixels?
• Answer: resample color value from
interpolated (prefiltered) source image
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Interpolation
Possible interpolation filters:
nearest neighbor
bilinear
bicubic (interpolating)
sinc / FIR
Needed to prevent “jaggies”
and “texture crawl”
Department of Computer Science and Engineering, Hanyang University