Transcript Poisson Image Editing

Poisson Image Editing
Patrick Perez Michel Gangnet
Andrew Blake
Microsoft Research UK
Outline


Introduction
Poisson solution to guided interpolation
 Guided Interpolation
 Discrete Poisson solver

Seamless cloning
 Importing gradients
 Mixing gradients

Selection editing





Texture flattening
Local illumination change
Local color change
Seamless tiling
Conclusion
Introduction



The Poisson equation has been used extensively in
computer vision.
The mathematical tool at the heart of this approach is the
Poisson partial differential equation with Dirichlet boundary
conditions which specifies the Laplacian of an unknown
function over the domain of interest, along with the
unknown function values over the boundary of domain.
Solving the Poisson equation also has an alternative
interpretation as a minimization problem:
 It computes the function whose gradient is the closet to some
prescribed vector field-the guidance vector field-under given
boundary conditions
Partial differential equation (PDE)

方程式類別
橢圓型
判斷式
b  4ac  0
2
拋物線型
b 2  4ac  0
雙曲線型
b  4ac  0
2
代表性範例
Laplace方程式
Poisson方程式
波動方程式
 2u  2u
 2 0
2
x
y
 2u  2u

 f ( x, y )
x 2 y 2
2
2

u

u
2

 2
2
x
t
起始條件和邊界條件的分類
 為了能獲得偏微分方程式之解答，其起始條
件和邊界條件可依其特性區分為三類。
第一類：Dirichlet Condition
第二類：Neumann condition
第三類：Robbins condition
Dirichlet Condiction
若依變數(T)本身，在某個獨立變數值時，
被指定，則此條件稱為Dirichlet
Condition，亦稱為essential邊界條件。
下圖為一典型的Dirichlet條件示意圖
由圖中很清楚的顯示，該平板之邊界條
T  f (t ),
件為
t0
T  f (t ) ,
T  T1 ,
T  T0 ,
T  T1 ,
t0
at x  0 , t  0
at x  1 , t  0
at x  0 , t  0
T0
0
1
x
圖 平版Dirichlet Condition 示意圖
Neumann condition
Neumann condition係指依變數之變化率之邊
界條件為定值，抑或獨立變數之函數之情況。
例如
T
 f (t ) , at t  0 ,
0  x 1
或
x
T
 0 , at x  1 ,
t0
Neumann型邊界條件，亦稱為natural
x
boundary condition。
Poisson solution to guided
interpolation

Guided Interpolation
 Let S, a closed subset of
 2, be the image definition domain, and
let  be a closed subset of S with boundary .
 Let f* be a know scalar function defined over S minus the interior of 
and let f be an unknown scalar function defined over the interior of 
 Let v be a vector field defined over 
Guided Interpolation

The simplest interpolant f of f* over  is the membrane
interpolant defined as the solution of the minimization
2
problem:
*
min
f


f
where . 
with f

 f

. .

is the gradient operator
x y
(1)
The minimizer must satisfy the associated Euler-Language
equation
f  0 over  whit f

 f*

2. 2.
where .  2  2 is the laplacian operator
x
y
(2)
Guided Interpolation

Equation 2 is a Laplace equation with Dirichlet boundary conditions.
Guided Interpolation

A guided field is a vector field v used in an extended version of
minimization problem (1):
min
f



f  v
2
with f

(3)
This solution is the unique solution of following Poisson equation
with Dirichlet boundary condition:
f  div v over  whit f
where div v 


 f*

 f*

(4)
u v

is the divergence of v  (u, v)
x y
Three Poisson equations of the form (4) are solved independently in
three color channels of the chosen color space.
Discrete Poisson solver
S  now become finite point sets defined on an
infinite discrete grid.
 Note that S can include all the pixels of an image
or only a subset of them.
 For each pixel p in S, let Np be the set of its 4connected neighbors which are in S, and let
<p,q> denote a pixel pair such that q  N p
 The boundary of  is now   { p  S \  : Np    0}
 Let fp be the value of f at p and f
 { f p , p  }


Discrete Poisson solver

The finite difference discretization of (3) yields the following
discrete, quadratic optimization problem:
*
min  ( f p  f q  v pq ) 2 , with f p  f p , for all p  
(6)
f

 p , q    0
where v pq is the projection of v(

pq
) on the oriented edge [ p, q ]
2
pq
i.e., v pq  v(
)  pq
2
Its solution satisfies the following simultaneous linear
equation:
for all p  , N p f p 
f
q
qN p 

f
q*
qN p 

v
qN p
pq
(7)
Discrete Poisson solver
Equation (7) form a classical, sparse (banded),
symmetric, positive-definite system.
 Results shown in this paper have been computed
using either Gauss-Seidel iteration with
successive overrelaxation or V-cycle multigrid.
 Both methods are fast enough

 i.e., 0.4s. per system on a Pentium 4 for a disk-shaped
region of 60,000 pixels.
Seamless cloning

Importing gradients
 The basic choice for the guidance field v is a gradient field taken
directly from a source image.
 Denoting by g this source image, the interpolation is performed
under the guidance of
v  g,
and (4) now reads
f  g over  whit f
(9)

 f*

(10)
 As for the numerical implementation, the continuous specification
(9) translates into
for all  p, q, v pq  g p  g q
(11)
Importing gradients
 The
seamless cloning tool thus obtained
ensures the compliance of source and
destination boundaries.
 It can be used to conceal undesirable image
features or to insert new elements in an
image, but with much more flexibility
Importing gradients
Importing gradients
Seamless cloning

Mixing gradients
 There are situations where it
is desirable to combine
properties of f* with those of
g, for example to add objects
with holes ,or partially
transparent ones, on top of a
textured or cluttered
background.
 An example is shown in
Fig.6
Mixing gradients


One possible approach is to define the guidance field v as
a linear combination of source and destination gradient
fields but not good.
At each point of  ,we retain the stronger of the
variations in f* or in g, using the following guidance field:
 f * ( x)
for all x  , v(x)  
g ( x)

if f * ( x)  g ( x)
otherwise
(12)
The discrete counterpart of this guidance field is
 f p *  f q * if f p *  f q *  g p  g q
v pq  
otherwise
 g p  g q
(13)
Mixing gradients
Selection edition

Texture flattening
 The image gradient
f is passed through a sparse
*
sieve that retains only the most salient features:
for all x  , v(x)  M ( x)f * ( x)
(14)
Where M is a binary mask turned on at a few locations
of interest
 A good choice for M is an edge detector, in which case
the discrete version of (14), to be plugged into (7)
 f p  f q if an edge lies between p and q
v pq  
otherwise
 0
for all  p, q
(15)
Selection edition
Selection editing

Local illumination changes
 A natural extension is to restrict the correction to a
selected region  ,using appropriate Dirichlet conditions
on 
 Using simplified version of the Fattal et al.
transformation, the guidance field is defined in the logdomain by:
v    f *

f *
with   0.2 times the average gradient norm of f * over ,
and   0.2
Selection editing
Selection editing

Local color changes
 Given an original color image and a selection
 , two
differently colored versions of this image can be mixed
seamless:
 One version provides the destination functions f *
outside  ,the other one provides the source function g to be
modified within  according to (10)
 Poisson editing frees the user from the tedium of precise
selection
Local color changes

Given a source color image g
 (a) the destination image f
*
is set to be the luminance
channel from g,
 (b) the user selects a region  containing the object,
and this may be some what bigger than the actual
object,
 (c) the Poisson equation (10) is solved in each color
channel.
Local color changes
Selection editing

Seamless tiling
 When the domain
 is rectangular, its content can be
made tileable by enforcing periodic boundary
conditions with the Poisson solver.
 The source image g is the original image, and the
boundary conditions are derived from the boundary
values of g, such that opposite sides of the rectangular
domain correspond t identical Dirichlet conditions.
 In Fig. 12, we have chosen
f *north  f * south  0.5( g north  g south)
,and similarly for the east and west borders.
Seamless tiling
Conclusion
 Using
the generic framework of guided
interpolation, we have introduced a variety
of tools to edit in a seamless and effortless
manner the contents of an image selection.
 An important common characteristic of all
these tools is that there is no need for
precise object delineation, in contrast with
the classic tools that address similar tasks.