Transcript LA TEORIA DELLA REGOLARIZZAZIONE NELLA COMPUTER …

REGULARIZATION THEORY OF INVERSE PROBLEMS
- A BRIEF REVIEW -
Michele Piana, Dipartimento di Matematica, Università di Genova
PLAN
• Ill-posedness
• Applications
• Regularization theory
• Algorithms
SOME DATES
1902 (Hadamard) - A problem is ill-posed when its solution is not
unique, or it does not exist or it does not depend
continuously on the data
Early sixties - ‘…The crux of the difficulty was that numerical
inversions were producing results which were
physically unacceptable but were mathematically
acceptable…’ (Twomey, 1977)
1963 (Tikhonov) - One may obtain stability by exploiting additional
information on the solution
1979 (Cormack and Hounsfield) – The Nobel prize for Medicine and
Physiology is assigned ‘for the
developement of computed assisted
tomography’
EXAMPLES
• Differentiation:
f1 ( x )  f ( x ) f 2 ( x )  f ( x )   sin( x )
Edge-detection is ill-posed!
• Image restoration:
g : blurred image
k : response function
| f1 ( x)  f 2 ( x) | 
| f1( x )  f 2( x ) |  
f : unknown object
g( x)  (k  f )( x)
k band-limited: invisible objects exist
Existence if and only if
ˆ ( ) |2 d  
ˆ
|
g
(

)
/
k

• Interpolation
Given
l
z  ( xi , yi )  X  Y i 1 find
such that f ( xi )  yi
i  1,...l
f : X Y
WHAT ABOUT LEARNING?
Learning from examples can be regarded as the problem
of approximating a multivariate function from sparse data
Data: the training set
z  ( xi , yi ) , xi  X yi Y 
obtained by sampling
l
i 1
X Y
according to some
probability distribution
Unknown: an estimator f : X  Y such that f (x ) predicts
y with high probability
Is this an ill-posed problem?
Is this an inverse problem?
Next talk!
MATHEMATICAL FRAMEWORK
H, K Hilbert spaces; A : H  K linear continuous:
find f given a noisy g such that g  Af
Ill-posedness: Ker ( A)  0 or R( A)  K or
A
1
is unbounded

• Pseudosolutions:  u , Au  g  min  or u , A Au  A g

• Generalized solution u  : the minimum norm pseudosolution
• Generalized inverse
Finding
u

A : A g  u 
is ill-posed
( A
bounded
Remark: well-posedness does not imply stability
R(A) closed
u 
u

 A A

g
g
)
REGULARIZATION ALGORITHMS
A regularization algorithm for the ill-posed problem
is a one-parameter family
R 
g  Af
of operators such that:
•  R : K  H is linear and continuous

• lim  0 R Af  A Af f  X
• Semiconvergence: given g noisy version of Af


R
g

A
Af

R
g

A
Af
such
that
opt
 opt

there exists

Tikhonov method
Af  g
2
K
 f
Two major points:
2
H
 min
R  ( A A  I )1 A
1) how to compute the minimum
2) how to fix the regularization parameter
COMPUTATION
Two ‘easy’ cases:
•
A is a convolution operator with kernel k
kˆ ( ) gˆ ( )
ˆ
( R g )( ) 
2
ˆ
k ( )  
•
A
is a compact operator
Singular system: Au j   j v j

j
j 1
 
R g  
2
j
( g, v j )u j
Av j   j u j
THE REGULARIZATION PARAMETER
Basic definition:
Let  be a measure of the amount of noise affecting the datum
Then a choice    ( ) is optimal if
lim  0 R  g  u


lim  0  ( )  0
Example: Discrepancy principle: solve
AR g  g  
• Generalized to the case of noisy models
• Often oversmoothing
Other methods: GCV, L-curve…
g:
ITERATIVE METHODS
Iterative methods can be used:
• to solve the Tikhonov minimum problem
• as regularization algorithms
In iterative regularization schemes:
• the role of the regularization parameter is played by the
iteration number
• The computational effort is affordable for non-sparse matrix
New, tighter prior constraints can be introduced
Example:
f n1 
PC ( f n  A ( g  Af n ) )
C : convex subset of the source space
Open problem: is this a regularization method?
f0  0
CONCLUSIONS
• There are plenty of ill-posed problems in the applied sciences
• Regularization theory is THE framework for solving linear
ill-posed problems
• What’s up for non-linear ill-posed problems?