Application of GRNN neural network in non texture image inpainting and restoration

Vahid K. Aliloua, FarzinYaghmaee Pattern Recognition Letters Volume 62, 1 September 2015, Pages 24–31 Reporter : 陳彥鈞、邱麟捷 1

Outline

 Introduction  General regression neural network Definition  GRNN-based image inpainting  Parameter settings  Experiments and comparisons  Conclusion 2

Introduction(1/5)

 Image inpainting is the technique of restoring and repairing lost parts of an image by using the information of their surrounding areas. It is a powerful technique for restoring old and scratched pictures or artworks  From the mathematical point of view, inpainting is essentially an interpolation problem 3

Introduction(2/5)

 Formally, the inpainting problem can be expressed in this way: given a corrupted image

I

pixel inside Ω with some missing region Ω , fill-in each with a value inferred from Ω 𝑐 .

4

Introduction(3/5)

 Recently, some researchers proposed to use artificial neural networks for inpainting applications  Artificial neural networks (ANNs) are one of the most powerful and popular tools for black-box modeling and are designed and inspired by real biological neural networks which have been applied in many fields, such as automotive, banking, electronics, financial, manufacturing and robotics 5

Introduction(4/5)

 In this paper, we present a method to fill-in the missing or damaged areas of images by using GRNN network .

 The missing regions are first separated and sorted according to their size. Then the algorithm proceeds with applying a GRNN network to each one in order to repair their damaged pixels. 6

Introduction(5/5)

7

General regression neural network (1/3)  The general regression neural network (GRNN) is a variation of radial basis neural networks which is designed for function approximation and regression.  It is established based on a one-pass learning algorithm and does not require to learn via the error back-propagation procedure of the training data  Example ： 𝑦 = 𝑓 𝑥 8

General regression neural network (2/3) 𝐷 𝑖 2 = 𝑋 − 𝑋 𝑖 𝑛 𝑖=1 𝑦 𝑖 × exp −𝐷 𝑖 2 2𝜎 2 𝑛 𝑖=1 exp 𝑇 × 𝑋 − 𝑋 𝑖 −𝐷 𝑖 2 2𝜎 2 9

General regression neural network (3/3) 10

GRNN-based image inpainting (1/3) 11

GRNN-based image inpainting (2/3) 12

GRNN-based image inpainting (3/3) 13

Parameter settings(1/3)  GRNN based inpainting of images enabled us to bypass the difficulty of working with complex mathematical models of PDEs and variational formulations.  The only parameter in this approach is to determine σ  If there exists a strong edge or an isophote that hits the boundary of the missing area, the value of σ must be small On the other hand, if we do not have much variation over the boundary pixels, we can choose larger value of σ . 14

Parameter settings(2/3)  We formulated the following equation to find the optimal spread parameter: 𝜎 = 𝑒 1− 𝜌 255  in which

ρ

is the maximum magnitude of the gradient of the boundary and is defined as:  𝜌 = 𝑚𝑎𝑥 𝐺 𝑥, 𝑦 , 𝑥, 𝑦 ∈ 𝜑 𝑖 and G is the magnitude of the gradient and can be calculated by the following equation: 𝐺 𝑥, 𝑦 = 𝐼 𝑥 𝑥, 𝑦 2 + 𝐼 𝑦 𝑥, 𝑦 2 15

Parameter settings(3/3)  And G is the magnitude of the gradient and can be calculated by the following equation: 𝐺 𝑥, 𝑦 = 𝐼 𝑥 𝑥, 𝑦 2 + 𝐼 𝑦 𝑥, 𝑦 2 where 𝐼 𝑥 and 𝐼 𝑦 are the gradients in

x

and

y

directions respectively.

16

Experiments and comparisons (1/2) 17

Experiments and comparisons (2/2) 18

Conclusion  GRNN networks are very fast at converging to the optimal regression surfaces using only a few number of data samples  The advantages of our algorithm are: (1) it is simple in principle; (2) it is easy to implement; (3) it is qualitatively better; (4) it is computationally efficient 19