Slide 1

Transcript Slide 1

Unit IV. Image processing and analysis.

Part II. Morphological processing and Feature Extraction Dr. Felipe Orihuela-Espina

An apology …

 ☞ This unit contains a lot of material that I prepared when I taught Image Processing at the Universidad Autónoma del Estado de México.

 The original slides were in Spanish and thus there might remain some non-translated slides. I’m working on the translation but also on updating the examples to medical images.

 Please accept my apologies for the inconvenience this may cause.

Outline

       Geometric and topological properties Domain transformation  Fourier  Wavelets Filtering  noise removal  signal enhancement Morphological Processing  Erosion and Dilation   Opening and Closing Hit-Miss transform Feature extraction  edge detection,  Hough transform Segmentation Registration © 2015. Dr. Felipe Orihuela-Espina 3

MORPHOLOGICAL PROCESSING

Morphological processing

 Erosion and dilation   Image polarization Edge detection     Opening and closing Hit-Miss transformation Thinning and Thicknenning Skeleton extraction 30/04/2020 Dr. Felipe Orihuela Espina 5

Morphological processing



Morphological processing

is the application of set operations to images.

 …ergo it is based on  set theory Images are understood as a space Z 2 30/04/2020 Dr. Felipe Orihuela Espina 6

Morphological processing

 Often morphological processing is targeted to binary images  There are of course extensions to grayscale images since the beginning of the 90s.

 Basic morphological processing for color already exist but still under research 30/04/2020 Dr. Felipe Orihuela Espina 7

Set operations

 Union :  A  B  A OR B 30/04/2020 Dr. Felipe Orihuela Espina 8

Set operations

 Intersection:  A  B  A AND B 30/04/2020 Dr. Felipe Orihuela Espina 9

Set operations

 Complement :  Ac={w|w  A}  NOT A 30/04/2020 Dr. Felipe Orihuela Espina 10

Set operations

 Substraction :  A-B=A  Bc  A AND NOT B 30/04/2020 Dr. Felipe Orihuela Espina 11

Logic Operations

30/04/2020 Dr. Felipe Orihuela Espina 12

Logic Operations

AND

30/04/2020 Dr. Felipe Orihuela Espina 13

Logic Operations

NOT

30/04/2020 Dr. Felipe Orihuela Espina 14

Logic Operations

Substraction A-B

30/04/2020 Dr. Felipe Orihuela Espina 15

Logic Operations

Substraction B-A

30/04/2020 Dr. Felipe Orihuela Espina 16

Logic Operations

XOR

30/04/2020 Dr. Felipe Orihuela Espina 17

Traslation

(A) z ={c|c=a+z para a  A} 30/04/2020 Dr. Felipe Orihuela Espina 18

Reflection

Â={w|w=-a, para a  A}  ☞ Nota: El formalismo matem ático anterior, supone que el origen de la reflexión coincide con el origen de coordenadas. En imágenes, esto casi nunca ocurre, ya que el origen de coordenadas es la esquina superior izquierda. Sin embargo, en ventanas de interés, considerando el centro de la ventana el origen es válido.

30/04/2020 Dr. Felipe Orihuela Espina 19

Reflection

 ☞ Watch out! Reflection is NOT the same as flip whether horizontal of vertical  They coincide if the origin of the reflection is the central pixel of the image and the image is square.

30/04/2020 Dr. Felipe Orihuela Espina 20

Reflection

Origin: (10,10) Origin: (8,8) 30/04/2020 Dr. Felipe Orihuela Espina Origin: (5,5) Origin: (14,11) 21

Expand



Expand

: EXPAND:[[sigma= Pi; if (sigma>0) Q0=1; else Q0=P0]] 30/04/2020 Dr. Felipe Orihuela Espina 22

Expand

30/04/2020 Dr. Felipe Orihuela Espina 23

Dilation

 The    EXPAND operation is capable of: Dilating objects, Removing salt and pepper noise Removing “breaks” smaller than 3 pixels in objects  It is a specific case of dilation .

30/04/2020 Dr. Felipe Orihuela Espina 24

Shrink



Shrink

: SHRINK:[[sigma= Pi; if (sigma<8) Q0=0; else Q0=P0]] 30/04/2020 Dr. Felipe Orihuela Espina 25

Shrink

30/04/2020 Dr. Felipe Orihuela Espina 26

Shrink

 The  SHRINK operation is capable of: Slimming objects  Removing salt and pepper noise.

 It is a particular case of

erosion

30/04/2020 Dr. Felipe Orihuela Espina 27

Mathematical morphology

 In mathematical morphology there are 2 fundamental operators:  Dilation  Erosion 30/04/2020 Dr. Felipe Orihuela Espina 28

Minkowski’s set operations

 Addition:  Substraction: 30/04/2020 Dr. Felipe Orihuela Espina 29

Dilation



Dilation

A  B addition coincides with Minkowski’s  …or alternatively: Dr. Felipe Orihuela Espina This means B “translated” by z; i.e. the “shifting” of the second signal in a convolution. See the translation operation above.

30 30/04/2020

Dilation

Erosion



Erosion

is similar (but not equal) to substraction: Reflection  …or alternatively: Watch out! This notation is ambiguous. Above it refers to Minkowski’s operation. Below it “already” refers to erosion. In books introducing Minkowski’s operations, they often denote erosion as E(A,B) and dilation as D(A,B), whereas in thos not introducing Minkowski operations, then the  symbol is used to denote erosion. Note also that dilation is not a problem as it coincides with Minkowski addition.

30/04/2020 Dr. Felipe Orihuela Espina This means B “translated” by z; i.e.

the “shifting” of the second signal in a convolution. See the translation operation above.

Erosion

Mathematical morphology

 Note that you need a set B  Set B is known as the

element mask

structuring

 This is was unnecessary for EXPAND SHRINK and 30/04/2020 Dr. Felipe Orihuela Espina 34

Mathematical morphology

 The structuring element is the equivalent of the spatial filter mask in spatial filtering for mathematical morphology.

 Trying to erode or dilate without specifying a mask, is like trying to filter without having a filter.

 The mask determines how much is eroded or dilated.

 Different masks yield different results 30/04/2020 Dr. Felipe Orihuela Espina 35

Mathematical morphology

 Although both erosion and dilation need a mask, the way to apply the mask is different because of the definition of these operations!

 The same mask applied to the erosion and dilation operation will yield different results 30/04/2020 Dr. Felipe Orihuela Espina 36

30/04/2020 1

Identity mask

 Some authors will further indicate a 0 in non-used positions (e.g. González and Woods) whilst some others prefer to leave them blank (e.g. Davies). It is totally irrelevant Dr. Felipe Orihuela Espina 37

Mask:

Dilation

1 30/04/2020 Dr. Felipe Orihuela Espina 38

Shifting masks

1 1 30/04/2020 1 Dr. Felipe Orihuela Espina 1 39

Mask:

Dilation

1 30/04/2020 Dr. Felipe Orihuela Espina 40

Mask:

Dilation

1 30/04/2020 Dr. Felipe Orihuela Espina 41

Mask:

Dilation

1 30/04/2020 Dr. Felipe Orihuela Espina 42

Mask:

Dilation

1 30/04/2020 Dr. Felipe Orihuela Espina 43

Masks

 ☞ Beware! Applying the identity mask or any shifting mask (or in general any mask with a single element) in the erosion operation will produce incorrect results.

 They might look like dilation.

30/04/2020 Dr. Felipe Orihuela Espina 44

1 1

Asymmetry masks

1 1 1 1 1 1 Dr. Felipe Orihuela Espina They produce an erosion or dilation in a certain direction 45 30/04/2020

Mask:

Dilation

1 1 30/04/2020 Dr. Felipe Orihuela Espina 46

Mask:

Dilation

1 1 30/04/2020 Dr. Felipe Orihuela Espina 47

Mask:

Dilation

1 1 30/04/2020 Dr. Felipe Orihuela Espina 48

Mask:

Dilation

1 1 30/04/2020 Dr. Felipe Orihuela Espina 49

Isotropic masks

1 1 1 1 4-neighbours 1 1 1 1 1 30/04/2020 Perhaps the most used.

1 1 1 1 1 1 1 1 8-neighbours 1 1 1 1 1 1 1 1 1 Dr. Felipe Orihuela Espina 50

Mask:

Dilation

1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 51

Mask:

Dilation

1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 52

Mask:

Dilation

1 1 1 1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 53

Mask:

Dilation

1 1 1 1 1 1 1 1 1 30/04/2020 This corresponds to the EXPAND operation Dr. Felipe Orihuela Espina 54

Erosion

Mask: 1 1 1 1 1 1 1 1 1 30/04/2020 This corresponds to the SHRINK operation Dr. Felipe Orihuela Espina 55

Erosion

Mask: 1 1 1 1 1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 56

Erosion

Mask: 1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 57

Erosion

Mask: 1 1 30/04/2020 Dr. Felipe Orihuela Espina 58

Erosion

Mask: 1 1 30/04/2020 Dr. Felipe Orihuela Espina 59

Erosion

Mask: 1 1 30/04/2020 Dr. Felipe Orihuela Espina 60

Erosion

Mask: 1 1 30/04/2020 Dr. Felipe Orihuela Espina 61

Erosion and Dilation

 ☞ Watch out! Erosion is not the opposite or complement of dilation: Erosion(Dilation(A,B),B)  A Dilation(Erosion(A,B),B)  A 30/04/2020 Dr. Felipe Orihuela Espina 62

Erosion after dilation

Mask: 1 1 1 1 1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 63

Dilation after erosion

Mask: 1 1 1 1 1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 64

Erosion and Dilation

 Needless to say, masks do not have to be 3x3 or symmetrical.

30/04/2020 Dr. Felipe Orihuela Espina 65

Image polarization

 Image polarization refers to the relation of background and foreground.

 The same erosion operation might yield a dilation operation if we invert polarization; that is, if we use background as foreground and viceversa.

30/04/2020 Dr. Felipe Orihuela Espina 66

Image polarization

 Incorrect polarization:  The image on the left was passed directly to the erosion operation  A dilation effect is appreciated Original image from Univ. de Warwick (UK) 30/04/2020 Dr. Felipe Orihuela Espina 67

Image polarization

 Correct polarization:  Being white background, the image was inverted before erosion  The result of erosion was then inverted back for correct visualization Original image from Univ. de Warwick (UK) 30/04/2020 Dr. Felipe Orihuela Espina 68

Border detection

 Border detection in binary images becomes trivial using erosion:  …with B being 1 1 1 1 1 1 1 1 1 Dr. Felipe Orihuela Espina 30/04/2020 69

Border detection

30/04/2020 Dr. Felipe Orihuela Espina 70

Opening and Closing



Opening

:   That’s a dilation after a erosion.

Closing

: Note: It refers to erosion and dilation, not to Minkowski’s operations  That’s an erosion after a dilation 30/04/2020 Dr. Felipe Orihuela Espina 71

Opening and Closing

Figure from [Prasad M; Image proessing techniques using OpenCV. Available at: http://www.slideshare.net/roboVITics/robocv-module-4-image-processing-techniques-using-opencv] © 2015. Dr. Felipe Orihuela-Espina 72

Opening and Closing

Opening

 Opening can:   Remove pepper noise Alleviate protuberances in the image  e.g. hair in a skin image  Closing can:    Remove salt noise Alleviate object “breaks” Fill small grooves, holes and concavities.

30/04/2020 Dr. Felipe Orihuela Espina 74

Opening

30/04/2020 Dr. Felipe Orihuela Espina 75

Closing

30/04/2020 Dr. Felipe Orihuela Espina 76

Opening and closing

 Opening and closing are severely affected by image polarization  Example: Opening with incorrect polarization  Note how it actually looks like the application of closing.

30/04/2020 Dr. Felipe Orihuela Espina 77

Opening and closing

 Opening and closing are idempotents: 30/04/2020 Dr. Felipe Orihuela Espina 78

Opening and closing

 Opening and closing in detection.

 Example:  In medical image, a common appliction is the removal of hair in skin images: 30/04/2020 Dr. Felipe Orihuela Espina 79

Opening and closing

 ☞ Watch out! Opening is not the opposite or complement of closing: Opening(Closing(A,B),B)  A Closing(Opening(A,B),B)  A 30/04/2020 Dr. Felipe Orihuela Espina 80

Figure from: [Gonzalez and Woods, 2002]

Apertura y Clausura

30/04/2020 Dr. Felipe Orihuela Espina 81

Opening after closing

30/04/2020 Dr. Felipe Orihuela Espina 82

Closing after opening

30/04/2020 Dr. Felipe Orihuela Espina 83

Hit-Miss transform

 Let A be animage and B1 and B2 masks disjoint ( B1  B2=  ). The two

hit-miss transform

is defined as: 30/04/2020 Dr. Felipe Orihuela Espina 84

Hit-Miss transform

 The hit-miss transform is a template matching operation  Basically, it detects equal forms to those indicated in B1 (with 1s) ensuring they are isolated from those pixels indicated in B2 (with 1s) 30/04/2020 Dr. Felipe Orihuela Espina 85

Hit-Miss transform

 Detecting points: B 1 = 1 B 2 = 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 86

Hit-Miss transform

30/04/2020 Dr. Felipe Orihuela Espina 87

Hit-Miss transform

 Detecting (horizontal) lines B 1 = 1 1 1 B 2 = 1 1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 88

Hit-Miss transform

Beware! The line as such is not detected ; only the pattern at the contiguous pixels… 30/04/2020 Dr. Felipe Orihuela Espina 89

Hit-Miss transform

 Detecting (2x2) upper-left corners B 1 = 1 1 1 1 B 2 = 1 1 1 1 1 30/04/2020 Dr. Felipe Orihuela Espina 90

Hit-Miss transform

Note how only the “centers” of the corners are detected and not the full corners 30/04/2020 Dr. Felipe Orihuela Espina 91

Hit-Miss transform

 With the hit-miss transform it is trivial to detect a particular shape in an image  It suffices to specify the correct template  Detection is dependent on scale and rotation.

 Masks do not have to be 3x3; they can have any size 30/04/2020 Dr. Felipe Orihuela Espina 92

Hit-Miss transform

 We can of course remove the detected patterns: 30/04/2020 Dr. Felipe Orihuela Espina 93

Hit-Miss transform

30/04/2020 Dr. Felipe Orihuela Espina 94

Hit-Miss transform

 To detect rotated or scaled or deform versions of the pattern:  We need to define the masks pairs for all possible variations of the shape  Then we take the union of the individual hit miss transforms 30/04/2020 Dr. Felipe Orihuela Espina 95

Adelgazamiento y Engrosamiento

 Adelgazamiento y engrosamiento, as í como la extracción del esqueleto, son operaciones que permiten el análisis de la conectividad de una imagen .

 Podemos analizar la conectividad que deseemos  4 conectividad, 8 conectividad, m conectividad, otras… 30/04/2020 Dr. Felipe Orihuela Espina 96

Adelgazamiento

 Es una operaci ón de erosión que mantiene la conectividad.

 No permite que los objetos se desvanezcan  Genera una l ínea mínimamente conectada dentro de los objetos a distancia equidistante de los bordes.

30/04/2020 Dr. Felipe Orihuela Espina 97

Adelgazamiento

Imagen obtenida de: www.mcm-design.dk

98 30/04/2020 Dr. Felipe Orihuela Espina

Adelgazamiento

 El

adelgazamiento

se puede definir en t érminos de la transformada éxito-fallo Thin(A,B1,B2)= A - HM(A,B1,B2) 30/04/2020 Dr. Felipe Orihuela Espina 99

Adelgazamiento

 La operaci ón adelgazamiento rara vez se aplica sola.

 Para obtener un adelgazamiento simétrico normalmente se usa una secuencia de adelgazamientos por refinamiento sucesivo 30/04/2020 Dr. Felipe Orihuela Espina 100

1 x 1 1 1 x 30/04/2020

M áscaras para Adelgazamiento simétrico

1 1 x 1 1 x 1 1 1 x 1 x X - condición de No importa x 1 1 1 1 x x 1 1 1 1 x x 1 x 1 1 1 Dr. Felipe Orihuela Espina x 1 1 x 1 1 x 1 1 1 x 1 101

Adelgazamiento

30/04/2020 Dr. Felipe Orihuela Espina Imagen obtenida de: www.owlnet.rice.edu

102

Adelgazamiento

30/04/2020 Imagen obtenida de: Universidad de Edimburgo homepages.inf.ed.ac.uk

Dr. Felipe Orihuela Espina 103

Engrosamiento

 El

engrosamiento

se puede definir en t érminos de la transformada éxito-fallo Thick(A,B1,B2)= A  HM(A,B1,B2)  De menor relevancia que la operaci ón de adelgazamiento  Suele utilizarse en secuencia 30/04/2020 Dr. Felipe Orihuela Espina 104

Engrosamiento

 Rara vez se utiliza en la pr áctica [GonzalezYWoods]  Permite obtener la cáscara convexa (convex hull) de un elemento  …pero hay mejores algoritmos para esto… 30/04/2020 Dr. Felipe Orihuela Espina 105

Extracci ón del esqueleto

 El esqueleto de una imagen produce un punto central o l ínea(s) para cada objeto en la imagen  Constituye la estructura interna de los objetos  Es la base de OCR 30/04/2020 Dr. Felipe Orihuela Espina 106

Extracci ón del esqueleto



Muy similar

a la operaci ón de

adelgazamiento

 Aunque la operaci ón de adelgazamiento puede dejar píxeles que se podrían eliminar sin afectar a la conectividad  Ya que el adelgazamiento es más sencillo, se suele tomar como una aproximación.

30/04/2020 Dr. Felipe Orihuela Espina 107

Extracci ón del esqueleto

 La definici ón informal del esqueleto es una línea representante de un objeto que es:    De 1 p íxel de grosor Pasa por la “mitad” del objeto Preserva la topología del objeto 30/04/2020 Dr. Felipe Orihuela Espina 108

Extracci ón del esqueleto

 El esqueleto puede ser expresado en t érminos de erosión y apertura [Serra,1982] 

(

) 

k K

  0  (



)  (



) 

 donde K es hasta una iteración antes de que la erosión de cómo resultado el conjunto vacío

(



)



(...(



)



)...



)

30/04/2020 Dr. Felipe Orihuela Espina 109

Extracci ón del esqueleto

 Otra forma de pensar en el esqueleto es como “el lugar de los centros de c írculos bi tangentes que se pueden ajustar completamente dentro del objeto” Imagen obtenida de: Universidad de Edimburgo homepages.inf.ed.ac.uk

30/04/2020 Dr. Felipe Orihuela Espina 110

Extracci ón del esqueleto

 La extracci ón del esqueleto es muy sensible a pequeños cambios en la imagen original 30/04/2020 Imagenes obtenida de: Universidad de Edimburgo homepages.inf.ed.ac.uk

Dr. Felipe Orihuela Espina esqueleto 111

original

Extracci ón del esqueleto

original esqueleto 30/04/2020 esqueleto 112 Dr. Felipe Orihuela Espina

Extracci ón del esqueleto

 Si quieres jugar un poco y aprender m ás, aqu í puedes encontrar una demo…  Universidad de Edimburgo (UK)  http://homepages.inf.ed.ac.uk/rbf/HIPR2/skele tondemo.htm

30/04/2020 Dr. Felipe Orihuela Espina 113

FEATURE EXTRACTION

Feature extraction

Detecci ón de discontinuidades Detección de filos Transformada de Hough Descriptores de entorno y región 30/04/2020 Dr. Felipe Orihuela Espina 115

Detecci ón de discontinuidades

en escala de grises

 Para variar usaremos m áscaras… 30/04/2020 M4 M3 M2 M5 M0 M1 M6 M7 M8 Máscara de gradiente R es una imagen intermedia, no la imagen Q de salida

0  8 

 0



Donde Pi son los píxeles de la imagen de entrada Dr. Felipe Orihuela Espina 116 

Detecci ón de puntos

 Se aplica la m áscara de gradiente -1 -1 -1 -1 8 -1 -1 -1 -1  …y luego simplemente se buscan aquellos puntos que son mayores a un umbral dado Q0(|R0|  8)=1 30/04/2020 Dr. Felipe Orihuela Espina 117

Detecci ón de puntos por segmentación

30/04/2020 Dr. Felipe Orihuela Espina 118

Detecci ón de líneas

 Se aplican las m áscaras de gradiente Horz.

Vert.

30/04/2020 -1 2 -1 -1 2 -1 -1 2 -1 -1 -1 -1 2 2 2 -1 -1 -1 -1 -1 2 -1 2 -1 2 -1 -1 45 o 2 -1 -1 -1 2 -1 -1 -1 2 -45 o Dr. Felipe Orihuela Espina 119

Detecci ón de líneas

 Y luego se busca aquellos que sean mayores que un umbral en algunas de las imágenes intermedias Q0(|R H 0|  t || |R v 0|  t || |R 45 0|  t || |R -45 0|  t)=1  Donde t es un umbral dado.

30/04/2020 Dr. Felipe Orihuela Espina 120

Detecci ón de líneas

 Este m étodo sólo busca líneas de 1 píxel de grosor.

 Podemos detectar s ólo una dirección si se aplica sólo una máscara de gradiente, o sólo las direcciones que queramos… 30/04/2020 Dr. Felipe Orihuela Espina 121

Detecci ón de líneas

Threshold=4 Observa que detecta líneas también en el fondo 30/04/2020 Dr. Felipe Orihuela Espina 122

Detecci ón de filos

 La detecci ón de filos es la forma más común de realizar la segmentación.

 Existen muchas técnicas que podemos agrupar en dos tipos:   Aproximaci ón de plantillas Gradiente diferencial (template matching, TM ) (differential gradient DG ) 30/04/2020 Dr. Felipe Orihuela Espina 123

Detecci ón de filos

 Ambas TM y DG buscan encontrar donde la magnitud del

gradiente g

es suficientemente grande para indicar un filo.

 TM es m ás sencilla y más rápida pero más inexacta que DG 30/04/2020 Dr. Felipe Orihuela Espina 124

Detecci ón de filos

 Perfil de niveles de gris  Funci ón continua que representa la intensidad del gris a lo largo de una l ínea de píxeles (no necesariamente horizontal)  Si fuese ideal sería un escalón…  Lo normal es que sea una rampa 30/04/2020 Dr. Felipe Orihuela Espina 125

Detecci ón de filos

 La pendiente de la rampa es inversamente proporcional al grado de difuminado (blur) de la imagen 30/04/2020 Dr. Felipe Orihuela Espina 126

Detecci ón de filos

 El grosor del filo, no es de 1 p íxel, si no que depende de la pendiente de la rampa  Imágenes más difuminadas resultan en filos más gruesos  Imágenes más nítidas resultan en filos más delgados  El filo es un conjunto de píxeles conectados.

30/04/2020 Dr. Felipe Orihuela Espina 127

Detecci ón de filos

 La primera derivada (

gradiente

) de la funci ón de perfil de niveles de gris permite detectar cuando hay una discontinuidad fuerte, i.e. Donde hay un filo.

30/04/2020 Dr. Felipe Orihuela Espina 128

Detecci ón de filos por TM

 Se aplican



m áscaras de convolución normalmente está entre 8 y 12  La magnitud del gradiente

se calcula como el máximo de las distintas máscaras g=max(g i ,i=1..n) 30/04/2020 Dr. Felipe Orihuela Espina 129

Detecci ón de filos por TM

 La orientaci ón del filo  se estima simplemente como la de la máscara que produce el valor máximo 30/04/2020 Dr. Felipe Orihuela Espina 130

Detecci ón de filos: Máscaras de gradiente comunes

 Prewitt  Kirsch  Robinson 30/04/2020 Dr. Felipe Orihuela Espina 131

M áscaras de gradiente para TM

 Prewitt 1 1 1 135 o 1 1 -2 -1 -1 -1 90 o 1 1 1 1 -2 1 -1 -1 -1 45 o 1 1 -1 -2 -1 -1 1 1 1 0 o -1 1 -1 -2 -1 1 1 1 1 30/04/2020 1 1 -1 1 1 -2 -1 1 180 o -1 1 1 1 225 o -1 -1 -2 -1 1 1 270 o -1 -1 -1 1 1 -2 1 1 1 Dr. Felipe Orihuela Espina -1 -1 1 -1 -2 1 1 1 315 o 1 132

M áscaras de gradiente para TM

 Kirsch 135 o 5 5 -3 5 0 -3 -3 -3 -3 90 o 5 5 5 -3 0 -3 -3 -3 -3 45 o -3 5 5 -3 0 5 -3 -3 -3 0 o -3 -3 -3 0 -3 -3 5 5 5 30/04/2020 5 -3 -3 5 0 -3 5 -3 -3 180 o 5 5 225 0 5 o -3 -3 -3 -3 -3 270 o -3 -3 -3 -3 5 0 5 -3 5 Dr. Felipe Orihuela Espina -3 -3 -3 -3 0 5 -3 5 315 o 5 133

M áscaras de gradiente para TM

 Robinson “3 niveles” 30/04/2020 1 1 0 135 o 1 0 0 -1 -1 -1 90 o 1 1 1 0 0 0 -1 -1 -1 0 45 1 -1 0 -1 -1 o 1 0 1 0 1 0 180 o -1 -1 -1 0 1 1 225 0 1 o -1 -1 -1 0 270 o -1 -1 -1 0 1 0 1 0 1 Dr. Felipe Orihuela Espina 1 1 0 -1 -1 -1 0 o 0 0 0 1 1 1 -1 -1 0 -1 0 1 0 1 315 o 1 134

M áscaras de gradiente para TM

 Robinson “5 niveles” 30/04/2020 2 1 0 135 o 1 0 0 -1 -1 -2 90 o 1 2 1 0 0 0 -1 -2 -1 0 45 1 -1 0 -2 -1 o 1 0 2 0 1 0 180 o -1 -2 -1 0 1 2 225 0 1 o -1 -2 -1 0 270 o -1 -2 -1 0 1 0 2 0 1 Dr. Felipe Orihuela Espina 2 1 0 -1 -2 -1 0 o 0 0 0 1 2 1 -2 -1 0 -1 0 1 0 1 315 o 2 135

Detecci ón de filos por DG

 La magnitud del gradiente g se calcula de forma vectorial con la transformaci ón



g x

2 

g y

2 Dr. Felipe Orihuela Espina 136

Detecci ón de filos por DG

 La orientaci ón del filo se estima por la ecuación  

arctan(

g y

/

g x

)

 30/04/2020 Dr. Felipe Orihuela Espina 137

Detecci ón de filos: Máscaras de gradiente comunes

 Roberts  Prewitt  Sobel 30/04/2020 Dr. Felipe Orihuela Espina 138

M áscaras de gradiente para DG

 Roberts 30/04/2020 Gx.

-1 0 0 1 No tienen un centro claramente definido 0 1 -1 0 Gy.

Dr. Felipe Orihuela Espina 139

M áscaras de gradiente para DG

 Prewitt Gx Horz Gy Vert.

30/04/2020 -1 0 1 -1 0 1 -1 0 1 -1 -1 -1 0 0 0 1 1 1 -1 -1 0 -1 0 1 0 -1 -1 1 0 -1 Dr. Felipe Orihuela Espina 0 1 1 1 1 0 45 o -45 o 140

M áscaras de gradiente para DG

 Sobel Gx Horz.

Gy Vert.

30/04/2020 -1 0 1 -2 0 2 -1 0 1 -1 -2 -1 0 0 0 1 2 1 -2 -1 0 -1 0 1 0 -1 -2 1 0 -1 Dr. Felipe Orihuela Espina 0 1 2 2 1 0 45 o -45 o 141

Gradiente

Prewitt 30/04/2020 Dr. Felipe Orihuela Espina 142

Gradiente

Sobel 30/04/2020 Dr. Felipe Orihuela Espina 143

Gradiente

Prewitt 30/04/2020 Dr. Felipe Orihuela Espina 144

Gradiente

Sobel 30/04/2020 Dr. Felipe Orihuela Espina 145

Gradiente

Prewitt vs Sobel 30/04/2020 Dr. Felipe Orihuela Espina 146

Gradiente

Orientation 30/04/2020 Dr. Felipe Orihuela Espina 147

Gradiente

Orientation 30/04/2020 Dr. Felipe Orihuela Espina 148

Hough Transform



Hough transform

permits a change in representation; from Cartesian to Polar * .

 With a clever trick, algorithms can be developed to detect virtually any shape (as long as it can be expressed mathematically)  …this include; lines, circles, ellipses, etc  ☞ For the sake of time; we will only see the detection of lines.

*As we will see, each point in the Cartesian space produces a (sinusoidal) curve in the Hough space; expressed in Polar coordinates.

30/04/2020

Digital line

 Parameterization of a line:  = x·cos  + y·sen   Note that a line in the X-Y plane becomes a single point in the (  ,  ) “polar” space.

Dr. Felipe Orihuela Espina 150

Digital line

 To convert a line expressed in the “classical” slope interception form: y = mx + b  to its polar representation: 30/04/2020 m = tan(90+  ) b=  ·sen(90)/sen(  )  by the sine theorem ☞ In case you don’t remember about slopes, tangents and the sine theorem, check the back up slides.

Dr. Felipe Orihuela Espina 151

Digital line

 Polar parameterization of the line simplifies the necessary computation to detect lines when compared to the classical slope-interception parameterization  The most obvious case are the as these have infinite slope.

“vertical” lines, [Duda72] 30/04/2020 Dr. Felipe Orihuela Espina 152

Hough Transform

 Basically, the

Hough transform

is the conversion from a Cartesian representation of a pixel (x,y) to a polar space (  ,  ) such that:  = x·cos  + y·sen   Since infinite lines can cross over this pixel, the pixel actually maps to a whole sinusoidal curve in the Hough (polar) space.

Hough Transform



Hough transform

algorithm is a technique to extract arbitrary forms (features) from an image * .

 The basic idea of the algorithm takes two steps: 1.

Look for “imperfect” instances of the form Reconcile the set of imperfect instances *Well, the (basic) Hough tranform just detects lines; but this idea was later generalized [Ballard, 1981] so that it can detect any arbitrary shape.

Hough Transform



Line detection

 with the Hough transform First we interrogate an arbitrary set of points (pixels).

 It does not matter at this point how this set were chosen;  they may have initialize by an edge detector; or simply “all” points in the foreground are transformed (both options are common)  We would like to know if these pixels form a line (i.e. they are collinear).

Transformada de Hough

 Stage 1 : Search all possibles lines crossing over each of the points of the set  For each point 1.

Draw a number of arbitrary lines through the point at different angles (continuous lines).

 For each of these lines, draw a perpendicular line to this one crossing through the coordinate origin (dashed lines).

 Finally, get the distance origin to the original (continuous) line, as well as the angle with the abscissa axis.

θ from the formed by the perpendicular (dashed) line Beware! The closest point to the original line does not necessarily coincide with the original point!

 In other words; get the Hough representation of the points.

Transformada de Hough

 Stage 2 Reconcile the instances.

Generate the Hough space with the Hough representation of all points This is called Hough space or accumulator space. Although conceptually polar coordinates; it is represented orthogonally with θ (angle) as the abscissa and  (distance) as the ordinate Figure from: [http://en.wikipedia.org/wiki/Hough_transform] © 2015. Dr. Felipe Orihuela-Espina 157

Transformada de Hough

 Stage 2 Reconcile the instances.

Finally, if there is any common crossing point in the Hough space, this point correspond to the description of a line along which the original points are collinear in the original space ( that’s the image) Collinear points in the original image intersect in peaks ( θ ,  ) in the Hough transform space.

Hough transform

C D X-ray image of a mouse femoral bone (A) and the edges detected (B). Image B shows the edges as white lines and line segments superimposed over the Sobel image. (C) Hough transform of the edges. (D) The inverse Hough transform of the local maxima in C superimposed on the original X-ray image Figure from: [http://what-when-how.com/biomedical-image-analysis/detecting-lines and-edges-with-the-hough-transform-biomedical-image-analysis/] © 2015. Dr. Felipe Orihuela-Espina 160

Hough transform



Circle detection

with the Hough transform  A circle is described by three parameters: radius (x c ,y c ) : r and the center coordinates r 2 =(x-x c ) 2 +(y-y c ) 2  Therefore, the Hough space for circles is 3D!

 A pixel is projected as a cone in Hough space since a cone satisfies the equation for increasing r .

Hough transform

© 2015. Dr. Felipe Orihuela-Espina Detection of the pupil and iris with the Hough transform: (A) the original eye photo and (B) the edge image obtained by filtering, edge detection, and thresholding. (C) One plane in Hough space. In this plane (r = 60), the cones from the pupil pixels intersect and form a local maximum (arrow). The inverse Hough transform of this local maximum leads to the smaller circle in (D). Although the smaller circle represents the pupil very well, the larger iris circle is affected by various conflicting edges caused, for example, by the eyelids. As a result, the circle is smaller than the actual iris.

Figure from: [http://what-when how.com/biomedical-image analysis/detection-of-circles-and ellipses-with-the-hough-transform biomedical-image-analysis/] 162

Generalized Hough transform

Ultrasound image of a gallbladder. A reference point r is defined, and the lines connecting each edge segment with the reference point are entered in the R table. When a shape is analyzed, each edge is displaced by all vectors in the R table in sequence.

Hough transform

 Algorithms based on the Hough transform are:    Often very efficient Tolerant to gaps in feature boundary descriptions Relatively unaffected by image noise.

 One disadvantage of the line Hough transform is that the lines are defined by two parameters only and therefore extend over the entire image. Additional steps are necessary to trim the lines.

Hough transform

 ☞  We won’t get into further details. To know more: Ballard BH (1981) Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognition 13(2):111 –122  Illingworth J and Kittler, J (1988) A Survey of the Hough Transform. Computer Vision, Graphics and Image Processing 44:87-116  Yuen HK, Illingworth J and Kittler, J (1988) Ellipse detection using the Hough transform  …does what says on the tin.

 Ashok A and Edwards J (1997) Generalized Hough Transform for natural shapes. Pattern Recognition Letters 18:473-480  Extension to forms which are similar but not identical © 2015. Dr. Felipe Orihuela-Espina 165

Representations and Descriptors



Representations

are different ways of encoding information  They aim at reducing information from thousands of pixels to a few data,  …which is of course more tractable computationally  Many common representations target the border of objects 

Descriptors

are quantities –i.e. variables- (also referred to as

features

) derived from the representation  They aim at permitting unambiguous description of a set of pixels or region representing an object.

 Once a representation is chosen, the object is described by its descriptors.

30/04/2020 Dr. Felipe Orihuela Espina 166

Representations

 Often * , a good representation is invariant to changes in:  Size ( Scale )  Displacement ( Traslation )  Angle ( Rotation ) 30/04/2020 * Of course, this STRONGLY depends on the specific problem at hand Dr. Felipe Orihuela Espina 167

Contour representations

 Some basic contour representations:     Chain codes Minimum perimeter polygons Signatures Contour segments Skeletons are shape descriptors.

30/04/2020 Dr. Felipe Orihuela Espina 168

Contour representations



Chain codes

:  Chain codes represent the contour as a vector sequence 30/04/2020 Dr. Felipe Orihuela Espina 169

Contour representations



Chain codes

:  Vectors are coded according to their orientations.

 4- or 8- connectivity can be used.

30/04/2020 Dr. Felipe Orihuela Espina 170

Contour representations



Chain codes

:  An starting point in the border is chosen  The contour (border) is expressed as the sequence of the vector orientations 0,0,0,7,7,…,2,1,0,2 30/04/2020 Dr. Felipe Orihuela Espina 171

Contour representations



Chain code

:  Chains are often very long  Reduction in information size is almost negligible  Variations due to noise or imprecise segmentations affect the chain   It is not invariant to scale Strictly, it is not invariant to rotation neither  …but rotation invariance can be achived by taking first derivatives (first differences).

 Starting pixel selection can be cumbersome 30/04/2020 Dr. Felipe Orihuela Espina 172

Contour representations



Minimum perimeter polygons (MPP)

:  They aim at reducing the border to a sequence of segments  The basic idea is like a board over which we nail down some nails and then we use a rubber band over the nails;   The rubber band “adapts” to the nails’ shape.

The rubber band defines the minimum perimeter polygon 30/04/2020 Dr. Felipe Orihuela Espina 173

Contour representations



Minimum perimeter polygons:

 In MPP, the “nails” are located over an arbitrary grid  Grids with bigger/wider cells yield shorter descriptions (i.e. less segments), but at the cost of larger errors.

 In the limit case, the grid coincide with the original pixels, and then MPP then becomes equivalent to the chain code, where segments only go from one pixel to its connected neighbour 30/04/2020 Dr. Felipe Orihuela Espina 174

Contour representations

 A bit more formally;  A

minimum-perimeter polygon (MPP)

complex J is any polygon p having J of a as an image, and such that there is no polygon of shorter perimeter also having J as an image.  ☞ Beware! Maths behind MPPs are not so easy. If you want to know more:  [Sklansky et al (1972) IEEE Trans. On Computers, C-21(3):260-268] © 2015. Dr. Felipe Orihuela-Espina 175

Contour representations



Minimum perimeter polygons

 : Grid of 3 pixels grid with an offset of 1 pixels.

30/04/2020 Dr. Felipe Orihuela Espina 176

Contour representations



Minimum perimeter polygons

 : Grid of 6 pixels grid with an offset of 2 pixels.

30/04/2020 Dr. Felipe Orihuela Espina 177

Contour representations



Minimum perimeter polygons

: 30/04/2020 Left: Original shape. Right: The MPP using a grid coinciding with the original pixel size.

Figure from: [GonzalezWoods,2002] Dr. Felipe Orihuela Espina 178

Contour representations



Minimum perimeter polygons :

 Robust to (small) noise in contour.

 It is not invariant to scale  Selection of grid is critical and might affect rotation.

30/04/2020 Dr. Felipe Orihuela Espina 179

Contour representations



Signatures:

 A centroid is chosen and the contour is expressed as the function: r=f(  ) 30/04/2020 Dr. Felipe Orihuela Espina 180

Contour representations



Signatures

:   Are invariant to traslation Can be made invariant to scale simply normalizing the function to an output range [0,1]  Can be made invariant to rotation departing axis the first eigenvector taking the  Disadvantage: The description of function r=f(  ) can be really complex 30/04/2020 Dr. Felipe Orihuela Espina 181

Contour representations



Contour segments

 Basically, the convex hull.

 Based on Delaunay tesellation  Convex Hull or Alpha Shape  A coarse approximation to convex hull can be obtained through dilation 30/04/2020 Dr. Felipe Orihuela Espina 182

Contour representations



Contour segments

30/04/2020 Figure from: [Orihuela-Espina,2005, PhD thesis] Dr. Felipe Orihuela Espina 183

Contour representations



Contour segments

30/04/2020 3D PCA of an original 5D space with 181.000 points Figure from: [Orihuela-Espina,2005, PhD thesis] Dr. Felipe Orihuela Espina 184

Descriptors

 Some basic descriptors:        Border length Object diameter Major and minor axes Basic rectangle Shape numbers Fourier descriptors …and of course many others 30/04/2020 Dr. Felipe Orihuela Espina 185

Descriptors



Border length

 Number of pixels in the border  

Object diameter



(

)



max



(

p i

,

p j

)

 

,



 where D(pi,pj) is the distance between two pixels in the border  Not necessarily Euclidean 30/04/2020 Dr. Felipe Orihuela Espina 186

Descriptors



Major axis

 The segment connecting the two points defining the diameter 

Minor axis

 The segment perpendicular to the major axis and that defines the minimum rectangle enclosing the object.



Basic rectangle

 The minimum rectangle enclosing the object; defined by the major and minor axis.

30/04/2020 Dr. Felipe Orihuela Espina 187

Descriptors



Shape numbers

 Number of clockwise directional changes of the border.

 Obtained from the chain code 30/04/2020 Dr. Felipe Orihuela Espina 188

Descriptors



Fourier descriptors

 Based on Fourier decomposition 

(

)  



c n e jn

 0

 Fourier descriptors are the coefficients c n the Fourier series.



c n

 1



(

)

e jn

 0

of 30/04/2020 Dr. Felipe Orihuela Espina 189 

Descriptors



Fourier descriptors

 The different pixels (x,y) of the border are expressed like comples numbers x+jy , and are understood as a sequence (discrete signal)  The border can be reconstructed using the inverse Fourier transform 30/04/2020 Dr. Felipe Orihuela Espina 190

Descriptors

Figure from: [GonzalezWoods,2002] 30/04/2020 Dr. Felipe Orihuela Espina 191

Descriptors



Fourier descriptors

 Are invariant to rotation (as long as the starting point is the same)     Are invariant to traslation Are invariant to scale (following normalization) Achieve large information reduction The border can be reconstructed as accurately as wanted  Thank to the FFT (Fast Fourier Transform) they no longer mean a computational burden 30/04/2020 Dr. Felipe Orihuela Espina 192

THANKS, QUESTIONS?

BACK UP

Sine theorem



Sine theorem

:  The sides of a triangle are proportional to the sines of the opposite angles.

Slope of a line

Remember; the tangent of an angle is: tan(θ)=sin(θ)/cos(θ) …yep! That’s the slope of the line Figure from: [http://labman.phys.utk.edu/phys135/modules/m1/derivatives.htm] © 2015. Dr. Felipe Orihuela-Espina 196