Lucia Ballerini

Digital Image Processing

Segmentation

Lecture 8 Course book reading: GW 10

Image analysis

Image Data

Pre-processing, image enhancement Binary operations Classification and matching

Segmentation

"What are the objects to be analyzed?" Morphological operations and feature extraction

Segmentation

► Full segmentation: Individual objects are separated from the background and given individual ID numbers (labels). ► Partial segmentation: separating objects from background) to speed up the further processing. The amount of data is reduced (usually by ► Segmentation is often the most difficult problem to solve in the process; there is no universal solution!

► The problem can be made much easier if solved in cooperation with the constructor of the imaging system (choice of sensors, illumination, background etc) .

Three types of segmentation

► Classification – Based on some similarity measure between pixel values. The simplest form is thresholding.

► Edge-based – Search for edges in the image. They are then used as borders between regions ► Region-based – Region growing, merge & split

Common idea:

search for discontinuities or/and similitudes in the image

Thresholding

global or local ► global: based on some kind of histogram: grey-level, edge, feature etc.

 Lighting conditions are extremely important, and it will only work under very controlled circumstances. ► Fixed thresholds: the same value is used in the whole image ► local (or dynamic thresholding): image. The image is divided into overlapping sections which are thresholded one by one. depends on the position in the

Classical automatic thresholding algorithm

3.

4.

5.

1.

2.

Select an initial estimate for T Segment the image using T. This produces 2 groups: G 1 pixels with value >T and G 2 , with value

, ► ► Very easy + very fast Assumptions: normal dist. + low noise

Optimal Thresholding

► based on the shape of the current image histogram. Search for valleys, Gaussian distributions etc.

Foreground Both Background Optimal threshold ?

Real histogram

To love…

Histograms

…and to hate

Thresholding and illumination

► Solutions:    Calibration of the imaging system percentile filter with very large mask Morphological operators

MR non-uniformity

median filtering thresholding

More thresholding

► Can also be used on other kinds of histogram: grey-level, edge, feature etc. Multivariate data (  see next lectures) ► Problems:    Only considers the graylevel pixel value, so it can leave “holes” in segmented objects.

► Solution: post-processing with morphological operators Requires strong assumptions to be efficient Local thresholding is better  see region growing techniques

Edge-based Segmentation

Based on finding discontinuities (local variations of image intensity) 1.

2.

3.

Apply an edge detector ex gradient operator (Sobel) second derivative (Laplace) Threshold the edge image to get a binary image   Depending on the type of edge detector: Link edges together to close shapes (using edge direction for ex) Remove spurrious edges

Gradient based procedure

Sobel Sobel

Zero-crossing based procedure

LoG

Laplacian of Gaussian

Edge-based Segmentation: examples

Prewitt: needs edge linking Canny: needs “cleaning”

Region based segmentation

► Work by extending some region based on local similarities between pixels   region growing (bottom-up method) region splitting and merging (top-down method)   Bottom-up: from data to representation Top-down: from model to data

1.

Region growing

(bottom-up method) Find starting points 2.

Include neighbouring pixels with similar features (grey-level, texture, color).

3.

Continue untill all pixels have been included with one of the starting points. ►  Problems: Not trivial to find good strating points, difficult to automate  Need good criteria for similarity.

Watershed

(a kind of region growing) ► Think of the grey-level image as a landscape. Let water rise from the bottom of each valley (the water from each valley is given its own label). As soon as the water from two valleys meet, build a dam, or watershed. These watersheds will then define the borders between different regions.

Example of watershed directly on a gray level image

Example of Watershed on a binary image

Watershed: problems and solutions

► Oversegmentation  watershed from markers ► Computation  new algorithm for fast watershed ► Graylevel might not be the optimal choice as the local similarity measure  bigger neighborhood when growing  other local features (statistical, edge enhanced image, distance transformed image…)

1.

2.

3.

4.

Split & Merge

(top-down metod) Set up som criteria for what is a uniform area (ex mean, variance, bimodality of histogram, texture, etc…) Start with the full image and split it in to 4 sub-images.

Check each sub-image. If not uniform, divide into 4 new sub images.

After each iteration, compare touching regions with neighboring regions end merge if uniform. The method is also called "quadtree" division (and is also used for compression)

Split & Merge

The Hough transform

► A method for finding global relationships between pixels.

Example: We want to find straight lines in an image    1. Apply edge enhancing filter (ex Laplace) 2. Set a threshold for what filter response is considered a true ”edge pixel” 3. Extract the pixels that are on a straight line using the Hough transform original image edge enhanced image thresholded edge image

The Hough transform

Finding straight lines: ► ► ► ► ► ► 1. consider a pixel in position (xi,yi) 2. equation of a straight line yi=axi+b 3. set b=-axi+ yi and draw this (single) line in ”ab-space” 4. consider the next pixel with position (xj,yj) and draw the line b=-axj+ yj ”ab-space” (also called parameter space). The poins (a’,b’) where the two lines intersect represent the line y=a’x+b’ in ”xy-space” which will go through both (xi,yi) and (xj,yj). 5. draw the line in ab-space corresponding to each pixel in xy space.

6. divide ab-space into accumulator cells and find most common (a’, b’) which will give the line connecting the largest number of pixels

y

The Hough transform

b xy-space x ab- or parameter space a

►

The Hough transform

In reality we have a problem with y=ax+b because a reaches infinity for vertical lines.

Use

x

cos  

y

sin    instead.

► It is common to use ”filters” for finding the intersection: ”butterfly filters” ► Different variations of the Hough transform can also be used for finding other shapes of the form g(v,c)=0, v is a vector of coordinates, c is a vector of coefficients.

► Possible to find any kind of simple shape ex. circle: (

x



c

1 ) 2  (

y



c

2 ) 2 

c

2 3 (3D parameter space)

The Hough transform

Conclusions

► 1.

The segmentation procedure Pre-processing 2.

3.

Segmentation Post-processing  Like any IP procedure ► ► There exists NO universal segmentation method Evaluation of segmentation performance is important

Snakes

► Example: segmentation of the brain in MRI Snake after initialization Snake at equilibrium User interaction

Snakes (active contours)

► A snake is an active contour parametrically represented by its position v(s)=(x(s), y(s)) ► Each position is associated to an energy:

E snake

 0  1

E int

 0  1

E ext

► The final position corresponds to the minimum of the energy

Internal Energy

The internal energy of the snake is due to bending and it is associated with constraints: a priori ► ►

E int

 1 2    a 

v



s

2  b  2

v



s

2 a (s) controls the tension of the contour b (s) controls its rigidity 2   

External Energy

► The external energy depends on the image and accounts for a posteriori information ► Several energy forms have been proposed based on features of interest in the image ► An energy commonly used to attract snakes towards edges is:

E ext

 2

Applications

Applications

(by Terzopoulos)

Considerations

► ► ► ► ► The number of nodes is an important factor for the behavior of the snake. Ability to resample the contour may be necessary.

If we want a closed contour, we set the first and the last point equal.

Anchor points are necessary to keep the snake in position if the image forces are not enough.

It may be necessary to allow a snake contour to divide into two contours, or two contours to merge into one contour.

Different applications may need different potential functions and different settings of the control parameters (damping, tension and rigidity).

Applications

► ► ► Tracking of a moving object   An initial estimate for the contour (e.g. interactively defined) is used in the first frame. The contour at equilibrium is used as the starting contour for the next frame. The snake locks on to the object.

Reconstruction from serial sections    The user draws an approximate contour in the first slice. The contour at equilibrium is used as the starting contour in the next slice.

The 3D object is reconstructed from the contours using triangulation.

…..

More segmentation

Important in Image Processing in general: “If you can use expert knowledge (user interaction, modelling,…) at relatively low cost (development, computational,…)”

JUST DO IT!!