Nottingham Image Analysis School, 23 – 25 June NITS Image Segmentation Guoping Qiu School of Computer Science, University of Nottingham |

Nottingham Image Analysis School, 23 – 25 June Lecture Plan Image segmentation in image analysis (ImageJ demo) A look into the algorithms … Convert an image (grey scale or color) to binary Clean up the binary image Extract/identify individual objects (regions) Describe/measure the objects (regions) Discussion/prospective

Nottingham Image Analysis School, 23 – 25 June First, a demo … ImageJ Demo Load image (File->Open) Make binary (Process->Binary->Make Binary) Process->Binary->Erode Process->Binary->Dilate Analyze->Analyze Particle

Nottingham Image Analysis School, 23 – 25 June Identify and marking region of interest Describe the objects: Shapes, sizes, populations …

Nottingham Image Analysis School, 23 – 25 June Identify and marking region of interest Describe the objects: Shapes, sizes, populations … Separating objects from the background and from each other Generally referred to as image segmentation A hard (unsolved) problem

Nottingham Image Analysis School, 23 – 25 June Identify and marking region of interest Describe the objects: Shapes, sizes, populations … Identify individual objects or regions (connected component labelling)

Nottingham Image Analysis School, 23 – 25 June Identify and marking region of interest Describe the objects: Shapes, sizes, populations … Measure the sizes, shapes, and other attributes of individual objects (regions)

Nottingham Image Analysis School, 23 – 25 June Image Segmentation A very hard (unsolved problem) Same organ, different signals Different organ, same signal

Nottingham Image Analysis School, 23 – 25 June Image Segmentation A very hard (unsolved problem) Background Or Foreground ? Even human find it hard/impossible

Nottingham Image Analysis School, 23 – 25 June Image Segmentation For simple cases, use thresholding

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Thresholding strategies One sided Two sided Set membership

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Clean up the thresholded binary image Thresholded binary image will have isolated points, objects will have holes etc, it is necessary to clean up the image, remove unwanted pixels.

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Clean up the thresholded binary image Morphological Operators Two basic operators Dilation Erosion Two combinations Closing Opening Structure elements

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Dilation For each background pixel (which we will call the input pixel) we superimpose the structuring element on top of the input image so that the origin of the structuring element coincides with the input pixel position. If at least one pixel in the structuring element coincides with a foreground pixel in the image underneath, then the input pixel is set to the foreground value. If all the corresponding pixels in the image are background, however, the input pixel is left at the background value.

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Dilation 1 1 1

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Dilation

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Erosion For each foreground pixel (which we will call the input pixel) we superimpose the structuring element on top of the input image so that the origin of the structuring element coincides with the input pixel coordinates. If for every pixel in the structuring element, the corresponding pixel in the image underneath is a foreground pixel, then the input pixel is left as it is. If any of the corresponding pixels in the image are background, however, the input pixel is also set to background value.

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Erosion 1 1 1

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Erosion

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Opening: an erosion followed by a dilation using the same structuring element for both operations

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Opening

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Closing: opening performed in reverse

Nottingham Image Analysis School, 23 – 25 June Image Segmentation Morphological Operators Closing: opening performed in reverse

Nottingham Image Analysis School, 23 – 25 June Identify Individual Object Connected Component Labeling: Give pixels belonging to the same object the same label (grey level or color)

Nottingham Image Analysis School, 23 – 25 June Identify Individual Object Connected component labelling works by scanning an image, pixel-by-pixel (from top to bottom and left to right) in order to identify connected pixel regions, i.e. regions of adjacent pixels which share the same set of intensity values.

Nottingham Image Analysis School, 23 – 25 June Identify Individual Object Connected Pixels Two pixels p and q are said to be connected if there is a sequence of foreground (1) pixels Such that

Nottingham Image Analysis School, 23 – 25 June Identify Individual Object Pixel Neighbors 010 1p1 010 Four nearest neighbors 111 1p1 111 Eight nearest neighbors

Nottingham Image Analysis School, 23 – 25 June Identify Individual Object 4- and 8-Connected Pixels When only the four nearest neighbors are considered part of the neighborhood, then pixels p and q are said to be “4- connected” When the 8 nearest neighbors are considered part of the neighborhood, then pixels p and q are said to be “8- connected” 4-connected 8-connected

Nottingham Image Analysis School, 23 – 25 June Labelling Algorithm The connected components labeling operator scans the image by moving along a row until it comes to a point p (where p denotes the pixel to be labeled at any stage in the scanning process) for which V={1}. When this is true, it examines the four neighbors of p which have already been encountered in the scan (in the case of 8-connectivity, the neighbors (i) to the left of p, (ii) above it, and (iii and iv) the two upper diagonal terms). Based on this information, the labeling of p occurs as follows: If all four neighbors are 0, assign a new label to p, else if only one neighbor has V={1}, assign its label to p, else if more than one of the neighbors have V={1}, assign one of the labels to p and make a note of the equivalences. After completing the scan, the equivalent label pairs are sorted into equivalence classes and a unique label is assigned to each class. As a final step, a second scan is made through the image, during which each label is replaced by the label assigned to its equivalence classes. For display, the labels might be different graylevels or colors.

Nottingham Image Analysis School, 23 – 25 June Labelling Algorithm Example (8-connectivity) Equivalent labels {1, 2} Original Binary ImageAfter first scan

Nottingham Image Analysis School, 23 – 25 June Labelling Algorithm Example (8-connectivity) Equivalent labels {1, 2} After 1 st scan Final label 1: {1, 2} 2: {3} After 2 nd scan (two individual objects/regions have been identified)

Nottingham Image Analysis School, 23 – 25 June Describe Objects Some useful features can be extracted once we have connected components, including Area Centroid Extremal points, bounding box Circularity Spatial moments

Nottingham Image Analysis School, 23 – 25 June Describe Objects Area Centroid

Nottingham Image Analysis School, 23 – 25 June Summary Identify and marking region of interest will normally involve the following steps Describe the objects: Shapes, sizes, populations …

Nottingham Image Analysis School, 23 – 25 June Summary Identify and marking region of interest will normally involve the following steps Convert an image (grey scale or color) to binary (also referred to as image segmentation). Clean up the binary image Extract individual objects (regions) Describe the objects (regions)

Nottingham Image Analysis School, 23 – 25 June Discussion (Automatic) Image segmentation is one of the most critical phases in image analysis (but it is still very hard!) Simple thresholding After some morphological processing

Nottingham Image Analysis School, 23 – 25 June Discussion Semi-automatic and more sophisticated segmentation algorithms exist User specify pixel classes

Nottingham Image Analysis School, 23 – 25 June Discussion Looking into the future - computer aided diagnosis … Describe the objects: Shapes, sizes, populations … Machine learning Mathematic Models AI Algorithms Object/region descriptions Diagnosis Output