Manuchehr Aminian Advisor: Prof. Andrew Knyazev University of Colorado Denver Algorithms for Generalized Image Segmentation

What is image segmentation? Given a digital image, we want to separate/partition it into a small number of segments Guidelines: Pixels in a given segment should be similar in some respect (for example, color or intensity) Pixels in adjacent segments should be dissimilar Segments should be contiguous regions

Some good/bad examples Let’s look at some good and bad segmentations with this image. Source:

Good example Each segment has similar colors, different segments are dissimilar, and each segment is contiguous.

Bad example The first segment is okay, but the second one has many pixels which are dissimilar.

What we use: Spectral image segmentation Using so-called “spectral” image segmentation satisfies all our requirements. Trust me. A mathematician said it works.

What is the algorithm?

A physical interpretation In physics, a classic problem is to attach masses to each other by springs and examine how they oscillate Masses connected by a stiffer spring have a tendency to oscillate together

A physical interpretation (cont. 1) On our image, connect a pixel to all its surrounding neighbors with “springs”. We connect pixels which are alike (in color or intensity) with stiff springs For example, if we take this image...

A physical interpretation (cont. 2) Doing this for all the pixels in the image, we give it a shake. The result? All the reds vibrate together, as do the blues. So, put all the reds/oranges into one segment, and all the blues/greens into the other.

Generalize to three dimensions! We follow the same basic procedure in three dimensions. The only thing that changes is the grid.

Examples of 3D images The third dimension is either time or spatial Some examples: – 3D images created from a 3D scanner or an MRI machine – Any animations or videos

Interesting things can happen Compare the following two segmentations:

Why are they so different? The first image was taken from the 3D algorithm run over the entire animation, whereas the second one was taken from the 2D algorithm run on the single frame.

Conclusions Spectral segmentation: Satisfies all our guidelines Has a (relatively) easy-to-understand physical interpretation Using freely available math software packages HYPRE and BLOPEX, this can be scaled up to parallel computers

Thanks to: UC Denver UROP, providing financial support for the research The University of Colorado Denver