Game Theoretic Image Segmentation Elizabeth Cassell Sumanth Kolar Alex Yakushev Theme created by Sakari Koivunen and Henrik Omma Released under the LGPL license.

Introduction – Image Segmentation Distinguish objects from background Analysis of underlying structures Different conditions and content Applications Robot vision Pattern Recognition Biomedical image processing

Some examples of Image Segmentation

Strategies for Image Segmentation Threshold techniques Colour Information Edge Based techniques Shape Information Region Based Growing Step by Step Computational Geometry Previous shape knowledge. (Circles/Ellipse) Several more.. Clustering, Histogram

2 class problem, foreground and background Current Problem 2 class problem, foreground and background No restriction of connected components Seed Image given as input. Noisy Images.

Game Theoretic Approach Image Segmentation - two-person non-zero-sum non cooperative game Two players, one minimizing, other maximizing an energy function Region based segmentation module Goal is to find the region based on color information Boundary finding module Goal is to find a closed boundary shape

An exhaustive search would require 2NxM operations Image representation Grayscale images are represented as NxM matrices. yi,j is the intensity at of a pixel at i,j A pixel is assigned a class xi,j, in the two-class case: xi,j is either 1 or 0. An exhaustive search would require 2NxM operations N M

Image Examples Additive noise Reasons - Input device sensor and low signal level, such as shadow regions or underexposed images

Player 1: Region Based Module Start from a seed image, compute the energy function E for neighboring pixels. Add the pixels, for which the value of E is below a certain threshold. Repeat, until no more pixels can be added.

Region Based Module (details) Minimization of the energy term – data fidelity term and second term enforces smoothness Where yi,j is a pixel value from the original image, xi,j is the classification of that pixel, and is,js represents the neighborhood of the pixel

Region Segmentation - Works

Region Segmentation - problems Given a very noisy image, a lot of pixels will be missed.

Player 2: Boundary Finding General idea is to find a closed boundary around the object of interest. Boundary constraints usually include smoothness, and closeness to a prior Boundary Finding method is trying to maximize a function of the curve parameters E.g. If we were looking for ellipses we would be looking for the right values of x0, y0, a, and b (center point, major and minor axes) Source: http://www.lems.brown.edu/~msj/cs292/project/intermediate.html

Boundary Finding Class of objects with smooth boundaries that are deformable. Impose global structure information on the segmentation

Boundary Finding - Implementation Morphological operation – Closing Dilation followed by erosion Closing tends to narrow smooth sections of contours, fusing narrow breaks and long thin gulfs, eliminating small holes, and filling gaps in contours.

Flow diagram - Integration Image Boundary Finding Region Segmentation Image regions (Ir) Object Boundary (p) Ir p

Region Based Segmentation - Integration

Boundary Finding - Integrated Posed as maximum a posteriori framework Last term incorporates region based information. Maximum when the classification by the region based module is correct.

Test Images

Results

Results

Objective of each player : Minimise pay off function Nash Equilibrium Objective of each player : Minimise pay off function Find Nash Equilibrium

Nash Existence For objective functions of this form, Nash equilibrium always exists for proper choices of alpha and beta

Conclusion Game Theory can be applied to image segmentation Produces better results than each of the individual modules Combination method is more robust to noise Future work includes learning the seed and using different region and boundary finding algorithms