1. Game Theoretic Image Segmentation
Elizabeth Cassell
Sumanth Kolar
Alex Yakushev
Theme created by
Sakari Koivunen and Henrik Omma
Released under the LGPL license.

2. Introduction – Image Segmentation
Distinguish objects from background
Analysis of underlying structures
Different conditions and content
Applications
Robot vision
Pattern Recognition
Biomedical image processing

3. Some examples of Image Segmentation

4. Strategies for Image Segmentation
Threshold techniques
Colour Information
Edge Based techniques
Shape Information
Region Based
Growing Step by Step
Computational Geometry
Previous shape knowledge. (Circles/Ellipse)
Several more.. Clustering, Histogram

5. 2 class problem, foreground and background
Current Problem
2 class problem, foreground and background
No restriction of connected components
Seed Image given as input.
Noisy Images.

6. Game Theoretic Approach
Image Segmentation - two-person non-zero-sum non cooperative game
Two players, one minimizing, other maximizing an energy function
Region based segmentation module
Goal is to find the region based on color information
Boundary finding module
Goal is to find a closed boundary shape

7. An exhaustive search would require 2NxM operations
Image representation
Grayscale images are represented as NxM matrices. yi,j is the intensity at of a pixel at i,j
A pixel is assigned a class xi,j, in the two-class case: xi,j is either 1 or 0.
An exhaustive search would require 2NxM operations N M

8. Image Examples Additive noise
Reasons - Input device sensor and low signal level, such as shadow regions or underexposed images

9. Player 1: Region Based Module
Start from a seed image, compute the energy function E for neighboring pixels.
Add the pixels, for which the value of E is below a certain threshold.
Repeat, until no more pixels can be added.

10. Region Based Module (details)
Minimization of the energy term – data fidelity term and second term enforces smoothness
Where yi,j is a pixel value from the original image, xi,j is
the classification of that pixel, and is,js represents the neighborhood of the pixel

11. Region Segmentation - Works

12. Region Segmentation - problems
Given a very noisy image, a lot of pixels will be missed.

13. Player 2: Boundary Finding
General idea is to find a closed boundary around the object of interest.
Boundary constraints usually include smoothness, and closeness to a prior
Boundary Finding method is trying to maximize a function of the curve parameters
E.g. If we were looking for ellipses we would be looking for the right values of x0, y0, a, and b (center point, major and minor axes)
Source:

14. Boundary Finding
Class of objects with smooth boundaries that are deformable.
Impose global structure information on the segmentation

15. Boundary Finding - Implementation
Morphological operation – Closing
Dilation followed by erosion
Closing tends to narrow smooth sections of contours, fusing narrow breaks and long thin gulfs, eliminating small holes, and filling gaps in contours.

16. Flow diagram - Integration
Image
Boundary Finding
Region Segmentation
Image regions (Ir)
Object Boundary (p) Ir p

17. Region Based Segmentation - Integration

18. Boundary Finding - Integrated
Posed as maximum a posteriori framework
Last term incorporates region based information. Maximum when the classification by the region based module is correct.

19. Test Images

20. Results

21. Results

22. Objective of each player : Minimise pay off function
Nash Equilibrium
Objective of each player : Minimise pay off function
Find Nash Equilibrium

23. Nash Existence
For objective functions of this form, Nash equilibrium always exists for proper choices of alpha and beta

24. Conclusion Game Theory can be applied to image segmentation
Produces better results than each of the individual modules
Combination method is more robust to noise
Future work includes learning the seed and using different region and boundary finding algorithms