1. Segmentation Lecture 04 Computer Vision

2. Material Citations Dr George Stockman
Professor Emeritus, Michigan State University
Dr Mubarak Shah
Professor, University of Central Florida
The Robotics Institute
Carnegie Mellon University
Dr David A. Forsyth
Professor, University of Illinois at Urbana- Champaign
Dr Kristen Grauman
Associate Professor, University of Texas at Austin

3. Suggested Readings Chapter 10
Linda G. Shapiro and George Stockman, “Computer Vision”, Upper Saddle River, NJ, Prentice Hall,
Chapter 15
David A. Forsyth and Jean Ponce, “Computer Vision A Modern Approach”, 2nd edition, Prentice Hall, Inc., 2003.
Chapter 5
Richard Szeliski , “Computer Vision: Algorithms and Applications”, Springer; 2011. 
Chapter 3
Mubarak Shah, “Fundamentals of Computer Vision”.

4. Recap (Edge Detection)
Marr-Hildreth and Canny edge detectors
Gaussian smoothing
Compute derivatives
In x and y directions
Find gradient magnitude
Threshold gradient magnitude
Difference between Marr-Hildreth and Canny
Marr-Hildreth use 2nd order derivative
Marr-Hildreth thresholds slope of zero-crossings

5. Marr-Hildreth Edge Detector
Image
2g(x,y)
Find
zero-crossings
compute
slope
Threshold

6. Canny Edge Detector gx(x,y) gy(x,y) Image Gradient magnitude direction
Non-maximum
suppression
Hysteresis
thresholding

7. Segmentation Thresholding Edge-based methods Region-based techniques
Clustering
K-Means etc.
Graph partitioning
Graph cuts etc.
Connectivity-preserving
Active Contour Model etc.
And more

8. Segmentation – Some applications
Object detection
Pedestrian detection
Face detection
Brake light detection
Locate objects in satellite images (roads, forests, crops, etc.)
Medical imaging
Locate tumors and other pathologies
Measure tissue volumes
Diagnosis, study of anatomical structure
Surgery planning
Virtual surgery simulation
Intra-surgery navigation

9. Segmentation – Some applications
Recognition Tasks
Face recognition
Fingerprint recognition
Iris recognition
Traffic control systems
Video surveillance

10. Segmentation - Example application
Object Recognition using region properties
Training
For all training samples of each model object
Segment the image
Compute region properties (features)
Recognition
Given an image of unknown object,
Compute its feature vector
Compare with the training set

11. Segmentation - Example application
MPEG4 Compression - Object Based Compression
Advantages of OBC
High compression ratio
Allows insertion deletion of objects
How does it work?
Find objects (Object Segmentation)
Code objects and their locations
Build mosaics of globally static objects
Render scene at receiver

12. Region Segmentation

13. Layer Representation

14. Segmentation Find set of regions R1, R2, ….,Rn such that
All pixels in region i satisfy some similarity constraint

15. Similarity Constraints
All pixels in any sub-image must have the same gray levels
All pixels in any sub-image must not differ more than some threshold
The standard deviation of gray levels in any sub-image must be small

16. Simple Segmentation

17. Image Histogram
Histogram graphs the number of pixels with a particular gray level as a function of the image of gray levels.
gray
level
number of pixels

18. Segmentation using Histogram - Simple case

19. Segmentation using Histogram - Simple case

20. Realistic Histograms Bimodal Histograms – Having two peaks
Not realistic
Real (noise)

21. Realistic Histograms Smooth out noise
Convolve histogram by averaging or 1D Gaussian filter
peak
valley
Identify peaks and valleys by detecting local maxima and local minima

22. Segmentation using Histogram
Compute the histogram of a given image
Smooth the histogram by averaging peaks and valleys in the histogram
Detect good peaks by applying thresholds at the valleys
Segment the image into several binary images using thresholds at the valleys
Apply connected component algorithm to each binary image and find connected regions.

23. Good Peaks Peakiness = Sharpness of a peak =
Smaller this ratio -sharper the peak.
When ration = 1, it’s a rectangle
Sharpness of a peak =
Ht. of valley to ht. of peak =
W = Width of the peak in grey level range from valley to valley
P = Height of the peak
Va, Vb = Height of valleys
N = Number of pixels in the image covered by the peak
If peakiness is greater then some threshold then it will be used for segmentation

24. Segmentation using Histograms
Select the valleys as thresholds
Apply threshold to histogram
Label the pixels within the range of a threshold with same label, i.e., a, b, c … or 1, 2, 3 …

25. Segmentation using Histograms
Have only used the grey level information for segmentation
Histograms do not give any spatial information
For connected group of pixels we need to apply connected component algorithm

26. Connectedness x
4 connected
8 connected
6 connected

27. Connected Components
Disjoint segments with same labels need to be split
may be added to segment c

28. Recursive Connected Component Algorithm
Scan the binary image left to right, top to bottom
If there is an unlabeled pixel with a value of ‘1’ assign a new label to it
Recursively check the neighbors of the pixel in step 2 and assign the
same label if there are unlabeled with a value of ‘1’
Stop when all the pixels of value ‘1’ have been labeled

29. Example - Detecting Finger Tips (marked white)

30. Example - Segmenting a bottle image
93 peaks

31. Example - Segmenting a bottle image
Smoothed histogram
(averaging using mask
Of size 5)
54 peaks (once smoothing)
After peakiness 7
Smoothed histogram
21 peaks (twice)
After peakiness 7
Smoothed histogram
11 peaks (three times)
After peakiness 4

32. Example - Segmenting a bottle image
Region from Peak 1
(0,40)
Region from Peak 2
(40, 116)
Region from Peak 3
(116,243)
Region from Peak 4
(243,255)

33. How to determine the three main intensities?
Example - Segmenting
Kristen Grauman, University of Texas at Austin
How to determine the three main intensities?
Kristen Grauman, University of Texas at Austin

34. Segmentation and Grouping
Purpose
Obtain a compact representation from an image/motion sequence/set of tokens
Tokens: Whatever we need to group (pixels, points, surface elements, etc.)
Major approaches
Forming image segments by grouping (or clustering)
Collect together tokens that “belong together”
Fitting a model
- Fitting a line to collection of points and associating the lines
- Associate a model with tokens (e.g. correspondence between pixels of the
two shots of same scene)
- Issues
Which model?
Which token goes to which element?

35. Segmentation and Grouping
Partitioning: Top down segmentation
– Decomposition of large sets into smaller ones according to their eligibility to the
selected model (or criterion)
• Decompose an image into regions that have coherent color or texture
• Decompose a video sequence into segments of video showing about the
same stuff (static area) from same view point
– Tokens belong together because they lie on the same object
Grouping: Bottom up segmentation
– Tokens belong together because they are locally coherent

36. Basic ideas of grouping in humans
Human visual illusions
Figure-ground discrimination
Grouping can be seen in terms of allocating some elements to a figure, some to ground
It is difficult to differentiate between figure and ground

37. Basic ideas of grouping in humans
Human visual illusions
Gestalt properties
Gestalt: A whole or a group
Muller-Lyer effect: Elements in a collection of elements can have properties that result from relationships
Gestalt Qualitat: The set of relationship that makes it a group
Gestalt factors: A series of factors affect whether elements should be grouped together
A human visual system uses them for grouping
Muller-Lyer effect

38. Gestalt factors Not grouped Proximity Similarity Common fate
Similar motion tendency
Some criteria that tend to cause tokens to be grouped.

39. Gestalt factors Similarity
Some criteria that tend to cause tokens to be grouped.

40. Gestalt factors Symmetry
Some criteria that tend to cause tokens to be grouped.

41. Gestalt factors Proximity
Some criteria that tend to cause tokens to be grouped.

42. Gestalt factors Common Fate
Some criteria that tend to cause tokens to be grouped.
Image credit: Arthus-Bertrand (via F. Durand)

43. Gestalt factors Parallelism Symmetry Continuity Closure
Parallel curves or tokens tend to be
grouped together
Tokens or curves that lead symmetric groups tend to be grouped together
Tokens that lead to continuous
curves tend to be grouped together
More such
Tokens or curves that tend to lead to
closed curves tend to be grouped together

44. Visual illusions - Grouping in humans
Occlusions and their effects
Can you understand what this picture is about?
Hint:
There are five numerals
Occlusion cues seem to be very important in grouping

45. Visual illusions - Grouping in humans
What numbers do you see?
but easy in this
Continuity factor can be used

46. Visual illusions - Grouping in humans
Illusory contours; an object appears to be occluding

47. Visual illusions - Grouping in humans
Elevator buttons with high probability to arrive at the
wrong floor was high
Elevator buttons with high probability to arrive at the right floor - Only because buttons are properly segmented