1. Objective of Computer Vision
• The field of computer vision can be divided into two areas
– Image enhancement
– Image analysis
Here we concentrate on fast methods typical for robot soccer and robot theatre applications

2. Binary image processing
Image with two gray levels 0 and 1
It contains the basic terms and concepts used in machine vision.
Its techniques are used in all aspects of a vision system.
Small memory requirements
Fast execution time

5. Gray level image vs. binary image

6. Image Enhancement

7. Point Operations

9. Region and segmentation
A subset of an image
Segmentation
Grouping of pixels into regions such that

10. Thresholding

11. Thresholding : Binary image Thresholding :
A method to convert a gray scale image into a binary image for object-background separation
: Thresholded gray image
Obtained using a threshold T for the original gray image .
: Binary image
Equivalent to

12. Three types of thresholding
where Z is a set of which elements are integer-valued intervals.

13. Original image and its histogram

14. Thresholding

15. Histogram Equalization

17. Vision System of Soccer Robot

18. Image notation for soccer
Image : a two-dimensional array of pixels
Pixel a[i, j]

19. Geometric properties
In many cases, some simple features of regions are useful to:
determine the locations of objects,
and to recognize objects.
Geometric properties:
Size
Position
Orientation

20. Total size is number of black dots
Size and position
Given an m x n binary image,
Size (area) A : zero-order moment
Position : the center of area
Total size is number of black dots
Average in x
Average in y

21. Example

22. How to calculate line orientation?
Step 1: calculate a,b,c coefficients
The center of object :
Let
By the least-squares fit of the line,
Step 2: calculate angle, line orientation

23. Size filter for Noise removal
It can effectively remove noise after component algorithm labeling.
If objects of interest have sizes greater than T, all components below T are removed by changing the corresponding pixels’ values to 0.
A noisy binary image and the resulting image after size filtering (T = 10)

24. How to get the position and angle of robot
Get frame-grabber, color CCD camera and computer.
Understand how you can read the image data from frame-grabber.
Find the position of a colored object in 2-D image.
Determine the robot uniform with two colored objects
Calculate the position and angle of the robot from the positions of two colored objects.

25. Position of colored object
1. Setting of ranges for YUV
[Ymin,Ymax], [Umin,Umax], [Vmin, Vmax]
2. Thresholding
3. Labeling (grouping)
4. Size filtering (noise elimination)
5. Finding the center of a colored object

26. Calculations for soccer field

27. Finding robots in the field

28. Robot position and heading
This cross sign is easy to recognize

29. Robot color and team color
This slide explains labeling robots and teams

30. Window tracking method
Processing only the data within a small window
Getting a fast vision processing

31. This is called line orientation
Orientation from of the axis of elongation
This is called line orientation

32. The distance, d, from any (x, y) within the object to the line :
Line equation :
: the minimum distance between the line and origin
: the angle from x-axis to the line
The distance, d, from any (x, y) within the object to the line :
which satisfies
Minimize
Our task is to find values of angle theta and rho for which this formula is minimum. This provides best fit to line equation

33. Given is object. Find its line orientation
The center of object :
Let
Calculate center
Calculate a, b, and c.

34. By the least-squares fit of the line,
Calculate center
Calculate a, b, and c.
Calculate theta

35. Binary algorithms Several definitions Neighbors
4-neighbors (4-connected)
8-neighbors (8-connected)

36. Path Foreground : Connectivity A sequence of neighbors
The set of all unity valued pixels in an image
Connectivity
A pixel is said to be connected to if there is a path from to consisting entirely of pixels of
Connected components
A set of pixels in which each pixel is connected to all other pixels.

37. Component labeling Component labeling algorithm
It finds all connected components in an image and assigns a unique label to all the points in a component.
One of the most common operations in machine vision
Recursive connected components algorithm
Sequential connected components algorithm
The points in a connected component form a candidate region for an object.

38. An image and its connected component image

39. Recursive algorithm for connected component labeling
Recursive connected components algorithm
Scan the image to find an unlabeled unity valued pixel and assign it a new label L.
2. Recursively assign a label L to all its unity valued neighbors.
3. Stop if there are no more unlabeled unity valued pixels.
4. Go to step 1.

40. Pseudo code for the recursive algorithm

41. Sequential algorithm for connected component labeling
Sequential connected components algorithm using 4-connectivity
Scan the image from left to right and top to bottom.
2. If the pixel is unity valued , then
If only one of its upper or left neighbors has a label, then copy the label.
(b) If both have the same label, then copy the same label.
(c) If both have different labels, then copy the upper pixel’s label and enter the labels in an equivalence table as equivalent labels.

42. Sequential algorithm for connected component labeling continued
(d) Otherwise assign a new label to this pixel and enter this label in the equivalence table.
3. If there are more pixels to consider, then go to step 2.
4. Find the lowest label for each equivalent set in the
equivalence table.
5. Scan the picture. Replace each label by the lowest label in its equivalent set.

43. Pseudo code for step 2 in the sequential algorithm

44. An example for the sequential algorithm

45. Digital Geometry Neighborhood Connectedness Distance Metrics I(i,j) j
(0,0)
Picture Element or Pixel i
We now turn our attention to the familiar concepts of geometry, such as distances between points, but from the point of view of a digital image.
A digital image, as already noted, can be thought of as an integer-valued array whose entries are drawn from the finite set {0...K-1}. Each entry in this array is called a 'picture element' or pixel. Each pixel has a unique 'address' in the array given by it's coordinates (i,j). The value at location (i,j) is the quantized value of the continuous signal sampled at that spatial location.
If K=2, the image is called a binary image, since all gray levels are either 0 or 1. For K>2, the image is called a grayscale image for the obvious reasons. When the value at a particular spatial location is a vector (implying multiple images), then the image is called a multispectral image. For example, a color image can be thought of as an image in which the value at a single pixel has three components (for each of the red, green, and blue components of the color signal) represented as a three element vector. 32
Pixel value I(I,j) =
Neighborhood
Connectedness
Distance Metrics
0,1 Binary Image
0 - K-1 Gray Scale Image
Vector: Multispectral Image

46. Connected Components Binary image with multiple 'objects'
Separate 'objects' must be labeled individually
Let's consider the concept of 'connectedness' in an image represented in discrete form on a rectangular grid. The simple binary image shown contains four 'objects': a central object that looks like a blob, a ring completely encircling it, and a second blob in the lower right hand corner, all represented by white pixels. The fourth object is the background, represented by black pixels. One would like to say that all the pixels in one of these objects are connected together and that the objects are not connected to each other or the background.
In this image, there are six connected components: the three objects and the three pieces of the background. All of the pixels in the ring, for example, share several features: they are all white, taken together they form the ring, and, for every pixel in the ring, a path can be found to every other pixel in the ring without encountering any 'non-ring' pixels. This latter feature illustrates the concept of connectedness in a digital image; its definition can be made much more precise.
6 Connected Components

47. Finding Connected Components
Two points in an image are 'connected' if a path can be found for which the value of the image function is the same all along the path. P1
connected to P2 P3
connected to P4
By definition, two pixels in an image are connected if a path from the first point to the second point (or vice-versa) can be found such that the value of the image function is the same for all points in the path. It remains to be defined exactly what is meant by a 'path'. P1
not connected to
P3 or P4 P2
not connected to
P3 or P4 P3
not connected to
P1 or P2 P4
not connected to
P1 or P2

48. Algorithm Pick any pixel in the image and assign it a label
Assign same label to any neighbor pixel with the same value of the image function
Continue labeling neighbors until no neighbors can be assigned this label
Choose another label and another pixel not already labeled and continue
If no more unlabeled image points, stop.
Given a binary image, its connected components can be found with a very simple algorithm. Pick any point in the image and give it some label (say A). Examine the neighbors of the point labeled A. If any of these neighboring points have the same gray value* (here, only 0 or 1 are possible gray values), assign these points the label A also. Now consider the neighbors of the set of points just labelled; assign the label A to any which have the same gray value. Continue until no more points can be labelled A. The set of pixels labelled A constitute one connected component, which was 'grown' from the central point that was originally labelled. Now pick any point in the image which has not been labelled and assign it a different label (say B) and grow out from it. Repeat the process of picking an unlabelled point, assigning it a unique label, and growing out from this point until there are no unlabelled points left in the image. Eventually, every image pixel will have been assigned a label; the number of unique labels corresponds to the number of connected components in the image.
Any pixel in a connected component is 'connected' to every other pixel in the same connected component but not to any pixel in a connected component with a different label. It is clear, therefore, that the notion of 'connectedness' in an image is intimately related to the definition of a pixel's neighbors. Let's explore the idea of the neighborhood of a pixel and the idea of the neighbors of a pixel in a little more detail.
But first, let's illustrate this algorithm with an example.
___________________
* Or color or texture or whatever other feature we are interested in.
Who's my neighbor?

49. Example
To illustrate the simple connected components algorithm, choose the top left pixel (arbitrary decision) and label it blue. This pixel has three neighbors, only two of which have the same gray level - the one to its immediate right and the one below it; label them both blue. In the next step, consider these two just-labeled points, starting with the right one. This point has 5 neighbors, one of which is our original point (already labeled) and one of which was labeled in the previous step. Of the three remaining pixels, only one has the same gray level, so label it blue. Now consider the second pixel labeled in the second step (just below the original start point). This pixel also has five neighbors, two of which are already labeled. Of the three remaining pixels, two have the same gray level and can therefore be labeled blue. This ends the third application of the growing process. Repeating this until no further pixels can be labeled results in the connected component shown in blue in the leftmost image of the middle row. Now pick any unlabeled point in the original image and start all over again.
At the end of the process, the image has been decomposed into five connected components. Connected component labeling is a basic technique in computer vision; we will repeatedly use this and similar algorithms throughout the remainder of the course.

50. Neighbor Consider the definition of the term 'neighbor'
Two common definitions:
In defining the notion of a path, and in the connected components algorithm, we used the term 'neighbor' in a very intuitive way. Let's examine this intuition a little more closely. There are two obvious definitions of neighbor and neighborhood commonly used in the vision literature. The first of these is called a 4-neighborhood; under this definition, the four pixels in gray surrounding the blue pixel are its neighbors and the blue pixel is said to have a neighborhood composed of the four gray pixels. These pixels are 'connected' to the blue pixel by the common edge between them. The second definition is called an 8-neighborhood and is shown on the right; the 8 neighbors of the blue pixel includes the four edge-connected neighbors and the four vertex connected neighbors.
Neither of these definitions is entirely satisfactory for the definition of a neighborhood. Consider the digital counterpart of a closed curve in the plane, which partitions the background into two components
Four Neighbor
Eight Neighbor
Consider what happens with a closed curve.
One would expect a closed curve to partition the plane into two connected regions.

51. Alternate Neighborhood Definitions
Suppose we have a closed curve in a plane. One of the properties of a closed curve is that it partitions the background into two components, one of which is inside the curve and one of which is outside the curve. The top image represents a digital version of such a curve. Let's apply the two definitions of 'neighborhood' to it and analyze what happens to the curve and the background in terms of their connectedness.
Let's consider the 4-neighborhood definition first. In this case, the diagonally connected pixels that make up the 'rounded' corners of the O are not connected. Hence, the O is broken into four distinct components. The curve we said was connected is not connected using the 4-neighborhood definition of a neighborhood. On the other hand, the background pixels are not connected across the diagonally connected points either, so the background is broken into two separate connected components. We are now in the curious position of having the background partitioned into two parts by a curve that is not connected (or closed).
Now consider the 8-neighborhood definition. In this case, the O is connected, since the vertex connected corners are part of the neighborhood. However, the inside and outside background components are also connected (for the same reason). We now have a closed curve which does not partition the background!

52. Possible Solutions
Use 4-neighborhood for object and 8-neighborhood for background
requires a-priori knowledge about which pixels are object and which are background
Use a six-connected neighborhood:
One possible solution to this dilemma is to use the 4-connected definition for objects and the 8-connected definition for the background. Of course, this implies that we must know which points correspond to the object and which parts correspond to the background. In general, the separation into object and background is not known a priori.

53. Digital Distances Alternate distance metrics for digital images
In many cases, it is necessary to measure the distance between two points on the image plane. There are three metrics commonly used in vision. First, the standard Euclidean distance metric is simply the square root of the sum of the squares of the differences in the coordinates of the two points. The computation of this metric requires real arithmetic and the calculation of a square root; it is computationally expensive, but the most accurate.
An approximation to the Euclidean distance is the city block distance, which is the sum of the absolute values of the differences in coordinates of the two points. The city block distance is always greater than or equal to the Euclidean distance.
The third metric is called the chessboard distance, which is simply the maximum of the absolute values of the vertical and horizontal distances.
These last two metrics have the advantage of being computationally simpler than the Euclidean distance. However, for precise mensuration problems, the Euclidean distance must be used. In other cases, the choice of the metric to use may not be as critical.
Euclidean Distance
City Block Distance
Chessboard Distance
= |i-n| + |j-m|
= max[ |i-n|, |j-m| ]
= (i-n) 2 + (j-m) 2

54. Connected Components / Image Labeling
Goal: To find clusters of pixels that are similar and connected to each other
How it works:
Assign a value to each pixel
Define what similar values mean
e.g., 10 +/- 2
Determine if like pixels are connected

55. Connected Components / Image Labeling

56. Component Labeling Definitions Neighbors Connected components
A set of pixels in which each pixel is connected to all other pixels.

57. An image and its connected component image

58. Connected Components / Image Labeling1 2 A A A A A A A B B A A A A A A C A A A D D C C A A D D C C A

59. Connected Components / Image Labeling1 2 A A A A A A A B B A A A A A A B A A A C C B B A A C C B B A

60. Binary Image Processing
After thresholding an image, we want to know something about the regions found ...
How many objects are in the image?
Where are the distinct “object” components?
“Cleaning up” a binary image?
Recognizing objects through their response to image masks
Describing the shape/structure of 2d objects
Pattern

61. Counting Objects
external corners
internal corners 1 1 1 1 1 1 1 1

62. Connected Component Labeling1 1 2

63. Connected Component Labeling
Algorithm 1
1. Image is A. Let A = -A;
2. Start in upper left and work L to R, Top to Bottom, looking for an unprocessed (-1) pixel.
3. When one is found, change its label to the next unused integer. Relabel all of that pixel’s unprocessed neighbors and their neighbors recursively.
4. When there are no more unprocessed neighbors, resume searching at step 2 -- but do so where you left off the last time. 1 2

64. Summary A simple procedure to get robot position and heading:
Thresholding,
labeling,
size filtering,
center of area calculation,
window tracking method
Trade-off between:
real-time vision processing
and robustness
Specific frame grabber required

65. Problems for students
Binary image processing and basic operations on binary images.
List and explain briefly operations used in image enhancement
What are point operations. Give examples.
Explain contrast stretching operation. Write simple program.
Definition of image segmentation.
Segmentation based on thresholding
Types of thresholding. Give examples of each with practical applications.
The general idea of histogram equalization. Show examples.
Geometric positions of objects in robot soccer.
Size and position of objects in robot soccer.

66. Problems for students
11. How to calculate quickly the center of the mass of an object?
12. How to calculate line orientation?
13. Describe and program size filter for noise removal. Emphasize speed not quality.
14. How to calculate position and angle of a robot using the ceiling camera?
15. Use of colors in vision of robot soccer.
16. Window tracking methods in robot soccer.
17. Use of Line orientation in robot soccer.

67. Problems for students
Binary image processing algorithms in robot soccer.
Component labeling
Recursive connected components algorithm
Sequential connected components algorithm
Algorithms to find connected components.
Alternate definitions of neighbors in connected components algorithms. Applications in robot soccer.
The concept of digital distance and its use.
Connected component labeling in binary images.
Counting objects. What are applications in robot soccer?