1Ellen L. Walker Recognizing Objects in Computer Images Ellen L. Walker Mathematical Sciences Dept Hiram College Hiram, OH

2Ellen L. Walker Why Recognize Objects? n “Put the clothing on the bed” n “Turn left at the apartment building”

3Ellen L. Walker Steps in Object Recognition

4Ellen L. Walker Formation of a 2D Image

5Ellen L. Walker Images are Composed of Pixels Image has 138 x 255 pixels (35190 total) Pixel values range from 0 (black) to 255 (white)

6Ellen L. Walker Edge Detection Use a technique called convolution to detect edges The 3x3 convolution of a pixel is the sum of the products of the pixel and its 8 neighbors with a 3x3 mask The convolution of the shaded cell with the mask is Abs(4x-1 + 3x0 + 2x1+ 4x-2 + 3x0 + 2x2 + 4x-1 + 3x0 + 2x1) = 8

7Ellen L. Walker Edge Detection with Convolution horiz. conv. vert. conv. both convolutions

8Ellen L. Walker Edges from Test Image After convolution and thresholding This is still an image … next find endpoints of straight segments

9Ellen L. Walker Fitting Line Segments to Edge Pixels n For each path of connected edge pixels: l draw a segment between the endpoints of the path l find the point farthest from the line l if it’s too far, then make that point an endpoint and recurse

10Ellen L. Walker Segments from Test Image n Circles denote endpoints n Note “broken” segments

11Ellen L. Walker Grouping Collinear Line Segments n Line segments are collinear if: angle between them is small enough (  ) l distance between them is small enough (PD) l “gap” between them is small enough (G)

12Ellen L. Walker Grouped Segments from Test Image n Long vertical and horizontal segments were found

13Ellen L. Walker Higher Levels of Grouping n Junctions n Curves & Polygons (no T-Junctions allowed)

14Ellen L. Walker Groups from the Test Image? n After grouping curves and polygons, and combining polygons that share an edge and no T-junctions n Finally, we’re ready to recognize!

15Ellen L. Walker Types of Models n Exact Models — Describe single objects l e.g. CAD models for factory parts l the object is needed to construct the model! n Generic Models —Describe classes of objects l Exact models with parameters (e.g length, width, radius) l Structural constraints n describe how the object is constructed l Functional constraints n describe how the object is used

16Ellen L. Walker Matching Method Depends on Model Level n Image l when objects are hard to describe (e.g. faces) l match by correlation (like convolution) n Features, Groups l good for geometric objects l match by finding correspondences between image and model features (or groups) n Complete Objects l good for classes of objects l match by applying constraints from model to objects derived from the image

17Ellen L. Walker Sample Object Model (Box) n Constraints l Adjacent faces are perpendicular (12 constraints) l Adjacent edges are perpendicular (8 constraints) l Opposite faces are parallel (3 constraints) l Opposite edges are parallel (6 constraints)

18Ellen L. Walker Finding Correspondences n Search the interpretation tree l each node is a correspondence l each path to a leaf is an interpretation l “prune” the tree based on constraints

19Ellen L. Walker Structural Constraints n Buildings — structural constraints l Buildings have rectangular walls and flat horizontal roofs l Each outer wall of a building intersects exactly two other walls, forming a closed ring l Each edge of the roof is the perpendicular intersection of the roof and a wall

20Ellen L. Walker Functional Constraints n Beds — functional constraints l A bed has a surface to sleep on that is relatively flat. l A bed has physical means of supporting the sleeping surface at a comfortable height. l A bed has clearance above and to at least one side of the sleeping surface.

21Ellen L. Walker Considering Occlusion in the Interpretation n Include names & parameters of objects n Include hypotheses for hidden parts when feasible n Specify only what is guaranteed!

22Ellen L. Walker Toward Real World Solutions n Increasingly complex models n Large number of alternatives to choose from n Uncertainty at all levels n Recognizing partially-visible objects n Recognition must be fast enough n Generating models automatically (learning!)