Vision: Scene Labelling Structure in the world allows us to interpret complex situations by propagating constraints. This is conveniently demonstrated in computer vision, using a technique known as Guzman labelling. CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Blocks-World Vision Problem Given: A 2-D line drawing representing a collection of polyhedral blocks on a table. Determine: Which faces (bounded regions in the drawing) go together as parts of the same objects. Which edges (line segments) are internal to objects and which are “occluding” edges. CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Example Scene Analysis Problem CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Guzman’s Vertex Labels Ell Arrow Fork Tee Arrow: There is an angle of more than 180 deg. Fork: Each of the 3 angles is of less than 180 deg. Tee: There is one angle of exactly 180 deg., and 2 smaller ones. CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Guzman’s Labelling Technique 1. Classify each vertex into one of the categories: ell, arrow, fork, tee, or other. 2. At each vertex, mark each incoming edge according to the role it plays at the vertex (e.g., shank of an arrow, stem of a tee). 3. Create single links between neighboring regions each time: a. they are divided by a line segment at a fork, b. they are divided by the shank of an arrow, c. they are in corresponding positions of a configuration of two opposing tees with colinear stems. 4. Create a “node” for each region. Whenever two regions are doubly linked, connect their nodes with a “same object” link. CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Example with Vertex Labels F L T A T F T L L L A A CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Example with Region Links CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Example with Object Links CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

Example with Internal and Occluding Edges Identified CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling

CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling Discussion The work of Guzman was refined by D. Huffman, M. Clowes, and D. Waltz to take into account additional kinds of edges, including crack edges, oriented occluding edges, and concave and convex internal edges. A key point of this research was to show that constraints imposed by the real world could make seemingly intractable combinatorial interpretation problems actually solvable. CSE 415 -- (c) S. Tanimoto, 2004 Scene Labelling