1. Lecture 8CSE 140 - Intro to Cognitive Science1 Interpreting Line Drawings II

2. Lecture 8CSE 140 - Intro to Cognitive Science2 Reprise….

3. Lecture 8CSE 140 - Intro to Cognitive Science3 We Interpret Line Drawings As 3D  We have strong intuitions about line drawings of simple geometric figures: We can detect possible 3D objects (although our information is coming from a 2D line drawing). We can detect the convexity or concavity of lines in the drawing. If a line is convex, we have strong intuitions about whether it is an occluding edge.

4. Lecture 8CSE 140 - Intro to Cognitive Science4 Huffman/Clowes Junction Labels  Huffman and Clowes each noted that adding information to Guzman’s approach makes the problem easier.  This is done by enriching the vocabulary of edge types from just connected/unconnected to include convexity information.

5. Lecture 8CSE 140 - Intro to Cognitive Science5 Convexity Labeling Conventions 1.A line labeled plus (+) indicates that the corresponding edge is convex ; 2.A line labeled minus (-) indicates that the corresponding edge is concave; 3.An arrow indicates an occluding edge. To its right is the body for which the arrow line provides an edge. On its left is space.

6. Lecture 8CSE 140 - Intro to Cognitive Science6 Edge Consistency Any consistent assignment of labels to the junctions in a picture must assign the same line label to any given line.

7. Lecture 8CSE 140 - Intro to Cognitive Science7 An Example of Edge Consistency  Consider an arrow junction with an L junction to the right:  A1 and either L1 or L6 are compatible since they both associate the same kind of arrow with the line.  A1 and L2 are incompatible, since the arrows are pointed in the opposing directions,  Similarly, A1 and L3 are incompatible.

8. Lecture 8CSE 140 - Intro to Cognitive Science8 The Generate and Test Algorithm 1.Generate all conceivable labelings. 2.Test each labeling to see whether it violates the edge consistency constraint; throw out those that do. BUT:  Each junction has on average 4.5 labeling interpretations.  If a figure has N junctions, we would have to generate and test 4.5 N different labelings.  Thus the Generate and Test Algorithm is O(4.5 N ).

9. Lecture 8CSE 140 - Intro to Cognitive Science9 Is O(4.5 N ) Bad??  A picture with 27 junctions, like the devil’s trident) will not be rejected until all 4.5 27 hypotheses have been rejected, leaving no interpretation.  4.5 27 = 433249302231073824.244664378464222  A computer capable of checking for edge consistency at a rate of 1 hypothesis per microsecond would take about 1 million years to establish that the devil’s trident has no consistent interpretation!

10. Lecture 8CSE 140 - Intro to Cognitive Science10 Onward….

11. Lecture 8CSE 140 - Intro to Cognitive Science11 Locally Consistent Search  The Generate and Test algorithm considers too many hypotheses that are simply outright impossible according to the edge consistency constraint.  A better algorithm would exploit locally consistent labelings to construct a globally consistent interpretation: 1.No junction label would be added unless it was consistent with its immediate neighbors; 2.If the interpretation could not be continued because no consistent junction label can be added, that interpretation would be abandoned immediately.

12. Lecture 8CSE 140 - Intro to Cognitive Science12 Search Trees to the Rescue! We could implement this by constructing a search tree: 1.Selecting some junction in the drawing as the root. 2.Label children at the 1 st level with all possible interpretations of that junction. 3.Label their children with possible consistent interpretations of some junction adjacent to that junction. 4.Each level of the tree adds one more labeled node to the growing interpretation. 5.Leaves represent either futile interpretations that cannot be continued or full interpretations of the line drawing.

13. Lecture 8CSE 140 - Intro to Cognitive Science13 The Top Three Levels of a Search Tree A1A3A2A1 L1 A1 L5 A2 L3 A3 L6

14. Lecture 8CSE 140 - Intro to Cognitive Science14 The Fourth Level of the Search Tree A1A3 A2 A1 L1 A1 L5 A2 L3 A3 L6 A1 L1 A1 L5 A3 A2 L3 XX A3 L6 A1 X

15. Lecture 8CSE 140 - Intro to Cognitive Science15 Breadth-First Search  In breadth-first search, all nodes at the 1 st level of the tree are generated, then all nodes at the 2 nd, then the 3 rd level, and so on until the leaves are reached.  If the average branching factor (the average number of children per node) is B and the depth of the tree is N, then the time complexity of the algorithm is T=O(B N ).  We have to keep track of the whole tree as we search, so the space complexity is also S=O(B N ).

16. Lecture 8CSE 140 - Intro to Cognitive Science16 Depth-First Search  Depth-first search follows one branch of the tree down to its leaf and then backtracks to follow the other branches down to their leaves.  Again, if the average branching factor is B and the depth of the tree is N, then the time complexity of the algorithm is still T=O(B N ).  But it requires less space since we only have the hold the junctions on one path from the root with information about which ones were choice points. So its space complexity is only S=O(N).

17. Lecture 8CSE 140 - Intro to Cognitive Science17 Constraint Propagation Waltz’s insight:  Pairs of adjacent junctions (junctions connected by a line) constrain each other’s interpretations!  These constraints can propagate along the connected edges of the graph. Waltz Filtering :  Suppose junctions i and j are connected by an edge. Remove any labeling from i that is inconsistent with the labeling assigned in j.  By iterating over the graph, the constraints will propagate.

18. Lecture 8CSE 140 - Intro to Cognitive Science18 When to Iterate, When to Stop? The crucial principle: Any algorithm must ensure that: If a label is removed from a junction i, then the labels on all of i’s neighbors will be reexamined. Why? Removing a label from a junction may result in one of the neighbors becoming edge inconsistent, so we need to check…

19. Lecture 8CSE 140 - Intro to Cognitive Science19 An Example of Constraint Propagation A1,A2,A3 L1,L2,L3, L4,L5,L6 A1,A2,A3 L1,L3, L5,L6 A1,A3 A1,A2, A3 L1,L3, L5,L6 A1,A2,A3 L1, L5,L6 A1,A3 L1, L5,L6 A1,A3

20. Lecture 8CSE 140 - Intro to Cognitive Science20 The Waltz/Mackworth Constraint Propagation Algorithm 1.Associate with each junction in the picture a set of all Huffman/Clowes junction labels appropriate for that junction type; 2.Repeat until there is no change in the set of labels associate with any junction: 3.For each junction i in the picture: 4.For each neighboring junction j in the picture: 5.Remove any junction label from i for which there is no edge-consistent junction label on j.

21. Lecture 8CSE 140 - Intro to Cognitive Science21 Inefficiencies  At each iteration, we only need to examine junctions i where at least one neighbor j has lost a label in the previous iteration, not all of them.  If j loses a label only because of edge inconsistencies with i, we don’t need to check i on the next iteration.  The only way in which i can be made edge- inconsistent by the removal of a label on j is with respect to j itself. Thus, we only need to check that the labels on the pair (i,j) are still consistent. These insights lead to a much better algorithm...

22. Lecture 8CSE 140 - Intro to Cognitive Science22 Arcs Given this notion of (i,j) junction pairs,  The crucial notion is an arc, where an arc is a directed pair of neighboring junctions.  For a neighboring pair of junctions i and j, there are two arcs that connect them: (i,j) and (j,i). ij (i,j) (j,i)

23. Lecture 8CSE 140 - Intro to Cognitive Science23 Arc Consistency We will define the algorithm for line labeling with respect to arc consistency as opposed to edge consistency: Arc Consistency Any consistent assignment of labels to the junctions in a picture must assign the same line label to an arc (i,j) as to the inverse arc (j,i).

24. Lecture 8CSE 140 - Intro to Cognitive Science24 Queues A queue is a list where:  Items are only added to one particular end of the queue, called the end of the queue.  Items are only removed from the other end of the queue, called the beginning of the queue. Queues are often called first-in first-out lists or FIFOs.

25. Lecture 8CSE 140 - Intro to Cognitive Science25 The AC-3 Algorithm 1.Associate with each junction in the picture a set of all Huffman/Clowes junction labels appropriate for that junction type; 2.Make a queue of all the arcs (i,j) in the picture; 3. Repeat until the queue is empty: 4.Remove an arc (i,j) from the queue; 5.For each label l on junction i, if there is no label on j that is arc consistent with l, remove l from i. 6.If any label is removed from i, then for all arcs (k,i) in the picture except (j,i), put (k,i) on the queue if it isn’t already there.

26. Lecture 8CSE 140 - Intro to Cognitive Science26 STATE 0 2.Queue: (1,2)(2,1)(2,3)(3,2)(3,4)(4,3) (4,1)(1,4)(1,3)(3,1) 4.Removing (1,2). 5.All labels on 1 consistent with 2, so no labels removed from 1. 6.No arcs added. Example of AC-3 (from Steedman)

27. Lecture 8CSE 140 - Intro to Cognitive Science27 AC-3: State 1 3.Queue: (2,1)(2,3)(3,2)(3,4)(4,3) (4,1)(1,4)(1,3)(3,1) 4.Removing (2,1). 5.L2 & L4 on 2 inconsistent with 1, so they are removed. 6.All arcs (k,2) except for (1,2) considered to add, but already in queue, so no arcs added.

28. Lecture 8CSE 140 - Intro to Cognitive Science28 AC-3: State 2 3.Queue: (2,3)(3,2)(3,4)(4,3)(4,1)(1,4) (1,3)(3,1) 4.Removing (2,3). 5.L3 on 2 inconsistent with 3, so it is removed. 6.Of arcs (k,2), (1,2) is not on queue, so it is added.

29. Lecture 8CSE 140 - Intro to Cognitive Science29 AC-3: State 3 3.Queue: (3,2)(3,4)(4,3)(4,1)(1,4)(1,3) (3,1)(1,2) 4.Removing (3,2). 5.A2 on 3 inconsistent with 2, so it is removed. 6.Of arcs (k,3) excluding (2,3) are on queue, so nothing added to queue.

30. Lecture 8CSE 140 - Intro to Cognitive Science30 AC-3: State 4 3.Queue: (3,4)(4,3)(4,1)(1,4)(1,3)(3,1) (1,2) 4.Removing (3,4). 5.All labels on 3 consistent with 4, so no labels removed from 3. 6.No arcs added.

31. Lecture 8CSE 140 - Intro to Cognitive Science31 AC-4: Final State 3.Queue: empty

32. Lecture 8CSE 140 - Intro to Cognitive Science32 AC-3: Complexity Analysis  Worst case: All junctions connected to every other junction, so each of N junctions must be compared against N-1 other junctions, so total # of arcs is: N * (N-1) So the resulting time complexity is O(N 2 ). For planar pictures, where lines can never cross, the number of arcs can only be linear in N, so for our pictures, the time complexity is only O(N)!!