1. 3D Shape Inference Computer Vision No.2-1

2. Pinhole Camera Model the camera center Principal axis the image plane

3. Perspective Projection the optical axis the image plane the camera center Focal length

4. Orthographic Projection the optical axis the image plane the camera center

5. Weak Perspective Projection the optical axis the image plane the camera center the reference plane

6. Para Perspective Projection the optical axis the image plane the camera center the reference plane

7. Orthographic Projection the optical axis the image plane the camera center

8. Obtain a 3D Information form Line Drawing u Given –Line drawing(2D) u Find –3D object that projects to given lines u Find –How do you think it’s a cube, not a painted pancake?

9. Line Labeling u Significance –Provides 3D interpretation(within limits) –Illustrates successful(but incomplete)approach –Introduces constraints satisfaction u Pioneers –Roberts(1976) –Guzman(1969) –Huffman&Clows (1971) –Waltz (1972)

10. Outline u Types of lines u types of vertices u Junction Dictionary u Labeling by constraint propagation u Discussion

11. Line Types convex concave occluding

12. Labeling a Line Drawing Easy to label lines for this solid →Now invert this in order to understand shape

13. V

14. Enumerating Possible Line Labeling without Constraints 9 lines 4 labels each → 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 ｘ 4 = 250,000 possibilities We want just one reality must reduce surplus possibilities → Need constraints (by 3D relationship)

15. Vertex Types Divide junctions into categories Need some constraints to reduce junction types

16. Restrictions u No shadows, no cracks u Non-singular views u At most three faces meet at vertex

17. Fewer Vertex Types

18. Vertex Labeling  Three planes divide space into octants  Enumerate all possibilities (Some full, some empty)  Trihedral vertex at intersection of 3 planes

19. Enumerating Possible Vertex Labeling(1) 0 or 8 octants full-- no vertex 2,4,6 octants full singular view 7octants full 1FORK 5octants full 2L,1ARROW

20. u 3octants full –upper behindL –right aboveL –left aboveL –straight aboveARROW –straight belowFORK Enumeration(2)

21. Enumeration(3) １ octant--Seven viewing octants supply

22. Huffman&Clows Junction Dictionary u Any other arrangements cannot arise u Have reduced configuration from 144 to 12

23. Constraints on Labeling u Without constraints-- 250,000 possibilities u Consider constraints →3 ｘ 3 ｘ 3 ｘ 6 ｘ 6 ｘ 6 ｘ 5 ＝ 29,000 possibilities  We can reduce more by coherency/consistency along line.

24. Labeling by Constraint Propagation u “Waltz filtering” u By coherence rule, line label constrains neighbors u Propagate constraint through common vertex u Usually begin on boundary u May need to backtrack

25. Example of Labeling

26. Ambiguity Line drawing can have multiple labelings

27. Necker Reversal(1) u Wire-frame cube –Human perception flips from one to the other –(After Necker 1832,Swiss naturalist)

28. Necker Reversal(2)

30. Impossible Objects u No consistent labeling u But some do have a consistent labeling –What’s wrong here?

31. Limitations of Line Labeling u Only qualitative;only gets topology u Something wrong

32. Summary(1) Preliminary 3D analysis of shape 1. Identify 3D constraint 2. Determine how constraint affects images 3. Develop algorithm to exploit constraint --> General method for 3D vision Tool:constraint propagation/satisfaction

33. Summary(2) Problems 1. Significant ambiguity possible 2. Assumes perfect segmentation 3. Can be fooled without quantitative analysis

34. Gradient Space Computer Vision No. 2-2

35. Gradient Space and Line Labeling u Last time: line labeling by constraint propagation u Use gradient space to represent surface orientation －－ ＋ ＋＋

36. Review of Line Labeling Problem Given a line drawing, label all the lines with one of 4 symbols ＋ convex edge － concave edge ←→ occluding edges Approach Narrow down the number of possible labels with a vertex catalog ＋＋ ＋ －－ －＋＋

37. Surface Normal Normal of a plane Rewrite Normal vector (A,B,C)

38. Surface Gradient Gradient of surface is Gradient of plane

39. Surface Gradient q p p1p1 p3p3 p2p2 y

40. Relationship of Normal to Gradient (p,q) 1 0 p q x y x p1p1 p4p4 p5p5 Normal Vector p1p1 p3p3 p2p2 y q p

41. Polyhedron in Gradient Space G H F E D C B I A ＋ ＋＋ ＋ ＋ ＋ ＋ ＋ ＋ ＋ ＋ ＋ ＋ ＋ ＋ ＋ － － － － － － － － x y A’ D’ C’ B’ I’ H’ G’ F’ E’ p q Top view of polyhedron A ∥ x-y plane Same order as left

42. Vector on a Surface Suppose vector on surface with gradient Under orthography, vector in scene projects to is surface normal vector, so

43. Vector on Two Surfaces Suppose vector on boundary between two surfaces Surfaces have gradients and If, then p q G1G1 G2G2

44. Ordering of Points Along Gradient Line Perpendicular to Connect Edge B1B1 B3B3 B2B2 S T A B1’B1’ B2’B2’ B3’B3’ A pq If connect edge ST convex, then points on gradient space maintain same order (left-right) as A and B i in image If ST concave, then order switches

45. How does this gradient space stuff help us to label lines? L is a “connect edge” (vector on two surface) Assume orthography Line in gradient space connecting R 1 and R 2 must be perpendicular to line L ＋

46. Line Labeling using Gradient Space 1. Assign arbitrary gradient (0,0) to A 2. Consider B lines 1,2 may be connect edges or may be occluding edges 3. Suppose line 1 a connect edge 4. Suppose line 2 a connect edge, then (line A’B’) (line 2) impossible. So line 2 occluding. B A C 1 2 3 4 5 B’ A’ p q B’ p q

47. Line Labeling using Gradient Space 5. Suppose lines 3 and 4 are connect edges 6. and so forth can get multiple interpretations B’ A’ p q B’ p q C’ C ＋ － － ＋ － ＋ B A C 1 2 3 4 5

48. Another Payoff: Detect Inconsistencies R2R2 R1R1 L2L2 L1L1 L1L1 L2L2

49. Summary Can use gradient space to –represent surface orientation –detect inconsistent line labels –constraint labeled line drawings –establish line labels without the vertex catalog

50. References u M.B. Clowes, “On seeing things,” Artificial Intelligence, Vol.2, pp.79-116, 1971 u D.A. Huffman, “Impossible objects as nonsense sentences,” Machine Intelligence, Vol.6, pp.295- 323, 1971 u A.K.Mackworth, “On reading sketch maps,” 5th IJCAI, pp.598-606, 1977