1. SPONSORED BY Data-driven Segmentation and Labeling of Freehand Sketches Zhe Huang, Hongbo Fu, Rynson W.H. Lau City University of Hong Kong

2. SA2014.SIGGRAPH.ORG SPONSORED BY Objective Derive part-based segmentation and labeling from a sketch.

3. SA2014.SIGGRAPH.ORG SPONSORED BY Motivation Assembly-based modeling using sketch input [Funkhouser et al. 2004]

4. SA2014.SIGGRAPH.ORG SPONSORED BY Related Works [Sun et al. 2012] Free Hand-Drawn Sketch Segmentation [Gennari et al. 2005] Combining geometry and domain knowledge to interpret hand-drawn diagrams [Noris et al. 2012] Smart Scribbles for Sketch Segmentation

5. SA2014.SIGGRAPH.ORG SPONSORED BY Handling Freehand Sketch is Hard Arbitrary stroke ordering A single stroke may cover multiple components. A single component may have multiple strokes.

6. SA2014.SIGGRAPH.ORG SPONSORED BY Handling Freehand Sketch is Hard Where is the chair back?

7. SA2014.SIGGRAPH.ORG SPONSORED BY Our Method

8. SA2014.SIGGRAPH.ORG SPONSORED BY Our Intuition Find the best combination of components to explain the sketch Leverage the relationship between parts to narrow down the search.

9. SA2014.SIGGRAPH.ORG SPONSORED BY Overview of Our Method Inferred Structure Sketch Labeling Input Sketch (Chair) Local Interpretation (Matching) Global Interpretation (Combinatorial Search) Shape Repository (Chair)

10. SA2014.SIGGRAPH.ORG SPONSORED BY Preprocessing

11. SA2014.SIGGRAPH.ORG SPONSORED BY Specifying 3D View Sketch Manually specified 3D view by aligning a 3D model User Interaction

12. SA2014.SIGGRAPH.ORG SPONSORED BY Over Segmentation Sketch (Color by strokes) Breaking at junction or corner Stroke Segments (Color by segments) In later stages: Segmentation = grouping stroke segments Labeling = labeling each stroke segment

13. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation

14. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Candidates

15. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Convolution Score Map

16. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Local Maximums

17. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Clustered

18. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() Cluster Centers

19. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() ICP Refinement before after Cluster Centers

20. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Element OutlineSketch conv() ICP Refinement before after Final Candidates Cluster Centers

21. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Sketch Element Outline conv() Match

22. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Sketch Element Outline Match All Candidates All chair backs

23. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Sketch Element Outline Match Filtered Candidates All chair backs

24. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation

25. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation

26. SA2014.SIGGRAPH.ORG SPONSORED BY Local Interpretation Possible Connections

27. SA2014.SIGGRAPH.ORG SPONSORED BY Global Interpretation

28. SA2014.SIGGRAPH.ORG SPONSORED BY Formulation Problem: Find an assignment to X and Y such that the cost E(X,Y) is minimized. Note: In the paper, the cost function is E(X,Y,U), where U is stroke segment assignment variable  i-th candidate is selected  i-th and j-th candidates are connected Decision Variables:

29. SA2014.SIGGRAPH.ORG SPONSORED BY Cost Function Fitness Connection is better than because

30. SA2014.SIGGRAPH.ORG SPONSORED BY Cost Function Overlap Coverage Others… (see the paper for detail) is better thanbecause is better than because Overlap No coverage

31. SA2014.SIGGRAPH.ORG SPONSORED BY Results and Evaluation

32. SA2014.SIGGRAPH.ORG SPONSORED BY Results Ours Direct Retrieval [Xu et al. 2011] [Eitz et al. 2012] [Shen et al. 2012]

33. SA2014.SIGGRAPH.ORG SPONSORED BY Quantitative Evaluation 10 classes, 300 sketches in total

34. SA2014.SIGGRAPH.ORG SPONSORED BY Quantitative Evaluation 10 classes, 300 sketches in total

35. SA2014.SIGGRAPH.ORG SPONSORED BY Application 3D reconstruction if the labeling is perfect.

36. SA2014.SIGGRAPH.ORG SPONSORED BY Limitation Incorrect local interpretation. Hollowness ambiguity Missing small elements.

37. SA2014.SIGGRAPH.ORG SPONSORED BY Thank You