1. Freehand Drawing System based on Geometric Constraints Kaisuke Nakajima Joint work with Takeo Igarashi User Interface Research Group The University of Tokyo

2. Two Parts of the Talk Introduction to Pegasus [Igarashi, UIST 1997 & CHI 1998] Improvement attempts My current study Any comments greatly appreciated!

3. Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

4. Problem Diagrams In documents, slides, … By drawing editors Difficult to satisfy geometric relations!

5. Example

6. CopyFlipMove

7. Example (contd.) In this case, grids works well, but…

8. Geometric constraints Another Example

9. The user attempts to draw a perpendicular line,

10. but grid prevents desired placements. GRID

11. He draws a box separately, and...

12. rotates it, but fails to get precise angle. ROTATE

13. Better strategy: copy the slope  rotate 90 degrees COPY ROTATE

14. The user must plan detailed strategy. Copy Flip Move Rotat e Grid Planning overhead Why Difficult?

15. Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

16. Our Idea Sketching is easy! Sketch  Beautify Interactive Beautification [Igarashi, UIST 1997] Sketch + Beautify

17. Draw and... Interactive Beautification

18. Beautify. Satisfied Constraints Draw and... Interactive Beautification

19. Beautify. Draw and... Interactive Beautification

20. Beautify. Interactive Beautification

21. But free strokes are ambiguous. Ambiguity Handling

22. Generate multiple candidates  Let the user select Ambiguity Handling

23. In this way, the user can construct precise geometric diagrams without any editing commands!!

24. Prototype System Pegasus Live Demo

25. Diagrams Drawn

26. Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

27. [Pavlidis 1985]... Beautification Systems

28. Batch-based too many errors! More Interaction (+ multiple candidates) Beautification Systems (contd.)

29. [Apple Newton] [SmartSketch]... Free Stroke Vectorization

30. Local context Free Stroke Vectorization (contd.)

31. Global contextLocal context Free Stroke Vectorization (contd.)

32. Drawing Systems based on Geometric Constraints [Sutherland 1963] [Nelson 1985]...

33. Drawing Systems based on Geometric Constraints (contd.) Previous work Explicitly specify constraints Our approach Infer constraints

34. Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

35. Beautification Algorithm Constraint Inference Segment coordinates Multiple constraints Constraint Solver Multiple candidates Candidate Evaluation Primary candidate

36. Constraint Inference Input Reference segments (array of (X 0, Y 0, X 1, Y 1 )) New stroke (x 0, y 0, x 1, y 1 ) Output Possible constraints (linear equations of (x 0, y 0, x 1, y 1 )) Perpendicular const * (x 1 – x 0 ) + const * (y 1 – y 0 ) = 0 Edge Connection x 0 = const, y 0 = const etc. (x 0, y 0 ) (x 1, y 1 )

37. Constraint Inference (contd.) How? Supported constraints if (distance(point1, point2) < 30) infer constraint(point1 == point2);

38. Constraint Solver Input Possible constraints (linear equations of (x 0, y 0, x 1, y 1 )) Output Beautification candidates (array of (x 0, y 0, x 1, y 1 )) Perpendicular Edge Connection Same Length Horizontal

39. Constraint Solver (contd.)

40. Candidate Evaluation Candidate nearest to the original stroke  Primary candidate

41. Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

42. To confirm the intuition ! “It’s difficult for novice users to use commands” “Interactive beautification is good” Purpose

43. Rapidness How fast? Precision Beautiful? Two Measures

44. Experimental Settings 18 subjects 3 diagrams, 3 editors (9 sessions / person)

45. Systems CAD... AutoSketch (AutoDesk), CAD Software. Draw... SmartSketch (Future Wave), Drawing Editor. Pegasus…Our Prototype system. On AMiTY (a pen computer)

46. Diagrams and Constraints A) B) C) Parallel-1 Parallel-2 Perpendicular Vertical and Horizontal Connected (all Vertices) Symmetric-1 Symmetric-2 Connected (all Vertices) Parallel-1 Parallel-2 Equal Interval Connected (all Vertices)

47. Results Rapidness Precision Averaged drawing time for three diagrams 0 100 200 300 400 500 600 CAD Draw Pegasus 0 20 40 60 80 100 CADDrawPegasus The ratio of sessions where all constraints are satisfied Pegasus outperformed CAD and Draw, in both rapidness and precision.

48. Evaluation: Conclusion Pegasus cannot do everything But: certainly useful in its target domain CAD Paint Geometric diagrams, rapidly & precisely!

49. Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

50. Ongoing Work Address limitations of Pegasus Only lines Too many candidates Forgets constraints

51. Limitation: Only Lines  Support curves Circles, ellipses, arcs, Bézier curves (Experimental demo)

52. Limitation: Only Lines (contd.) Challenges with curves More ambiguity Shape recognition Constraint inference Complex constraints: how to represent? e.g. line tangent to piecewise Bézier curve Complex constraints: how to solve efficiently? Exponential growth of candidate solutions

53. Limitation: Too Many Candidates  Overlapping…  Constraint switching interface Visualize conflicting constraints Enable manual switching Challenges How to detect conflicting constraints efficiently?

54. Limitation: Forgets Constraints  Remember constraints Maintain them during manipulation Chorus [Hosobe 2001] Stretch! Skewed 

55. Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

56. Summary Pegasus: freehand drawing system User study showed promising results Geometric constraints play an important role in sketch beautification Further improvements to be made

57. Thank you Papers & demos available at http://www-ui.is.s.u-tokyo.ac.jp/