Freehand Drawing System based on Geometric Constraints Kaisuke Nakajima Joint work with Takeo Igarashi User Interface Research Group The University of Tokyo

Two Parts of the Talk Introduction to Pegasus [Igarashi, UIST 1997 & CHI 1998] Improvement attempts My current study Any comments greatly appreciated!

Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

Problem Diagrams In documents, slides, … By drawing editors Difficult to satisfy geometric relations!

Example

CopyFlipMove

Example (contd.) In this case, grids works well, but…

Geometric constraints Another Example

The user attempts to draw a perpendicular line,

but grid prevents desired placements. GRID

He draws a box separately, and...

rotates it, but fails to get precise angle. ROTATE

Better strategy: copy the slope  rotate 90 degrees COPY ROTATE

The user must plan detailed strategy. Copy Flip Move Rotat e Grid Planning overhead Why Difficult?

Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

Our Idea Sketching is easy! Sketch  Beautify Interactive Beautification [Igarashi, UIST 1997] Sketch + Beautify

Draw and... Interactive Beautification

Beautify. Satisfied Constraints Draw and... Interactive Beautification

Beautify. Draw and... Interactive Beautification

Beautify. Interactive Beautification

But free strokes are ambiguous. Ambiguity Handling

Generate multiple candidates  Let the user select Ambiguity Handling

In this way, the user can construct precise geometric diagrams without any editing commands!!

Prototype System Pegasus Live Demo

Diagrams Drawn

Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

[Pavlidis 1985]... Beautification Systems

Batch-based too many errors! More Interaction (+ multiple candidates) Beautification Systems (contd.)

[Apple Newton] [SmartSketch]... Free Stroke Vectorization

Local context Free Stroke Vectorization (contd.)

Global contextLocal context Free Stroke Vectorization (contd.)

Drawing Systems based on Geometric Constraints [Sutherland 1963] [Nelson 1985]...

Drawing Systems based on Geometric Constraints (contd.) Previous work Explicitly specify constraints Our approach Infer constraints

Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

Beautification Algorithm Constraint Inference Segment coordinates Multiple constraints Constraint Solver Multiple candidates Candidate Evaluation Primary candidate

Constraint Inference Input Reference segments (array of (X 0, Y 0, X 1, Y 1 )) New stroke (x 0, y 0, x 1, y 1 ) Output Possible constraints (linear equations of (x 0, y 0, x 1, y 1 )) Perpendicular const * (x 1 – x 0 ) + const * (y 1 – y 0 ) = 0 Edge Connection x 0 = const, y 0 = const etc. (x 0, y 0 ) (x 1, y 1 )

Constraint Inference (contd.) How? Supported constraints if (distance(point1, point2) < 30) infer constraint(point1 == point2);

Constraint Solver Input Possible constraints (linear equations of (x 0, y 0, x 1, y 1 )) Output Beautification candidates (array of (x 0, y 0, x 1, y 1 )) Perpendicular Edge Connection Same Length Horizontal

Constraint Solver (contd.)

Candidate Evaluation Candidate nearest to the original stroke  Primary candidate

Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

To confirm the intuition ! “It’s difficult for novice users to use commands” “Interactive beautification is good” Purpose

Rapidness How fast? Precision Beautiful? Two Measures

Experimental Settings 18 subjects 3 diagrams, 3 editors (9 sessions / person)

Systems CAD... AutoSketch (AutoDesk), CAD Software. Draw... SmartSketch (Future Wave), Drawing Editor. Pegasus…Our Prototype system. On AMiTY (a pen computer)

Diagrams and Constraints A) B) C) Parallel-1 Parallel-2 Perpendicular Vertical and Horizontal Connected (all Vertices) Symmetric-1 Symmetric-2 Connected (all Vertices) Parallel-1 Parallel-2 Equal Interval Connected (all Vertices)

Results Rapidness Precision Averaged drawing time for three diagrams CAD Draw Pegasus CADDrawPegasus The ratio of sessions where all constraints are satisfied Pegasus outperformed CAD and Draw, in both rapidness and precision.

Evaluation: Conclusion Pegasus cannot do everything But: certainly useful in its target domain CAD Paint Geometric diagrams, rapidly & precisely!

Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

Ongoing Work Address limitations of Pegasus Only lines Too many candidates Forgets constraints

Limitation: Only Lines  Support curves Circles, ellipses, arcs, Bézier curves (Experimental demo)

Limitation: Only Lines (contd.) Challenges with curves More ambiguity Shape recognition Constraint inference Complex constraints: how to represent? e.g. line tangent to piecewise Bézier curve Complex constraints: how to solve efficiently? Exponential growth of candidate solutions

Limitation: Too Many Candidates  Overlapping…  Constraint switching interface Visualize conflicting constraints Enable manual switching Challenges How to detect conflicting constraints efficiently?

Limitation: Forgets Constraints  Remember constraints Maintain them during manipulation Chorus [Hosobe 2001] Stretch! Skewed 

Outline 1. Problem 2. Our Idea & Demonstration 3. Related Work 4. Algorithm 5. Evaluation 6. Ongoing Study 7. Summary

Summary Pegasus: freehand drawing system User study showed promising results Geometric constraints play an important role in sketch beautification Further improvements to be made

Thank you Papers & demos available at