CALI: An Online Scribble Recognizer for Calligraphic Interfaces By Manuel J. Fonseca Cesar Pimentel Joaqui, A. Jorge 11 November 2002

Introduction n Alternative user interface based on scribble recognition n Recognition of basic shapes (such as triangle, rectangles, circles) and basic commands (such as copy and delete) n More robust than others, trainable, multi-stroke Recognizable Shapes

Non-trainable Recognizer n Global geometric properties n Filters for identification of shapes and removal of unwanted shapes n Fuzzy logic for uncertainty in sketches n Strokes  Scribbles  Shapes n Return a list of plausible shapes ordered by degree of certainty

Geometric features (non-train.) n Compute the convex hull of shape’s points n Compute largest area triangle, quadrilateral inscribed in the convex hull and smallest area enclosing rectangle, area and perimeter n Compute percentiles of distributions of feature values for each shape (class) PolygonsPercentiles

Deriving Fuzzy Sets n Each shape is defined by several fuzzy sets n E.g.: thinness for lines n For each feature there is a trapezoidal fuzzy set with 4 values n All design of fuzzy sets is done manually Fuzzy Sets

Recognition Rules n If Scribble is very thin, then shape is a line n If A(lt)/A(lq) is like diamond and A(lq)/A(ch) is not like Ellipse andA(lq)/A(ch) is not like Ellipse and A(lq)/A(er) is not like Bold Line andA(lq)/A(er) is not like Bold Line and A(lq)/A(er) is not like rectangleA(lq)/A(er) is not like rectangle Then shape is a diamondThen shape is a diamond n Line style is identified after basic shapes

Re-segmentation n Some shapes cannot be recognized based entirely on global geometric properties n E.g.: Arrows and Crosses n If strokes >= 2 and Last stroke is like triangle orLast stroke is like triangle or Last stroke is like moveLast stroke is like move Then shape is an arrowThen shape is an arrow n If strokes == 2 and First stroke is like line andFirst stroke is like line and Second stroke is like lineSecond stroke is like line And first stroke intersects second strokeAnd first stroke intersects second stroke Then shape is a crossThen shape is a cross Arrows

Ambiguity n Return a list of plausible candidates when in doubt, ordered by degree of membership AmbiguityFuzzy Sets

Trainable Recognizer n Need for new gesture (shape) specification n Naïve Bays training n KNN should be better after implementation of indexing structure for high-D data (in the works)

Results n Non-train.: about 50 ms on a Pentium II n 95.8% recognition rate n Train.: 93.3% Confusion Matrix

Thank you for your attention Have a nice evening...

Shapes Identified by CALI

Polygons used to estimate features

Percentiles of feature values n H(er)/W(er) - aspect ratio n Bars are for 25%-75% n Lines are for 10%-90% n Line (small) n Per(ch) 2 /A(ch) - thinness n Bars are for 25%-75% n Lines are for 10%-90% n Circle (small but visible) n Lines (so small that’s invisible)

Fuzzy Sets n b and c correspond to 10% and 90% percentiles n a and b are for min and max after clearing out outliers

Different types of arrows

Ambiguity cases among shapes

Fuzzy sets for ambiguity

Confusion Matrix for non-train.