Building Synthetic Graphical Documents for Performance Evaluation of Symbol Recognition M. Delalandre 1, E. Valveny 1 and T. Pridmore 2 1 CVC, Barcelona, Spain 2 SCSIT, Nottingham, UK SymbolRec Meeting CVC, Barcelona, Spain Thursday 13th December 2007

Plan Introduction Building Engine Constraint Editor Use Cases Conclusions

Introduction (1/4) Graphics Analysis Systems and Performance Evaluation Analysis Document Characterisation Groundtruth Groundtruthing Results Performance evaluation Evaluation Performance evaluation 1. Analysis and Evaluation 2. Groundtruthing 3. Problematic & Approach 4. Our System Symbol recognition systems results label + localization Symbol Recognition spotting, retrieval, understanding, …

Introduction (2/4) Groundtruthing Document Groundtruth Groundtruthing Based on real-life documents Advantages : based on real life documents Drawbacks : copyright, cost, length, error, content control Akoy’00 binary noise vectorial noise Valveny’03 Based on synthetic documents Document Groundtruth Groundtruthing Setting Advantages : easy to produce, few constraint Drawbacks : not real Past works 1. Analysis and Evaluation 2. Groundtruthing 3. Problematic & Approach 4. Our System

Introduction (3/4) Problematic & Approach 1. Analysis and Evaluation 2. Groundtruthing 3. Problematic & Approach 4. Our System symbol background Graphical documents are composed of two layers To use a same background layer with different symbol layers To associate symbols to positioning constraints and to select them at random Our approach c2c2 c1c1 Problematic How to build whole documents ? How to deal with the real in the synthetic ? How to make an usable approach ? connected 1-connected 2-connected Production rules 0-n 1 1 O-n ‘Creasy’ but well formed drawing

Introduction (4/4) Our System GT Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints Four components The Engine Symbol Model & Factory Symbol Positioning Building Manager Constraint Editor

Plan Introduction Building Engine Constraint Editor Use Cases Conclusions

Building Engine Symbol Models & Factory Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints (x,y) begin end width begi n end center width center radius Symbol models One file, one model Vector graphics format (point, line, arc, circle) Symbol factory x,y x’,y’ dxdx dydy  l arc is directed to  because  is lower than γ 0 2Π2Π  γ  trigonometric d1d1 d2d2 d2d2 2π -   d 1 and d 2 are the two lines’ directions

Building Engine Constraints & Symbol Positioning Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints dxdx dydy (2)(3) (4) l  (1) (2) (3) (4) How to select the control point ? Define for each constraint by polar unit coordinate (ρ,  ) computed from bounding box 0 1 0,5 r How to select the anchor point ? (a) (b) 0 1 0,5 l = r  L L b e r p l point 0  01 The polygon includes the point if |  01 +  12 + ……  60 | = 2  π  x0x0 y0y0 y 0 +dy x 0 +dx x 0 +r x d x y 0 +r y d y 0  r x,y  1 Just a fixed value (x,y) Taken at random belong a line Taken at random inside a zone control point anchor point selection (4) positioning on background Constraint, main principle (2) computation of control point (1) rotation(3) sloping

Building Engine Building Manager (1/2) Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints L overlap p if dx 1  dx 2 < 0 or dy 1  dy 2 < 0 dx 1 dx 2 dy 1 dy 2 p L L overlaps L if e or b are overlapped be Boxes are overlapped if L 1 overlaps L 2 or L 2 overlaps L 1 and L 3 overlaps L 4 or L 4 overlaps L 3 L1L1 L2L2 L3L3 L4L4 Symbol overlapping L covers L if e and b are overlapped be L left bottom right up symbol overflows L if right or up or left or bottom is not covered by L Line overflow p1p1 p2p2 p3p3 p4p4 symbol overflows P if P doesn’t include p 1 or p 2 or p 3 or p 4 P P Zone overflow w c1 = 0.33 w c2 = 0.33 w c3 = 1.00 w s1 = 0.33 w s2 = 0.33 w s3 = 0.66 w s4 = 0.33 w s5 = 0.33 w s6 = 1.00 p s1 = 0.11 p s2 = 0.11 p s3 = 0.22 p s4 = 0.11 p s5 = 0.11 p s6 = 0.33 s1s1 s2s2 s3s3 s4s4 s5s5 s6s6 c1c1 c2c2 c3c3 (c)onstraint and (s)ymbol Symbol selectionStopping criterion When building failures (overlapping, overflow, capacity) become upper to number of wished symbol we stop the process

Plan Introduction Building Engine Constraint Editor Use Cases Conclusions

Constraint Editor Introduction Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints Constraint Setting Model Selector Constraint Selector Model Viewer Building Bar Constraint Locator Building Viewer Symbol Models Building Engine Run View Edit

Constraint Editor Background and Models Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints Model loading, selection and viewing loading select ‘scroll & click’ automatic zooming attach focus ‘click and well’ shift ‘drag’ Background attachment and browsing

Constraint Editor Locate and Link the Constraints ‘click’ ‘click, move, click’ or’ ‘pressed, drag, release’ ‘click(s), double click’ Locate the constraints select ‘click’ link model add ‘right double click’ delete ‘left double click’ Link the constraints Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints

Constraint Editor Building View Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints view ‘CRTL+V’ view focus ‘click and well’ shift ‘drag’ keeping view Viewing and browsing Some views

Constraint Editor Wrap, Scale and Rotate Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints Wrap ScaleRotate

Constraint Editor Control and Slope Symbol Models Building Manager Symbol Factory Symbol Positioning Constraints Control Slope

Plan Introduction Building Engine Constraint Editor Use Cases Conclusions

Use Cases Work Group Marçal Rusinol (CVC), Ernest Valveny (CVC), Mathieu Delalandre (CVC), Jean-Yves Ramel (LI), Rashid Qureshi (LI), Philippe Dosch (LORIA), Tony Pridmore (SCSIT) and Hervé Locteau (LITS). LITIS Rouen CVC Barcelona SCSIT Nottingham LI Tours LORIA Nancy

Use Cases Architectural drawings Dataset: 1000 images, symbols, 16 models Constraints: connectivity and heterogeneous backgrounds 10 backgrounds filled, empty, textured

Use Cases Bags of symbol Dataset:1600 images,  symbols, 150 models Constraints:scalability, rotation and scaling rotation & scaling 25,50, 100 and 150 models 16 bases 100 images (1024) 2 / base 10 symbols / image

Conclusions and Perspectives Conclusions First valid system to build whole graphic synthetic documents Built documents kept meaning and seem realistic No constraint on database sizes (Thousands …..) Contents of documents can be controlled Can be use on several backgrounds with “reasonable” work costs using the GUI (1/4 hour per background) Perspectives Databases under packaging for GREC contest 2007 To build different document types (diagram, maps) To propose a characterization algorithm working with the produced groundtruth