Download presentation
Presentation is loading. Please wait.
1
Graphics Recognition – from Re-engineering to Retrieval Karl Tombre, Bart Lamiroy LORIA, France
2
Document Analysis in the IR era n Information is at the core of industrial strategies n A lot of digital or digitized information, but often in very “poor” formats n The challenge: not necessarily re- engineering of documents, but enrich poorly structured information, add (limited) amount of semantics, build indexes ðPurposes: browsing, navigation, indexing ðDAR methods and tools useful, but must be adapted
3
Specific challenges of large- scale IR applications n Genericity: we cannot necessarily build a complete and exhaustive a priori model of contextual knowledge (ontology) n Adaptability: various input data – scanned paper, PDF, DXF, HTML, GIF… – various resolutions n Robustness: “back-office” applications n Efficiency: online searching in heterogeneous data n Scaling: methods have to scale to increasing number of symbols/features
4
DAR and IR n Media without (or with very little) contextual knowledge ðImage-based indexing and retrieval, indexing of video sequences n Documents do explicitly convey information from one person to another person ðMuch more structure, syntax and semantics
5
DAR and IR – some examples n Indexing and/or searching scanned text without OCR ðSimilarities, signatures n Query or index on layout structure n Table spotting n Keyword spotting n …
6
What about Graphics Recognition? n Subfield of DAR, for graphics-rich documents n Numerous methods for various analysis and recognition problems u Raster-to-vector conversion u Text/graphics separation u Symbol recognition n Many specific technical areas: maps, architectural drawings, engineering drawings, diagrams and schematics, …
7
Graphics recognition methods n Text/graphics separation
8
n Vectorization Graphics recognition methods
9
Graphics recognition and IR applications n Usual text-based indexing and retrieval still useful n But need for access to other kinds of information: u Symbols u Text-drawing connections u Description-illustration connections
10
Some contributions n Syeda-Mahmood – maintenance drawings IEEE Trans. On PAMI 21(8):737-751, Aug. 1999
11
Some contributions n Arias et al., Najman et al. – use of information contained in legend / title block Proc. GREC’01, Kingston (Ontario, Canada), p.19-26, Sept. 2001
12
Some contributions n Samet & Soffer – symbols from legend IEEE Trans. On PAMI 18(8):783-798, Aug. 1996
13
Some contributions n Müller & Rigoll – graphical retrieval in database of engineering drawings Proc. ICDAR’99, Bangalore (India), pp. 697-700, Sept. 1999
14
Some contributions n Boose et al. (Boeing) – Generation of Layered Illustrated Parts Drawings (GREC’ 03) Proc. GREC’03, Barcelona, pp. 139-144
15
Wishful thinking? Symbol DB Or even better…
16
Symbol recognition n Natural features for indexing and retrieval n Most methods work with known databases of reference symbols – what about interactive querying of arbitrary symbols? n From segmentation followed by recognition, to segmentation-free recognition, or segmenting while recognizing n Scalability ûEfficiency / complexity ûDiscrimination power ðSignatures Before we move on: 1st contest on symbol recognition held last week See IAPR TC10 homepage for further details
17
Image-based signatures n Compute invariant signatures on binary document image F -signatures (ICDAR’01) u Radon transform: R-signatures [Tabbone & Wendling] u Ridgelets [Ramos Terrades & Valveny – GREC’03] – aka wavelet transform of Radon transform
18
R-signatures Detection of arrowheads [Girardeau & Tabbone] DEA degree thesis, INPL, Nancy, Jul. 2002
19
R-signatures Another example [Girardeau & Tabbone]
20
Ridgelets [Ramos Terrades & Valveny – GREC’03] Proc. GREC’03, Barcelona, pp. 202-211
21
Vector-based signatures [Dosch & Lladós – GREC’03] n Based on set of basic graphical features: u Parallelism u Overlap u Collinearity u T- and V-junctions n Quality factor associated with the various relations n Match signatures of reference symbols with signatures of buckets
22
Vector-based signatures Proc. GREC’03, Barcelona, pp. 159-169
23
Towards symbol spotting n Pre-compute – or compute on the spot – a set of basic signatures n Can be sufficient for symbol spotting and retrieval n Followed by classical symbol recognition if more discrimination is needed
24
Symbol spotting n [Jabari & Tabbone] : graph matching through probabilistic relaxation, with nodes=segments and vertices=relations DEA degree thesis, INPL, Nancy, Jul. 2003
25
Symbol spotting n [Jabari & Tabbone] : another example
26
Combining Text and Graphics n Extracting Text/Graphics relationships within document n Using Text matching for inter-document relationships n Transitive inter-document Graphics matching n No need for complex graphics matching n Restricted to well known document types
27
Example: continuation of Wiring Diagrams (Boeing) n [Baum et al. – GREC’03] Proc. GREC’03, Barcelona, pp. 132-138
28
Scan2XML Example Proc. GREC’01, Kingston (Ontario, Canada), pp. 312-325
29
Indexing and Semantics n Signature + metric n Semantics = measured distance to signature n Applies only to homogenous contexts u Pre-segmented images u Pre-determined image classes u Implicit application of domain kowledge u... n Semantics = Syntax
30
Example Signature type A Metric M Semantics1 = ( 1, 1) Semantics2 = ( , 2) Signature value M( M( semantics = measurement to reference value
31
Heterogenous Document Bases n Semantics do not have a unique syntax anymore n Syntax metrics may be context sensitive n Semantics = Syntax + Context Context needs to be considered
32
Two different contexts from the automobile industry
33
Example Context 1: Signature type A Metric M ( 1, 1) = Semantics1 = ( 1, 1) ( , 2) = Semantics2 = ( , 2) Context 2: Signature type B Metric N Signature value What if M( and N(
34
A step to taking into account context (while consolidating existing approaches) Component Algebra : u Image Analysis = Pipeline u Syntax + algorithm = semantics Algorithm Data (syntax) Data (semantics) Algorithm Data (semantics) Syntax and semantics need not be distinguished
35
Component Algebra n Components : Known and implemented document analysis algorithms, taking input data from one domain, and producing data into another domain. n Application Context : Set of all available Components. n Semantics : Data sets needed by or produced by Components.
36
Component Algebra is a Graph Component Data Component Data Component
37
Advantages n Each node is a semantic concept, semantic relationships are explicitly expressed. n Structure may support automatic reasoning and knowledge inference. n Context is embedded in components, different contexts give different paths in the graph. n Highly scalable and open architecture. n Bridge between signal-level document analysis and high-level document representation.
38
However... The formalism exists, the realization doesn't (yet) n What about parametrization ? n How context independant can you get ? n What about « guessing » context appropriateness ? n How to design fully interoperable components ?
39
Conclusion n A lot of DA methods – and more specifically GR methods – can be of direct use in IR, indexing and browsing applications n Specific challenges u Scaling and efficiency u Heterogeneous sets of documents u Incomplete domain knowledge u Symbol spotting u On-the-fly symbol searching n Sketch of open framework for including document semantics when context can be heterogeneous
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.