1. The Diagram Understanding System Inspector http://www.ccs.neu.edu/home/bguthrie/dusi Brian Guthrie bguthrie@ccs.neu.edu 11/10/2005

2. What is DUSI? ➔ Visual aid for writers of diagram grammars ➔ Prototype for end user systems – interact with the literature ➔ New modality of information presentation ➔ Written for CCIS's Biological Knowledge Laboratory

3. Understanding Diagrams ➔ Vector images (objects) vs. raster images (pixels) ➔ PDF, SVG or similar formats ➔ Analysis strategies and systems for diagrams ➔ Spatial index for grapheme identification

4. Some Examples of Graphemes Graphemes are made up of, or are, graphic primitives. Diagrams are made up of graphemes. graphic primitive : grapheme :: letter : word grapheme : diagram :: word : essay A Vertical Tick Horizontal TickLine SegmentCurve Callout Adjacent Rectangles Bars Branch Data Point

5. Why parse diagrams? ➔ Knowledge/data mining and extraction – Identify diagram “type” – Build a corpus of categorized diagrams ➔ Image transformation (resize, recolor) ➔ Semantic characterization of content ➔ Interaction with components

6. Design goals for DUSI ➔ Inspect internals of diagram and its parse structure ➔ User-driven manipulation of diagrams ➔ Attractive, functional UI ➔ Communication with other systems, external API

7. DUSI's Role: The Parsing Process ➔ Display partially or fully parsed diagram ➔ Allow authors of grammars to examine constituent structure ➔ Analyze the diagram's grapheme structure (a partial parse) Line2D Ellipse2D Area Polygon Polygon Rectangle... VTickMark (22) HTickMark (22) LineSegment (34) HAdjRect (11) LineIntersect (3) Machine Learning – Mingyan Shao (Perceptron, LogitBoost) Visualization and refinement of the parse (DUSI)

8. DUSI's Role: User Interface ➔ Active links among: – diagram itself – parse tree – text references ➔ Rich, interactive alternative to HTML, PDF ➔ Interface into open-access publishers like BioMedCentral

9. Why open-access publishing? ➔ “Open source” for scientific literature – Free to read, copy, download, distribute, print, search, link, index, restructure, republish – Publication charge paid by author (members of many institutions publish free) ➔ Large corpus of documents already in XML/PDF format (BioMedCentral, http://biomedcentral.com)

10. Future goals for DUSI and the BKL ➔ Direct link to diagram/text corpus, allow search ➔ Hyperlinks between text content and diagram content ➔ Ability for user to “mark up” a diagram, add semantic meaning ➔ Maybe someday AJAX, RMI

11. Questions? Everybody loves a kitten.