Superimposing Conceptual Models with SPARCE Sudarshan Murthy 1, David Maier 1, Lois Delcambre 1, Shawn Bowers 2 http://cse.ogi.edu/sparce mailto:smurthy@cse.ogi.edu 1 Department of CSE, OGI School of Science & Engineering at OHSU 2 San Diego Supercomputer Center, University of California, San Diego

Superimposing Conceptual Models with SPARCE *Picture courtesy of Prof. James Pankow, Dept. of EBS, OGI Annotating* Mark up paper Place sticky notes on the paper Write on 3x5 cards and leave pointers to relevant portions An annotation might not capture all the information we need. We sometimes examine context to reconnect 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Observations People often superimpose new interpretations onto existing information They like to combine existing information and their interpretations (in some meaningful way) to get “their” view 18-Feb-19 Superimposing Conceptual Models with SPARCE

Tasks Influence Models Environmental scientists at OGI intend to study nicotine and carcinogen delivery in mainstream tobacco smoke Main information source: tobacco industry documents The Legacy Tobacco Documents Library (Univ. of California, San Francisco) has over 6 million documents 18-Feb-19 Superimposing Conceptual Models with SPARCE

Initial Classification 18-Feb-19 Superimposing Conceptual Models with SPARCE

Detailed Classification and Annotation 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Information Overlaying new information on top of existing information Add new data Impose new schema and model Benefits No replication of existing information Preserve context Make new connections between parts of existing information Flexible (multiple) organization Abstractions: Mark, Context Heterogeneous sources: Word, Excel, PDF, HTML,… 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Goal Develop middleware architecture to facilitate superimposed information management (at sub-document granularity) The Superimposed Pluggable Architecture for Contexts and Excerpts (SPARCE) is our middleware architecture 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Outline Motivation and goal Superimposed applications and conceptual models SPARCE overview Related work Future work and conclusion 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposed Applications These are applications that manipulate superimposed information They are free to choose display and data models based on their needs They need to accommodate marks and contexts A user can activate a mark to navigate to the base layer or examine context without expressly navigating to the base layer 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Superimposed Applications for the US Dept. of Agriculture, Forest Service 18-Feb-19 Superimposing Conceptual Models with SPARCE

RID 18-Feb-19

Superimposing Conceptual Models with SPARCE RIDPad A superimposed application an editor might use to collect and organize information for a RID letter 18-Feb-19 Superimposing Conceptual Models with SPARCE

RIDPad’s Information Model Name RIDPadDoc ID Size Location Group Description Item Belongs to 0..1 * Contains Address Mark 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Groups Items Uses Word Mark 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Schematics Browser A superimposed application Forest Service personnel might use to review past appeal decisions A superimposed schematic is a ER schema over base information Superimposed information is stored in a relational database 18-Feb-19 Superimposing Conceptual Models with SPARCE

SPARCE: Superimposed Pluggable Architecture for Contexts and Excerpts 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Overview Superimposed Layer Base Layer Acrobat SA 1 SPARCE XML Marks Word SA 2 <mark ID=“…”> <type>…</type> <address>…</address> … </mark> Relations 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Marks A Mark is a reference to a base-layer element Several mark implementations exist Addressing scheme usually depends on the base type PDF mark uses page no. + starting and ending word index; MS Word mark uses starting and ending character index Marks provide uniform interface across base types and access protocols 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Excerpts and Context Name Value Excerpt Cheatgrass, Bromus tectorum, grows near many caves in this project area. HTML Cheatgrass,  <i>Bromus tectorum </i>,   grows near many caves in this project area. Section Heading Noxius Weeds and Exotic Plants Context is information about a marked region Context element is one piece of context information Context definition varies across and within base types 18-Feb-19 Superimposing Conceptual Models with SPARCE

Architecture Diagram (Simplified) An excerpt is a context element 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Related Work Visions Memex (1945), Evolutionary List File (1965) Hypertext NoteCards, Intermedia, Dexter (1987-1994) Compound document OpenDoc, OLE 2 (1994-95) Others Multivalent Documents (1996) 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Future Work Quantitative architecture evaluation Incremental superimposed modeling Representing context Other operations on marks and contexts Querying contexts Caching context Sharing superimposed information 18-Feb-19 Superimposing Conceptual Models with SPARCE

Conclusion SPARCE allows superimposing of different conceptual models over base information Heterogeneous base types No interference with base information Sub-document granularity It allows development of new superimposed applications, and to support new base-layer types and contexts with minimal effort Contact me for a SPARCE demo 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Questions? 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Excerpts and Contexts An Excerpt is the content of a marked region Type of an excerpt varies: text, graphics, … Context is information related to a marked region as gleaned from the base layer Context element is one piece of context information Section heading, containing paragraph, font name are example context elements 18-Feb-19 Superimposing Conceptual Models with SPARCE

The Forest Service Appeal Process* *With thanks to John Davis, USDA FS The Forest Service Appeal Process* The US Dept. of Agriculture, Forest Service (FS) routinely makes decisions to solve or prevent problems concerning forests Any FS decision can be appealed An editor reviews issues raised and prepares an appeal packet for a reviewing officer The packet contains a RID letter 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Accommodating Marks Marks bridge the conceptual models used in the two layers The model in the base layer can’t be changed Extend the superimposed model with marks Extend the superimposed schema Use constructs in the superimposed model to insert marks 18-Feb-19 Superimposing Conceptual Models with SPARCE

Example: XML Superimposed Model <Group name=‘Decision’> <Item name=‘FONSI’> <sparce:Mark id=‘…’> … </Mark> Finding of no significant … </Item> </Group> <Group name=‘Decision’> <Item name=‘FONSI’ sparce:mark-id=‘…’> Finding of no significant … </Item> … </Group> <Group name=‘Decision’> <Item name=‘FONSI’> Finding of no significant … </Item> … </Group>  18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE RID Letter Challenges Composing a RID letter requires an editor to maintain large working sets (and several threads of organization) Too many documents (windows) open at once Some appeal packets contain over 100 documents Locating information in large documents can be tedious 18-Feb-19 Superimposing Conceptual Models with SPARCE

Tasks Influence Models Researchers collect information without much thought about its organization in initial stages Don’t know enough to create a schema In later stages of research they like the information system to provide more structured access A schema is likely imposed; a query system might be expected The base corpus in the two cases might intersect, but the behaviors in the superimposed layer differ 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Example Some environmental scientists at OGI intend to study nicotine and carcinogen delivery in mainstream tobacco smoke Main information source: tobacco industry docs UCSF LTDL has > 6 million docs (> 15 million pages) First, they like to simply identify relevant docs. Later, they like better definition, finer classification and varying organizations for data They might vary tools/models to suit their tasks 18-Feb-19 Superimposing Conceptual Models with SPARCE

Benefits of Superimposing Information No replication of existing information Preserve context Make new connections between parts of existing information Flexible (multiple) organization Assure freshness 18-Feb-19 Superimposing Conceptual Models with SPARCE

What is Superimposed Information? Heterogeneous sources: Word, Excel, PDF, HTML, … 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Objectives Address base information regardless of its type, location, and access protocol Retrieve excerpts and contexts Use the same programmatic interface to work with any base layer Allow superimposed applications to use any display and data model View excerpts and contexts side by side with superimposed information 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Schematics Browser A superimposed application FS personnel might use to review past appeal decisions A superimposed schematic is a schema in the ER model superimposed over base information Uses a model more complex than RIDPad’s Stores superimposed information in a relational database 18-Feb-19 Superimposing Conceptual Models with SPARCE

Schematic Browser’s Information Model 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Summary of Results Used SPARCE in two superimposed applications Schematics Browser now has access to all features of SPARCE No base-type specific change made to SPARCE Context agents take care of specific needs Added support for new base-layer types, context kinds and elements without needing to change SPARCE or its applications 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Implementation Microsoft Windows using ActiveX Base applications currently supported MS: Word, Excel, Internet Explorer , Media Player Adobe Acrobat Kinds of context elements managed Content, presentation, containment, placement, sub-structure, topology, document, application 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Reference Model 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Considerations Excerpt type varies, many context kinds possible Contexts can vary between base-layer types Excel mark has row number, PDF mark does not Contexts can vary between marks of same base-layer type Some HTML marks may have a “column heading” Context of a mark may change with time A mark to a figure in MS Word may lose “caption” 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Some Statistics Applications Develop RIDPad: 30 hours Alter Schematics Browser: 6 hours Base types and contexts MS Word and Excel: 7 hours (combined) Windows Media Player: 8 hours Adobe Acrobat: 12 hours (includes six hours research and SDK learning time) 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Comparison NoteCards Intermedia Dexter OpenDOC OLE 2 SPARCE Base type 2 3 Any Base location Proprietary FS Base granularity Whole Part Both Dependency Lisp MacDraw None CORBA COM Context kinds Content Many 18-Feb-19 Superimposing Conceptual Models with SPARCE

Applications Need Contexts <font name=”Times New Roman” size=”3”>Cheatgrass,  <i>Bromus tectorum</i>, grows near many caves in this project area.</font> <font size=”3”>Cheatgrass,  <i>Bromus tectorum</i>, grows near many caves in this project area.</font> Cheatgrass,  <i>Bromus tectorum</i>, grows near many caves in this project area. Three possible HTML markups for the highlighted region 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Desiderata Functionality Reusability Modifiability Extensibility Usability Package flexibility Testability 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Mark Creation 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Reusable Viewers Context Browser Browse context hierarchy Context Viewers View any context element Clipboard Viewer Select mark fodder Other user interface elements Property pages and selection dialogs 18-Feb-19 Superimposing Conceptual Models with SPARCE

SPARCE Classes and Interfaces Class/Interface Description Mark A mark to base-layer information. Container The base document (or a portion of document) in which a mark is made. Application The base application in which a mark is made. Context-Aware Object (interface) Interface to any base-layer element that is able to provide its context. The classes Mark, Container, and Application implement this interface. Context Element A single piece of context information about a context-aware object. Context The context of a context-aware object. It is a collection of context elements. Implemented as a property set. Context Agent (interface) Interface to any base-layer. An implementation will retrieve context from a context-aware object. There is usually one implementation per base-layer type. SPARCE Manager Creates, stores, and retrieves marks; associates context-aware objects with appropriate context agents. 18-Feb-19 Superimposing Conceptual Models with SPARCE

Supporting New Base-Layer Type Study the base layer to understand support for marking This study should include understanding the addressing scheme for the base layer Decide which mark creation scenarios to support Implement mark fodder generation Mark fodder generation is done when mark fodder is copied to the Clipboard Decide what context elements to support Implement a context agent for the base layer The context agent must be packaged in an ActiveX server and registered with the operating system for the current SPARCE implementation 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Evaluation Procedure Evaluation Step Qualities Evaluated Implement SPARCE, support one base-layer type (MS Word) Functionality, usability Implement one superimposed application (RIDPad) Support new base-layer types and new context elements (MS Excel, Adobe Acrobat) Modifiability, reusability, extensibility Implement viewers Functionality, usability, reusability, Testability Implement second superimposed application (Schematics Browser) Functionality, reusability Change deployment scenarios Package flexibility 18-Feb-19 Superimposing Conceptual Models with SPARCE

Tools and Technologies MS Visual Studio 6 (C++ and Visual Basic) MS Office MS XML Parser 4.0 Adobe Acrobat 5 SDK ActiveX (Component Object Model) 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Packages Package Type Mark and context management ActiveX EXE Mark fodder generation and Clipboard monitoring Clipboard Viewer ActiveX DLL Clipboard access helpers Regular DLL Context Browser and Viewers Adobe Acrobat plug-in MS Office and PDF context agents MS Office add-in 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE DN, FONSI 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE EA 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Appeal Letter 18-Feb-19 Superimposing Conceptual Models with SPARCE

Superimposing Conceptual Models with SPARCE Appeal Decision 18-Feb-19 Superimposing Conceptual Models with SPARCE