Superimposed Information - INRIA - April Lois Delcambre Technology for Superimposed Information Lois Delcambre with Shawn Bowers, David Maier, Mat Weaver Database and Object Technology Lab Computer Science and Engineering Department Oregon Graduate Institute

Superimposed Information - INRIA - April Lois Delcambre Outline introduction to superimposed information a superimposed application: SLIMPad (DLI2 Project) model-based representation and transformation of information harvesting information to sustain our forests (NSF Digital Government project)

Superimposed Information - INRIA - April Lois Delcambre What is Superimposed Information? data “placed over” existing information sources to:  highlight  annotate  elaborate  select  collect  organize  connect  reuse information elements often to support new applications, beyond the original

Superimposed Information - INRIA - April Lois Delcambre Examples of Superimposed Information  Non-electronic examples:  Commentaries on religious texts, law, literature  Concordances, citation indexes  Electronic examples:  Your bookmark file in your web browser  RDF metadata

Superimposed Information - INRIA - April Lois Delcambre Why work on it now? Broadening range of digital information Accessibility/addressability to base information –Reference (e.g., URL) can be resolved quickly –Addressing at various levels of granularity Emerging Standards: RDF, Topic Maps, Xlink Emerging Applications: –Third Voice, Multi-Valent Documents,...

Superimposed Information - INRIA - April Lois Delcambre The superimposed and base layers with marks Superimposed Layer Base Layer Information Source 1 Information Source 2 Information Source n … marks Focus: building generic technology

Superimposed Information - INRIA - April Lois Delcambre Superimposed Applications enhanced base layer application simultaneous use of base and superimposed applications web browser (with extra functions such as annotation) superimposed application base application 1 base application 2...

Superimposed Information - INRIA - April Lois Delcambre Outline introduction to superimposed information a superimposed application: SLIMPad (DLI2 Project) model-based representation and transformation of information harvesting information to sustain our forests (NSF Digital Government project)

Superimposed Information - INRIA - April Lois Delcambre Paul Gorman, MD Lois Delcambre, PhD David Maier, PhD

Superimposed Information - INRIA - April Lois Delcambre Bundles in the wild……….. Observational team: Paul Gorman Joan Ash Mary Lavelle Jason Lyman …………..Bundles in captivity Computer science team: Lois Delcambre Dave Maier Shawn Bowers Longxing Deng Mathew Weaver

Superimposed Information - INRIA - April Lois Delcambre Let’s take a trip to the ICU

Superimposed Information - INRIA - April Lois Delcambre (Wild) Bundles

Superimposed Information - INRIA - April Lois Delcambre (Wild) Bundles

Superimposed Information - INRIA - April Lois Delcambre (Wild) Bundles

Superimposed Information - INRIA - April Lois Delcambre (Wild) Bundles manage information for diverse, complex tasks contain selected, collected, structured, annotated are often used in settings with: –high uncertainty –low predictability –potentially grave outcomes –time & attention are highly constrained

Superimposed Information - INRIA - April Lois Delcambre (Wild) Bundles There is benefit in creating (active processing of information) There is benefit in reusing (trigger memory) There is benefit in sharing (establish collective, situated awareness)

Superimposed Information - INRIA - April Lois Delcambre Given…. bundles are everywhere! access to bundles provides access to important information information in bundles is often copied from other information sources we can keep copied/referenced information linked through the use of marks

Superimposed Information - INRIA - April Lois Delcambre (Captive) Bundles SLIMPad - a scratchpad application to create bundles but….with marks linked electronically built using our architectural for building superimposed applications inspired by the observational work (but not focused on a specific medical task) a very simple tool … with minimal interface to base layer

Superimposed Information - INRIA - April Lois Delcambre SLIMPad demo

Superimposed Information - INRIA - April Lois Delcambre Superimposed Layer Information Manager (SLIM): Contributions of the Architecture Mark Management - to create/resolve marks SLIM DMI (data manipulation interface) - for the application developer TRIM store - for generic storage of superimposed information

Superimposed Information - INRIA - April Lois Delcambre

Superimposed Information - INRIA - April Lois Delcambre

Superimposed Information - INRIA - April Lois Delcambre SLIM API: as seen by application

Superimposed Information - INRIA - April Lois Delcambre What’s Next for this Project? Validation - cardiologists, ICU nurses, … Extend the informational model of SLIMPad Extend SLIMPad to suit a selected medical task Validation of observational work and SLIMPad technology in another domain (forest information - preparation of a watershed assessment)

Superimposed Information - INRIA - April Lois Delcambre Outline introduction to superimposed information a superimposed application: SLIMPad (DLI2 Project) model-based representation and transformation of information harvesting information to sustain our forests (NSF Digital Government project)

Superimposed Information - INRIA - April Lois Delcambre Model Schema Data Instance Data with Marks Information Source 1 Information Source 2 Superimposed Layer Base Layer marks Model-Based Superimposed Information But the model and schema are optional

Superimposed Information - INRIA - April Lois Delcambre Our Goals Represent superimposed information generically, for various data models (with the schema optional) Build a single management system for superimposed information Mix information from various models Convert information from one representation scheme to another

Superimposed Information - INRIA - April Lois Delcambre Transforming Information Generic Rep. (XML model) Generic Rep. (XML model) convert Generic Rep. (Topic Map model) XML DB XML Viewer SQL TM Browser Painting Painter by painter Influenced by mentionedbiographymentionedcritiqued convert Generic Rep. (Relational model)

Superimposed Information - INRIA - April Lois Delcambre Our Approach Metamodel –to represent multiple data models Generic, Uniform Representation Scheme –to store model, schema, and instances for model-based information Mapping Formalism –to transform between representation schemes

Superimposed Information - INRIA - April Lois Delcambre The Metamodel Provides a level of abstraction above models Describes the structural features of models Topic Map Topic Map Defintions Topic Map Instances XML DTD XML Document Basic Set of Abstractions Model Constructs and Relationships Schema-Level Data Instance-Level Data Metamodel

Superimposed Information - INRIA - April Lois Delcambre XML Model, Schema, and Instance Elements, Element Types, Attributes, Attribute Types Elements contain Attributes Elements can be nested PDX YVR $ XML Model XML DTD (Schema) XML Document (Instances) Model constructs and relationships defined using the metamodel

Superimposed Information - INRIA - April Lois Delcambre Topic Map Example Painting Painter by painter Influenced by “Captive” “Paul Klee” by painter influenced by “Francisco de Goya” “1914” by painter mentioned biography mentioned biography critiqued mentioned

Superimposed Information - INRIA - April Lois Delcambre Topic Map Model in UML TopicType ttypename : String TopicRelType relType : String AnchorType anchorRole : String TopicInstance title : String topicInsID : Number TopicRelInst AnchorInst > Address markID : String * * * * * * > topic_instOf > rel_instOf > anchor_instOf address topicIns topicType 11 ** topic Type1 topic Type2 11 ** topic Ins1 topic Ins2

Superimposed Information - INRIA - April Lois Delcambre Generic, Uniform Representation We use RDF and RDF Schema to represent model, schema, and instance uniformly creator (creator, ‘ person1) (name, ‘person1’, ‘John Smith’) Class Property creator type Person WebPage type domain range (type, ‘creator’, Property) (domain, ‘creator’, WebPage) (range, ‘creator’, Person) (type, ‘Person’, Class) (type, ‘WebPage’, Class) person1 ‘John Smith’ name RDF Triples RDF Graph RDF Schema Triples RDF Schema Graph

Superimposed Information - INRIA - April Lois Delcambre The Metamodel Definition Construct Structural Connector MarkLexicalConformanceGeneralization connects 2 constructs Basic Metamodel Elements Special Elements  Construct : A basic structural unit Mark : A connection-point to the base-layer Lexical : A primitive-value type  Connector : A relationship between 2 constructs Conformance : A schema-instance relationship Generalization : An inheritance relationship

Superimposed Information - INRIA - April Lois Delcambre Representing Models (instanceOf, “TopicType”, Construct) (instanceOf, “TopicInstance”, Construct) (instanceOf, “topic_instOf”, Conformance) (domain, “topic_instOf”, TopicInstance) (range, “topic_instOf”, TopicType) (domainMult, “topic_instOf”, “*”) (rangeMult, “topic_instOf”, “1”) (instanceOf, “ttypename”, Connector) (domain, “ttypename”, TopicType) (range, “ttypename”, String) (domainMult, “ttypename”, “*”) (rangeMult, “ttypename”, “1”) TopicType ttypename : String TopicInstance * 1 > topic_instOf

Superimposed Information - INRIA - April Lois Delcambre Representing Schema (instanceOf, “painting_tt”, TopicType) (ttypename, “painting_tt”, “painting”) (instanceOf, “painter_tt”, TopicType) (ttypename, “painter_tt”, “painter”) (instanceOf, “byPainter_rt”, TopicRelType) (relType, “byPainter_rt”, “by painter”) (topicType1, “byPainter_rt”, painting_tt) (topicType2, “byPainter_rt”, painter_tt) (instanceOf, “biography_at”, AnchorType) (anchorRole, “biography_at”, “biography”) (topicType, “biography_at”, painter_tt) Topic Types (schema): painting, painter Topic Rel Types (schema): by painter Anchor Types (schema): biography painting painter by painter biography

Superimposed Information - INRIA - April Lois Delcambre Representing Instances (instanceOf, “painter1”, TopicInstance) (title, “painter1”, “Paul Klee”) (topicInsID, “painter1”, “5”) (topic_instOf, “painter1”, painter_tt) (instanceOf, “painting1”, TopicInstance) (title, “painting1”, “Captive”) (topicInsID, “painting1”, “19”) (topic_instOf, “painting1”, painting_tt) (instanceOf, “byPainter1”, TopicRelInst) (rel_instOf, “byPainter1”, byPainter_rt) (topicIns1, “byPainter1”, painting1) (topicIns2, “byPainter1”, painter1) (instanceOf, “biography1”, AnchorInst) (anchor_instOf, “biography1”, biography_at) (address, “biography1”, a1) (instanceOf, “a1”, Address) (markID, “a1”, Topic (instances): Paul Klee, Captive Topic Relationship (instance): a by painter relationship Anchor (instance): a biography anchor Address (instance): mark to URL

Superimposed Information - INRIA - April Lois Delcambre Basic Types of Mappings Mapped Converted Inter-Model Inter-Schema Model-to-Schema Model 2 Schema 1 Instances 1 Model 1 Schema 1 Instances 1 Model 1 Schema 1 Instances 1 Model 1 Schema 1 Instances 1 Model 1 Schema 2 Instances 1 Model 2 Schema 2 Instances 2 Mapped

Superimposed Information - INRIA - April Lois Delcambre S(‘source’,  (‘instanceOf’, X, ‘TopicInstance’))  S(‘target’,  (‘instanceOf’, X, ‘XMLElem’)) XMLElem TopicInstance Mapped Mapping Rules Simple production rules over triples

Superimposed Information - INRIA - April Lois Delcambre Mapping Rules (cont.) XMLElem TopicInstance XMLElemType TopicType Mapped elem_instOf topic_instOf S(‘source’,  (‘topic_instOf’, X, Y)) S(‘target’,  (‘instanceOf’, X, ‘XMLElem’)) S(‘target’,  (‘instanceOf’, Y, ‘XMLElemType’))  S(‘target’,  (‘elem_instOf’, X, Y))

Superimposed Information - INRIA - April Lois Delcambre

Superimposed Information - INRIA - April Lois Delcambre Applications SLIM Pad –Scratchpad application with Bundle-Scrap model (uses superimposed information) XML Extractor –“Extracts” XML information and transforms it into a Topic Map for searching/browsing Translation among message formats: IDMEF - Intrusion Detection, CISL, SNMP v3,... XML Files Generic Rep. (XML model) Generic Rep. (TM model) DBMS Topic Map Browser XML Extractor XML Extractor outmapped stored in

Superimposed Information - INRIA - April Lois Delcambre What’s Next for this Work? Extending the metamodel - to accommodate broader range of models Formalizing the metamodel Develop abstract addressing modes (for canonical marks - with translation to various instantiations)

Superimposed Information - INRIA - April Lois Delcambre Harvesting Information to Sustain our Forests: Creating an Adaptive Management Portal NSF DIGITAL GOVERNMENT PROGRAM Tim Tolle & Lois Delcambre Co-Project Directors

Superimposed Information - INRIA - April Lois Delcambre Project focuses on the: Adaptive Management Areas USDA Forest Service USDI Bureau of Land Management USDI Fish and Wildlife Service

Superimposed Information - INRIA - April Lois Delcambre Adaptive Management Portal: a value-added, Internet-based service Provide multiple access paths to forest information. Preserve local autonomy and local focus of each site. Support diverse users and types of information. Use proposed, existing, and de facto standards for content, classification, and technology. Be low-cost, scalable, extensible.

Superimposed Information - INRIA - April Lois Delcambre Project Funding Duration: 3 years Budget: $1.5 million Principal financial sponsors –National Science Foundation –Bureau of Land Management (Oregon State Office) –Forest Service (R-6 and PNW Station) –National Park Service (Western Region)

Superimposed Information - INRIA - April Lois Delcambre Team Members Tim Tolle Regional Coordinator for AMA, US Forest Service Eric Landis Forest Information System Specialist, Consultant Craig Palmer Natural Resources Monitoring Expert, UNLV Fred Phillips Professor, Head, Mgt. of Science and Tech., OGI Patty Toccalino Asst. Prof., Environmental Science and Eng., OGI Lois Delcambre Professor, Computer Science and Eng., OGI David Maier Professor, Computer Science and Eng., OGI Shawn Bowers PhD Student, Computer Science and Eng., OGI Mat Weaver PhD Student, Computer Science and Eng., OGI Forest/environmental expertise Computer science expertise

Superimposed Information - INRIA - April Lois Delcambre Staff Scientist, Pacific Northwest National Laboratory Mark Whiting Science Advisor, USDI, National Park Service Regina Rochefort Communications Director, USDA Forest Service, PNW Research Station Cynthia L. Miner Chief, Office of Technical Support, Forest Resources, USDI Fish and Wildlife Service Monty Knudsen Executive Director, IMFN Secretariat Fred Johnson MD, Asst. Professor, Division of Medical Informatics and Outcomes Research, OHSU Paul Gorman Sustainable Northwest Martin Goebel USDA Forest Service, Pacific NW Region Robert Devlin President, IUFRO, Oxford Forestry Institute, Dept of Plant Sciences Jeff Burley Co-Inventor of the Topic Map Model Michel Biezunski Advisory Board Forest/environmental expertise Computer science expertise

Superimposed Information - INRIA - April Lois Delcambre Task 1 – Status Snoqualmie Pass Adaptive Management Area, Cle Elum, WA (June and July) Interviews with Forest Service Corvallis Forest Sciences Lab and USGS FRESC, Corvallis ( August) Interviews with Central Cascades Adaptive Management Area, Eugene (August) Interviews with the Applegate Partnership and its associated agencies (August) Rainier National Park (planned for October)

Superimposed Information - INRIA - April Lois Delcambre Things we’ve learned from Task 1 NSF Digital Government work is project-based multiple agencies are involved each agency serves as information gatherer; information broker; information consumer primary product is information: assessments, studies, surveys, environmental impact statements even though information is a primary product, information technology is secondary (stewardship of the land is the primary mission)

Superimposed Information - INRIA - April Lois Delcambre Building the Sandbox … to support similarity search across multiple domains

Superimposed Information - INRIA - April Lois Delcambre Documents “Document” used very loosely – could be just about anything Vary in format (text, map, dataset, …) Vary in purpose (formal assessment, informal letter, …)

Superimposed Information - INRIA - April Lois Delcambre Traditional Metadata Author: Parley Pratt Purpose: Environmental Assessment Location: Wenatchee National Forest Keyword: Interstate 90 Keyword: Snoqualmie Pass Fields with values May or may not use “controlled vocabulary” Allows for basic searches

Superimposed Information - INRIA - April Lois Delcambre Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Instead of fields and values, we use explicit properties and terms Metadata is represented by connecting a document with a term via a property

Superimposed Information - INRIA - April Lois Delcambre Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Editor NEPA Cle Elum Ranger District

Superimposed Information - INRIA - April Lois Delcambre Super Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Editor NEPA Cle Elum Ranger District Explicit hierarchy of properties and terms allows for enhanced searching.

Superimposed Information - INRIA - April Lois Delcambre Super-Duper Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Editor NEPA Cle Elum Ranger District Western Hemlock Additional relationships of various types between terms allow for super-duper enhanced searching (i.e. similarity search, …)

Superimposed Information - INRIA - April Lois Delcambre For More Information Lois Delcambre Maier Bowers Weaver

Superimposed Information - INRIA - April Lois Delcambre

Superimposed Information - INRIA - April Lois Delcambre Research Issues Models for the superimposed layer How does the superimposed model influence the capabilities it supports? How does the form of superimposed information affect the effort to construct and maintain it? –Are some forms more robust to updates in the base layer –What forms map onto current information management tools

Superimposed Information - INRIA - April Lois Delcambre Research Issues (2) Challenges when superimposed and base layer have different models –E.g., structured over unstructured, or vice versa Bi-level tools –Browsing between layers –Queries over both layers How do we delimit the universe of discourse in the base layer? Is it easier to fuse superimposed information than base information?

Superimposed Information - INRIA - April Lois Delcambre Research Issues (3) Variations on the conceptual architecture –Commingled layers –“Super-superimposed information” How do capabilities of base layer affect structure and operations over superimposed information? –Addressing modes –Address comparison –Querying Addressing for non-web sources –Relational, object-oriented DBs

Superimposed Information - INRIA - April Lois Delcambre Research Issues (4) How to extend DBMSs to better deal with information they don’t store. How to help population superimposed information spaces. What are good formats for representation and exchange of superimposed information?

Superimposed Information - INRIA - April Lois Delcambre Why Databases Don’t (Currently) Solve It Seems closely related to view and data integration However –Superimposed information can’t always be derived from the base data –DB approaches assume schema and common model –DBs like to work with data they control –Traditional approaches are heavy weight semantic analysis schema integration query mapping On a source-by-source basis