1. Superimposed Information - INRIA - April 20011 Lois Delcambre (lmd@cse.ogi.edu) Technology for Superimposed Information Lois Delcambre with Shawn Bowers, David Maier, Mat Weaver Database and Object Technology Lab Computer Science and Engineering Department Oregon Graduate Institute

2. Superimposed Information - INRIA - April 20012 Lois Delcambre (lmd@cse.ogi.edu) Outline introduction to superimposed information a superimposed application: SLIMPad (DLI2 Project) model-based representation and transformation of information harvesting information to sustain our forests (NSF Digital Government project)

3. Superimposed Information - INRIA - April 20013 Lois Delcambre (lmd@cse.ogi.edu) What is Superimposed Information? data “placed over” existing information sources to:  highlight  annotate  elaborate  select  collect  organize  connect  reuse information elements often to support new applications, beyond the original

4. Superimposed Information - INRIA - April 20014 Lois Delcambre (lmd@cse.ogi.edu) Examples of Superimposed Information  Non-electronic examples:  Commentaries on religious texts, law, literature  Concordances, citation indexes  Electronic examples:  Your bookmark file in your web browser  RDF metadata

5. Superimposed Information - INRIA - April 20015 Lois Delcambre (lmd@cse.ogi.edu) Why work on it now? Broadening range of digital information Accessibility/addressability to base information –Reference (e.g., URL) can be resolved quickly –Addressing at various levels of granularity Emerging Standards: RDF, Topic Maps, Xlink Emerging Applications: –Third Voice, Multi-Valent Documents,...

6. Superimposed Information - INRIA - April 20016 Lois Delcambre (lmd@cse.ogi.edu) The superimposed and base layers with marks Superimposed Layer Base Layer Information Source 1 Information Source 2 Information Source n … marks Focus: building generic technology

7. Superimposed Information - INRIA - April 20017 Lois Delcambre (lmd@cse.ogi.edu) Superimposed Applications enhanced base layer application simultaneous use of base and superimposed applications web browser (with extra functions such as annotation) superimposed application base application 1 base application 2...

8. Superimposed Information - INRIA - April 20018 Lois Delcambre (lmd@cse.ogi.edu) Outline introduction to superimposed information a superimposed application: SLIMPad (DLI2 Project) model-based representation and transformation of information harvesting information to sustain our forests (NSF Digital Government project)

9. Superimposed Information - INRIA - April 20019 Lois Delcambre (lmd@cse.ogi.edu) Paul Gorman, MD Lois Delcambre, PhD David Maier, PhD

10. Superimposed Information - INRIA - April 200110 Lois Delcambre (lmd@cse.ogi.edu) Bundles in the wild……….. Observational team: Paul Gorman Joan Ash Mary Lavelle Jason Lyman …………..Bundles in captivity Computer science team: Lois Delcambre Dave Maier Shawn Bowers Longxing Deng Mathew Weaver

11. Superimposed Information - INRIA - April 200111 Lois Delcambre (lmd@cse.ogi.edu) Let’s take a trip to the ICU

12. Superimposed Information - INRIA - April 200112 Lois Delcambre (lmd@cse.ogi.edu) (Wild) Bundles

13. Superimposed Information - INRIA - April 200113 Lois Delcambre (lmd@cse.ogi.edu) (Wild) Bundles

14. Superimposed Information - INRIA - April 200114 Lois Delcambre (lmd@cse.ogi.edu) (Wild) Bundles

15. Superimposed Information - INRIA - April 200115 Lois Delcambre (lmd@cse.ogi.edu) (Wild) Bundles manage information for diverse, complex tasks contain selected, collected, structured, annotated are often used in settings with: –high uncertainty –low predictability –potentially grave outcomes –time & attention are highly constrained

16. Superimposed Information - INRIA - April 200116 Lois Delcambre (lmd@cse.ogi.edu) (Wild) Bundles There is benefit in creating (active processing of information) There is benefit in reusing (trigger memory) There is benefit in sharing (establish collective, situated awareness)

17. Superimposed Information - INRIA - April 200117 Lois Delcambre (lmd@cse.ogi.edu) Given…. bundles are everywhere! access to bundles provides access to important information information in bundles is often copied from other information sources we can keep copied/referenced information linked through the use of marks

18. Superimposed Information - INRIA - April 200118 Lois Delcambre (lmd@cse.ogi.edu) (Captive) Bundles SLIMPad - a scratchpad application to create bundles but….with marks linked electronically built using our architectural for building superimposed applications inspired by the observational work (but not focused on a specific medical task) a very simple tool … with minimal interface to base layer

19. Superimposed Information - INRIA - April 200119 Lois Delcambre (lmd@cse.ogi.edu) SLIMPad demo

20. Superimposed Information - INRIA - April 200120 Lois Delcambre (lmd@cse.ogi.edu) Superimposed Layer Information Manager (SLIM): Contributions of the Architecture Mark Management - to create/resolve marks SLIM DMI (data manipulation interface) - for the application developer TRIM store - for generic storage of superimposed information

21. Superimposed Information - INRIA - April 200121 Lois Delcambre (lmd@cse.ogi.edu)

22. Superimposed Information - INRIA - April 200122 Lois Delcambre (lmd@cse.ogi.edu)

23. Superimposed Information - INRIA - April 200123 Lois Delcambre (lmd@cse.ogi.edu) SLIM API: as seen by application

24. Superimposed Information - INRIA - April 200124 Lois Delcambre (lmd@cse.ogi.edu) What’s Next for this Project? Validation - cardiologists, ICU nurses, … Extend the informational model of SLIMPad Extend SLIMPad to suit a selected medical task Validation of observational work and SLIMPad technology in another domain (forest information - preparation of a watershed assessment)

25. Superimposed Information - INRIA - April 200125 Lois Delcambre (lmd@cse.ogi.edu) Outline introduction to superimposed information a superimposed application: SLIMPad (DLI2 Project) model-based representation and transformation of information harvesting information to sustain our forests (NSF Digital Government project)

26. Superimposed Information - INRIA - April 200126 Lois Delcambre (lmd@cse.ogi.edu) Model Schema Data Instance Data with Marks Information Source 1 Information Source 2 Superimposed Layer Base Layer marks Model-Based Superimposed Information But the model and schema are optional

27. Superimposed Information - INRIA - April 200127 Lois Delcambre (lmd@cse.ogi.edu) Our Goals Represent superimposed information generically, for various data models (with the schema optional) Build a single management system for superimposed information Mix information from various models Convert information from one representation scheme to another

28. Superimposed Information - INRIA - April 200128 Lois Delcambre (lmd@cse.ogi.edu) Transforming Information Generic Rep. (XML model) Generic Rep. (XML model) convert Generic Rep. (Topic Map model) XML DB XML Viewer SQL TM Browser Painting Painter by painter Influenced by mentionedbiographymentionedcritiqued convert Generic Rep. (Relational model)

29. Superimposed Information - INRIA - April 200129 Lois Delcambre (lmd@cse.ogi.edu) Our Approach Metamodel –to represent multiple data models Generic, Uniform Representation Scheme –to store model, schema, and instances for model-based information Mapping Formalism –to transform between representation schemes

30. Superimposed Information - INRIA - April 200130 Lois Delcambre (lmd@cse.ogi.edu) The Metamodel Provides a level of abstraction above models Describes the structural features of models Topic Map Topic Map Defintions Topic Map Instances XML DTD XML Document Basic Set of Abstractions Model Constructs and Relationships Schema-Level Data Instance-Level Data Metamodel

31. Superimposed Information - INRIA - April 200131 Lois Delcambre (lmd@cse.ogi.edu) XML Model, Schema, and Instance Elements, Element Types, Attributes, Attribute Types Elements contain Attributes Elements can be nested PDX YVR $213.84... XML Model XML DTD (Schema) XML Document (Instances) Model constructs and relationships defined using the metamodel

32. Superimposed Information - INRIA - April 200132 Lois Delcambre (lmd@cse.ogi.edu) Topic Map Example Painting Painter by painter Influenced by “Captive” “Paul Klee” by painter influenced by “Francisco de Goya” “1914” by painter mentioned biography mentioned http://... biography http://... critiqued mentioned http://...

33. Superimposed Information - INRIA - April 200133 Lois Delcambre (lmd@cse.ogi.edu) Topic Map Model in UML TopicType ttypename : String TopicRelType relType : String AnchorType anchorRole : String TopicInstance title : String topicInsID : Number TopicRelInst AnchorInst > Address markID : String * * * * * *1 1 111 1 > topic_instOf > rel_instOf > anchor_instOf address topicIns topicType 11 ** topic Type1 topic Type2 11 ** topic Ins1 topic Ins2

34. Superimposed Information - INRIA - April 200134 Lois Delcambre (lmd@cse.ogi.edu) Generic, Uniform Representation We use RDF and RDF Schema to represent model, schema, and instance uniformly http://…/~john creator (creator, ‘http://…/~john’, person1) (name, ‘person1’, ‘John Smith’) Class Property creator type Person WebPage type domain range (type, ‘creator’, Property) (domain, ‘creator’, WebPage) (range, ‘creator’, Person) (type, ‘Person’, Class) (type, ‘WebPage’, Class) person1 ‘John Smith’ name RDF Triples RDF Graph RDF Schema Triples RDF Schema Graph

35. Superimposed Information - INRIA - April 200135 Lois Delcambre (lmd@cse.ogi.edu) The Metamodel Definition Construct Structural Connector MarkLexicalConformanceGeneralization connects 2 constructs Basic Metamodel Elements Special Elements  Construct : A basic structural unit Mark : A connection-point to the base-layer Lexical : A primitive-value type  Connector : A relationship between 2 constructs Conformance : A schema-instance relationship Generalization : An inheritance relationship

36. Superimposed Information - INRIA - April 200136 Lois Delcambre (lmd@cse.ogi.edu) Representing Models (instanceOf, “TopicType”, Construct) (instanceOf, “TopicInstance”, Construct) (instanceOf, “topic_instOf”, Conformance) (domain, “topic_instOf”, TopicInstance) (range, “topic_instOf”, TopicType) (domainMult, “topic_instOf”, “*”) (rangeMult, “topic_instOf”, “1”) (instanceOf, “ttypename”, Connector) (domain, “ttypename”, TopicType) (range, “ttypename”, String) (domainMult, “ttypename”, “*”) (rangeMult, “ttypename”, “1”) TopicType ttypename : String TopicInstance * 1 > topic_instOf

37. Superimposed Information - INRIA - April 200137 Lois Delcambre (lmd@cse.ogi.edu) Representing Schema (instanceOf, “painting_tt”, TopicType) (ttypename, “painting_tt”, “painting”) (instanceOf, “painter_tt”, TopicType) (ttypename, “painter_tt”, “painter”) (instanceOf, “byPainter_rt”, TopicRelType) (relType, “byPainter_rt”, “by painter”) (topicType1, “byPainter_rt”, painting_tt) (topicType2, “byPainter_rt”, painter_tt) (instanceOf, “biography_at”, AnchorType) (anchorRole, “biography_at”, “biography”) (topicType, “biography_at”, painter_tt) Topic Types (schema): painting, painter Topic Rel Types (schema): by painter Anchor Types (schema): biography painting painter by painter biography

38. Superimposed Information - INRIA - April 200138 Lois Delcambre (lmd@cse.ogi.edu) Representing Instances (instanceOf, “painter1”, TopicInstance) (title, “painter1”, “Paul Klee”) (topicInsID, “painter1”, “5”) (topic_instOf, “painter1”, painter_tt) (instanceOf, “painting1”, TopicInstance) (title, “painting1”, “Captive”) (topicInsID, “painting1”, “19”) (topic_instOf, “painting1”, painting_tt) (instanceOf, “byPainter1”, TopicRelInst) (rel_instOf, “byPainter1”, byPainter_rt) (topicIns1, “byPainter1”, painting1) (topicIns2, “byPainter1”, painter1) (instanceOf, “biography1”, AnchorInst) (anchor_instOf, “biography1”, biography_at) (address, “biography1”, a1) (instanceOf, “a1”, Address) (markID, “a1”, “URLMarkManager@954308545”) Topic (instances): Paul Klee, Captive Topic Relationship (instance): a by painter relationship Anchor (instance): a biography anchor Address (instance): mark to URL

39. Superimposed Information - INRIA - April 200139 Lois Delcambre (lmd@cse.ogi.edu) Basic Types of Mappings Mapped Converted Inter-Model Inter-Schema Model-to-Schema Model 2 Schema 1 Instances 1 Model 1 Schema 1 Instances 1 Model 1 Schema 1 Instances 1 Model 1 Schema 1 Instances 1 Model 1 Schema 2 Instances 1 Model 2 Schema 2 Instances 2 Mapped

40. Superimposed Information - INRIA - April 200140 Lois Delcambre (lmd@cse.ogi.edu) S(‘source’,  (‘instanceOf’, X, ‘TopicInstance’))  S(‘target’,  (‘instanceOf’, X, ‘XMLElem’)) XMLElem TopicInstance Mapped Mapping Rules Simple production rules over triples

41. Superimposed Information - INRIA - April 200141 Lois Delcambre (lmd@cse.ogi.edu) Mapping Rules (cont.) XMLElem TopicInstance XMLElemType TopicType Mapped elem_instOf topic_instOf S(‘source’,  (‘topic_instOf’, X, Y)) S(‘target’,  (‘instanceOf’, X, ‘XMLElem’)) S(‘target’,  (‘instanceOf’, Y, ‘XMLElemType’))  S(‘target’,  (‘elem_instOf’, X, Y))

42. Superimposed Information - INRIA - April 200142 Lois Delcambre (lmd@cse.ogi.edu)

43. Superimposed Information - INRIA - April 200143 Lois Delcambre (lmd@cse.ogi.edu) Applications SLIM Pad –Scratchpad application with Bundle-Scrap model (uses superimposed information) XML Extractor –“Extracts” XML information and transforms it into a Topic Map for searching/browsing Translation among message formats: IDMEF - Intrusion Detection, CISL, SNMP v3,... XML Files Generic Rep. (XML model) Generic Rep. (TM model) DBMS Topic Map Browser XML Extractor XML Extractor outmapped stored in

44. Superimposed Information - INRIA - April 200144 Lois Delcambre (lmd@cse.ogi.edu) What’s Next for this Work? Extending the metamodel - to accommodate broader range of models Formalizing the metamodel Develop abstract addressing modes (for canonical marks - with translation to various instantiations)

45. Superimposed Information - INRIA - April 200145 Lois Delcambre (lmd@cse.ogi.edu) Harvesting Information to Sustain our Forests: Creating an Adaptive Management Portal NSF DIGITAL GOVERNMENT PROGRAM Tim Tolle & Lois Delcambre ttolle@fs.fed.us lmd@cse.ogi.edu Co-Project Directors

46. Superimposed Information - INRIA - April 200146 Lois Delcambre (lmd@cse.ogi.edu) Project focuses on the: Adaptive Management Areas USDA Forest Service USDI Bureau of Land Management USDI Fish and Wildlife Service

47. Superimposed Information - INRIA - April 200147 Lois Delcambre (lmd@cse.ogi.edu) Adaptive Management Portal: a value-added, Internet-based service Provide multiple access paths to forest information. Preserve local autonomy and local focus of each site. Support diverse users and types of information. Use proposed, existing, and de facto standards for content, classification, and technology. Be low-cost, scalable, extensible.

48. Superimposed Information - INRIA - April 200148 Lois Delcambre (lmd@cse.ogi.edu) Project Funding Duration: 3 years Budget: $1.5 million Principal financial sponsors –National Science Foundation –Bureau of Land Management (Oregon State Office) –Forest Service (R-6 and PNW Station) –National Park Service (Western Region)

49. Superimposed Information - INRIA - April 200149 Lois Delcambre (lmd@cse.ogi.edu) Team Members Tim Tolle Regional Coordinator for AMA, US Forest Service Eric Landis Forest Information System Specialist, Consultant Craig Palmer Natural Resources Monitoring Expert, UNLV Fred Phillips Professor, Head, Mgt. of Science and Tech., OGI Patty Toccalino Asst. Prof., Environmental Science and Eng., OGI Lois Delcambre Professor, Computer Science and Eng., OGI David Maier Professor, Computer Science and Eng., OGI Shawn Bowers PhD Student, Computer Science and Eng., OGI Mat Weaver PhD Student, Computer Science and Eng., OGI Forest/environmental expertise Computer science expertise

50. Superimposed Information - INRIA - April 200150 Lois Delcambre (lmd@cse.ogi.edu) Staff Scientist, Pacific Northwest National Laboratory Mark Whiting Science Advisor, USDI, National Park Service Regina Rochefort Communications Director, USDA Forest Service, PNW Research Station Cynthia L. Miner Chief, Office of Technical Support, Forest Resources, USDI Fish and Wildlife Service Monty Knudsen Executive Director, IMFN Secretariat Fred Johnson MD, Asst. Professor, Division of Medical Informatics and Outcomes Research, OHSU Paul Gorman Sustainable Northwest Martin Goebel USDA Forest Service, Pacific NW Region Robert Devlin President, IUFRO, Oxford Forestry Institute, Dept of Plant Sciences Jeff Burley Co-Inventor of the Topic Map Model Michel Biezunski Advisory Board Forest/environmental expertise Computer science expertise

51. Superimposed Information - INRIA - April 200151 Lois Delcambre (lmd@cse.ogi.edu) Task 1 – Status Workshops @ Snoqualmie Pass Adaptive Management Area, Cle Elum, WA (June and July) Interviews with Forest Service Corvallis Forest Sciences Lab and USGS FRESC, Corvallis ( August) Interviews with Central Cascades Adaptive Management Area, Eugene (August) Interviews with the Applegate Partnership and its associated agencies (August) Rainier National Park (planned for October)

52. Superimposed Information - INRIA - April 200152 Lois Delcambre (lmd@cse.ogi.edu) Things we’ve learned from Task 1 NSF Digital Government work is project-based multiple agencies are involved each agency serves as information gatherer; information broker; information consumer primary product is information: assessments, studies, surveys, environmental impact statements even though information is a primary product, information technology is secondary (stewardship of the land is the primary mission)

53. Superimposed Information - INRIA - April 200153 Lois Delcambre (lmd@cse.ogi.edu) Building the Sandbox … to support similarity search across multiple domains

54. Superimposed Information - INRIA - April 200154 Lois Delcambre (lmd@cse.ogi.edu) Documents “Document” used very loosely – could be just about anything Vary in format (text, map, dataset, …) Vary in purpose (formal assessment, informal letter, …)

55. Superimposed Information - INRIA - April 200155 Lois Delcambre (lmd@cse.ogi.edu) Traditional Metadata Author: Parley Pratt Purpose: Environmental Assessment Location: Wenatchee National Forest Keyword: Interstate 90 Keyword: Snoqualmie Pass Fields with values May or may not use “controlled vocabulary” Allows for basic searches

56. Superimposed Information - INRIA - April 200156 Lois Delcambre (lmd@cse.ogi.edu) Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Instead of fields and values, we use explicit properties and terms Metadata is represented by connecting a document with a term via a property

57. Superimposed Information - INRIA - April 200157 Lois Delcambre (lmd@cse.ogi.edu) Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Editor NEPA Cle Elum Ranger District

58. Superimposed Information - INRIA - April 200158 Lois Delcambre (lmd@cse.ogi.edu) Super Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Editor NEPA Cle Elum Ranger District Explicit hierarchy of properties and terms allows for enhanced searching.

59. Superimposed Information - INRIA - April 200159 Lois Delcambre (lmd@cse.ogi.edu) Super-Duper Enhanced Metadata Parley Pratt Author Environmental Assessment Purpose Wenatchee National Forest Location Interstate 90 Snoqualmie Pass Keyword Editor NEPA Cle Elum Ranger District Western Hemlock Additional relationships of various types between terms allow for super-duper enhanced searching (i.e. similarity search, …)

60. Superimposed Information - INRIA - April 200160 Lois Delcambre (lmd@cse.ogi.edu) For More Information www.cse.ogi.edu/footprints www.cse.ogi.edu/forest lmd@cse.ogi.edu Lois Delcambre maier@cse.ogi.eduDave Maier shawn@cse.ogi.eduShawn Bowers mweaver@cse.ogi.eduMat Weaver

61. Superimposed Information - INRIA - April 200161 Lois Delcambre (lmd@cse.ogi.edu)

62. Superimposed Information - INRIA - April 200162 Lois Delcambre (lmd@cse.ogi.edu) Research Issues Models for the superimposed layer How does the superimposed model influence the capabilities it supports? How does the form of superimposed information affect the effort to construct and maintain it? –Are some forms more robust to updates in the base layer –What forms map onto current information management tools

63. Superimposed Information - INRIA - April 200163 Lois Delcambre (lmd@cse.ogi.edu) Research Issues (2) Challenges when superimposed and base layer have different models –E.g., structured over unstructured, or vice versa Bi-level tools –Browsing between layers –Queries over both layers How do we delimit the universe of discourse in the base layer? Is it easier to fuse superimposed information than base information?

64. Superimposed Information - INRIA - April 200164 Lois Delcambre (lmd@cse.ogi.edu) Research Issues (3) Variations on the conceptual architecture –Commingled layers –“Super-superimposed information” How do capabilities of base layer affect structure and operations over superimposed information? –Addressing modes –Address comparison –Querying Addressing for non-web sources –Relational, object-oriented DBs

65. Superimposed Information - INRIA - April 200165 Lois Delcambre (lmd@cse.ogi.edu) Research Issues (4) How to extend DBMSs to better deal with information they don’t store. How to help population superimposed information spaces. What are good formats for representation and exchange of superimposed information?

66. Superimposed Information - INRIA - April 200166 Lois Delcambre (lmd@cse.ogi.edu) Why Databases Don’t (Currently) Solve It Seems closely related to view and data integration However –Superimposed information can’t always be derived from the base data –DB approaches assume schema and common model –DBs like to work with data they control –Traditional approaches are heavy weight semantic analysis schema integration query mapping On a source-by-source basis