1. Electronic Notebooks: An Interface Component for Semantic Records Systems James D. Myers, Michael Peterson, K Prasad Saripalli, Tara Talbott Mathematics and Computational Science Directorate Pacific Northwest National Laboratory

2. 2 OutlineOutline Why have an electronic notebook? The changing science/IT landscape Semantic repositories Scientific Annotation Middleware ENs on semantic repositories The ELN on SAM

3. 3 Secure shared WWW based space Hierarchical Chapters/Pages/Notes Add/View/Search Notes File upload, sketch, text, equations, forms, image capture, … Interactive views of data Editor/Viewer APIs Cross-out capability Digital Signatures/Timestamps Java Client, Perl and Java (2001+) servers … PNNL Electronic Laboratory Notebook (ELN) ~1995+

4. 4 What distinguishes ENs from other tools? Emphasis on multimedia human-entered information Chronological, page-oriented display Master/personal project record Records functionality: Non-repudiation - digital signatures and timestamps Persistence/completeness - write-once/no deletions/audit trail Standardized lifecycle – signing/witnessing policies, archiving, retention schedules, …

5. 5 The Systems Science Revolution Community Resources Bi-directional flow/feedback of information Partial results being combined to produce new knowledge Experiment/Theory/Model comparisons Multiscale optimizations Rapid Evolution High Complexity Shifting/Emerging disciplinary boundaries Resources will be distributed With multiple curators

6. 6 Advances in Problem Solving Environments/Grids/Semantic Technologies Multiple Applications recording data Pedigree/Provenance Experiment Metadata Project Organization Workflow Categorization Detected Features Instrument logs … Replica Locations Endorsements Community Annotations … How do we provide EN capabilities in this larger context?

7. 7 Semantic Repositories Use self-describing metadata/relationships Triple-stores RDF OWL Aggregate information generated by multiple applications Allows browsing, searching, reasoning across integrated information

8. 8 Scientific Annotation Middleware (SAM) - 5 yr DOE funded research project Develop middleware to create semantic repositories Enable the sharing of this information among portals and problem solving environments, software agents, scientific applications, and electronic notebooks With different levels of sophistication Without global schema Improve the completeness, accuracy, and availability of the scientific record. http://www.scidac.org/SAM/

9. 9 SAM Architecture Notebook Services Semantic Services Metadata Services DataGrid Database Web DAV, DASL, JMS, SAM Extensions DAV, JDBC, GridFTP

10. 10 Web Distributed Authoring and Versioning (WebDAV) An early web service Put/Get data with arbitrary properties (dynamic) Properties can be discovered and accessed independently DASL, Versioning, Transactions, … Widely supported (MS Office, databases, file system drivers,…)

11. 11 Binary Format Description (BFD) Language XML Language to describe ASCII, Binary, and XML data formats Generic Parser to extract and semantically tag data in files/streams The meaning of data can be captured, regardless of format, for future use Data Format Description Language Standard XSL Stylesheet (reformat) XSLT Processor XML Format 2 BFD Parser BFD Description 1 XML Format 1 File Format 1 meters <XBFDvalue-of select="/XSIL/Param [@Name='numColumns']"/> 4 <Stream Type=“remote” XBFDStreamnumber=“0” Encoding = “biinary”/>

12. 12 SAM Metadata Services Layer Jakarta Slide DAV server plus configurable: Mapping to Data Store(s) Property Generation from binary/ASCII/XML files Dynamic Virtual Translations Server generated Properties and Relationships Timestamp, size, CopyOf Fortran Application ‘Local Disk’ DAV DAV+ Content ELNProp1 Prop2 Prop1 hastranslation BFD Web Service XSLT … Translated Content RDF Export

13. 13 SAM Semantic Services Layer SAM Metadata Layer plus configurable: Relation-scoped Queries Translation of DAV Properties to RDF Triples RDF/GXL Pedigree Generation …

14. 14 Back to ENs… What is needed to be able to provide Unstructured human entry of information? Chronological, page-oriented display? A master/personal project record? Records functionality?

15. 15 Creating Notes A ‘standard’ ELN client can create notes Stored as content with a hasNote relationship with pages, notes Plus…any app can store notes the same way Page generation – works as before

16. 16 ENs as a Primary View? Instruments, PSEs, etc. may organize parts of the experiment that an EN should not duplicate  define other relationships as part of the EN chapter/page/note hierarchy: Project Experiment1 Experiment2 Data1 Data2  Notebook1 Chapter1 Chapter2 Page1 Page2 Defined by PSEInterpreted by EN

17. 17 Records?Records? Digital Signatures, Timestamps, etc. are services that can be exposed as repository services and associated metadata But What do we sign (content/metadata)? Where is the edge of the record? How deep do we travel through the web of relations? How do we stop other applications from changing/deleting signed content?

18. 18 Multiple Options Simple: Sign content plus defined subset of metadata Stop at edge of server Treat relationship cycles as links Lock content and metadata subset when signed Advanced: Multiple self-describing signatures (e.g. XMLSignature) Allow records across servers via trust, cached metadata/data Define fine-grained retention schedules

19. 19 SAM Notebook Services Layer SAM Metadata and Semantic Layers plus: Notebook Management, Page Display, … Digital Signatures Canonicalization Notarized Timestamps Data/Signature Migration Capabilities Notebook API, Notebook Components Supports ELN 5.1, Annotation Applet, new portal-based EN client EN Portlets

20. 20 Collaboratory for Multiscale Chemical Science (CMCS) SAM as primary data system, pedigree, notebook NEESgrid/CHEF Portal/NMI Grid User Computing Environments ELN, SAM as a metadata/pedigree store? Genomes-To-Life SAM as annotation/metadata repository, notebook Internal PNNL Projects Concept Map Repository, Interface to Lustre, Biological Data Annotation DOE2000 Notebook Community (1500+ email addresses) Upgrades to DOE2K Notebooks E.g. Columbia University Environmental Science Lab Notebooks CollaborationsCollaborations

21. 21 A Scientific Content Repository Vision Notebooks become just one view of the scientific information Applications contribute data, metadata, and relationships directly Records functionality provided by middleware, available to multiple applications Content is stored in multiple repositories managed independently The scientific record becomes richer and re-integrated

22. 22 AcknowledgmentsAcknowledgments Carina Lansing, PNNL Al Geist, Jens Schwidder, David Jung, ORNL U.S. Department of Energy Pacific Northwest National Laboratory Pacific Northwest National Laboratory is a multiprogram national laboratory operated by Battelle Memorial Institute for the U.S. Department of Energy under Contract DE-AC06-76RL0 1830 Oak Ridge National Laboratory Oak Ridge National Laboratory is a multiprogram national laboratory operated by UT-Battelle, LLC for the U.S. Department of Energy under Contract DE-AC05-00OR22725 Mathematical, Information and Computational Sciences Division of the Office of Science