1. NEESGrid Data and MetaData Technology Kincho Law, Jun Peng, Jim Eng, Terry Weymouth, Paul Hubbard, Charles Severance

2. Outline n Model activities n Metadata tools and repository n Data as video n What is done n What we have to do

3. The Slide Metadata Data Data Viewers Data Mappers Data Ingestors There is a layer is where we develop tools which take advantage and begin to depend on of the “meaning” of the data – where we begin to depend on the meaning of a second. Where we make a viewer capable of viewing a certain type of object. This is where we build things which make use of knowledge. This layer will never be complete but it is a large focus of the coming months. Concepts Search 09/2003

4. How to prioritize model exploration and development n Focus on the following areas: –Areas where we have or are building tools –Areas where we already have incoming data in some format –Build the model through experiment based deployment - solve real problems in an open way and see if (with some adaptation) the solutions apply more broadly (i.e. Minnesota ) 11/2003

5. Go Forward - Tools n Evaluate the ORST interface and use it to implement experiment-based interface to meta data repository n Investigate tools to represent structural data (like SAC data) n Extend and improve viewers – publish API so that sites can extend the viewers n Improve notebook –Single signon using CHEF/Grid credentials –Integration with Metadata –Smother integration with CHEF n Explore automated synchronized video and data capture and after-experiment replay of synchronized video and data (ORST UMinn) n Explore the capture of high quality still images as data (UMinn) n Investigate adopting a data-editing tool (XMLSpy) 09/2003

6. RDF Integration n Some of the data and meta data task force members are using Protégé-2000 to develop their models and expressing them in RDF. n RDF and NEESML are very similar but not identical so it may be challenging to ingest any arbitrary RDF n We expect that we will be able to map a subset of RDF to NEESML for ingestion or adapt an RDF parser (Jena or Raptor) to ingest that subset directly into the repository 11/2003

7. Data Model Activities n Groups have been formed to develop coordinated models –Shake Table –Centrifuge –Tsunami –GeoTech n Gokhan Pekhan is organizing this effort n Kincho / Jun / Jim pushed forward with a RDF-Based Shake Table Model from Protege-2000 and developed software

8. Models + Data Model Repo Models Configure Data Load Configure RDF/ OWL RDF

9. Protégé - 2K Models + Data Model Repo Models Configure Data Load Configure RDF/ OWL RDF

10. Electronic Notebook n Collaborative effort with the DOE SciDAC –Electronic notebook - metadata entry –Data mapping –Data provenance –Data display –Slide data/metadata jakarta.apache.org/slide/ n Ultimate integration will be via JSR-170 n www.scidac.org/SAM/ n collaboratory.emsl.pnl.gov/docs/collab/sam/samtechoverview.html

11. DOE ELN / NEESgrid Integration (to date) SAM / Slide Repository Chef Grid Security NEESgrid Repository MyProxy Technology Celebration

12. DOE ELN / NEESgrid Integration (ultimate) Chef Grid Security NEESgrid Repository MyProxy

13. DAQ 0 3 4 0 6 8 0 3 4 0 6 8 09/2003 “Skunkworks” project

14. Data Turbine n Commercial, free data streaming toolkit

15. Data Turbine (cont) n Existing data viewers will be adapted to access and display data from data turbine n Data acquisition software will be adapted to place information in Data Turbine Channels n Metadata elements will be developed to represent data turbine live, stored, and derived channels n New efforts (video as data) will be developed from the ground up using Data Turbine n outlet.creare.com/rbnb/ 11/2003

16. Video as Data n Follow on to initial demonstration at ORST n Experiment based development: Minnesota n Design phase complete n Joint effort, NEESGrid SI, ORST, Minnesota, UC Davis, Texas, Buffalo, and others

17. DT Main System CTL NTCP Plugin Data Turbine - Today NTCP Control DAQ Data Capture DT Client BT848 rbnbjcap DT Client NEES NSDS Driver Control Plugin AXIS / DT Gateway Axis

18. DT Main System Data Turbine NTCP Control Plugin Thumbs Control Make Smoothie Technology Celebration

19. DT Main System PTZ/ USB Still Capture DT Client DT Capturing Camera Control DAQ Data Capture DT Client Audio Encoder DT Client BT848 rbnbjcap DT Client Still Capture - Minnesota / Paul Hubbard Video capture - From Creare Audio capture - From Creare (TBD) Data Capture - From sites (upwards compatible) NEES NSDS Driver - Paul Hubbard Camera Control Plugin - Mich / Minn NEES NSDS Driver Each still capture produces two channels - Small 1-5fps stream + large single images when picture is taken Control Plugin

20. DT Main System User Views / Still Camera Quicktime Storage System Quicktime Slicing Tool Still Image / Camera Control ~ <> ^ ^ <> Control Plugin Data Viewer Thumbnail Process Thumbnail - uBuffalo / Umichigan Thumbnail viewer - Creare / Mich Camera Control Applet - Minn / Mich Quicktime Slicing tool - Mich (low) Stored Data Viewer - Mich JPEG Viewer - Creare Quickime Viewer - Apple Thumbnail + Audio + Data <>+

21. Minnesota Mock - up If you area developer and interested in following / helping / participating in this activity, join the mailing list neesgrid-dv@neesgrid.org

22. Tool List - To Do n Next release of repository n Integrate ELN into repository n DAQ Control Panel in CHEF –Set/Retrieve Metadata –Start / Stop –Ingest data from staging space n Data Turbine Control Panel in CHEF –Start / Stop / Configure Sources Video | Audio | Data | Thumbnail –Control permanent storage of video

23. To Do (cont) n NTCP Debugging and Monitoring in CHEF –Needed Data Turbine n Data as Video Client Tools in CHEF –New Monitor Tool –Still Image –Camera Control n Data Turbine Audio Capture n We may need to support XML Schema n QuickTime Capabilities –Archive, retrieve, slice, dice, convert, present –Probably will not be completed as part of SI effort

24. Data Model Work n Data Curation Summit –Understand issues form go-forward plan –Meeting 3/18/2004 n Data and Metadata Task Force –Finish the tsunami and centrifuge models n DSAC Committee –Meeting 3/19/2004

25. Summary n In September 2003, we met and “re-visioned” data n A bunch of requirements gathering and development has been done –The “high risk” elements are working now n There is more to do - We will run out of time –Evolutionary development approach - there will always be usable working code - we will stop when we run out of time

26. The people… n Gokhan Peckan - Data Models n Kincho Law - Data Models / Software design n Jun Peng - Data Models / Software design n Jim Eng - Parse / Ingest / RDF / Project Browser n Jim Myers - Electronic Notebook n Terry Weymouth - Data Turbine n Paul Hubbard - DAQ and NTCP n Joe Futrelle - Data / Metadata Repository