1. EUROCRIS2013, Porto, 2013-11-14/15 Publishing structural health monitoring data Fábio Costa, Gabriel David, Álvaro Cunha INESC TEC, Faculty of Engineering University of Porto {fabiopcosta, gtd, acunha}@fe.up.pt

2. EUROCRIS2013, Porto, 2013-11-14/15 Context n Research group in Civil Engineering Structural Health Monitoring Continuous data streams for several years n External requirement Publish research data n Internal requirement Manage data files Organize, reuse, visualize n Preserve the context Both organizational and technical

3. EUROCRIS2013, Porto, 2013-11-14/15 Outline n Project goals n Metadata model for projects relying on monitoring data n Prototype digital archive n Conclusions

4. EUROCRIS2013, Porto, 2013-11-14/15 Publishing data n Setting up scientific experiments and collecting data from them Investment and effort justify publishing the data n Cross check the results n Enable further research n Policy of several S&T funding agencies data collected within funded projects must be publicly shared n Monitoring natural phenomena data sets become non-repeatable documents of actual facts

5. EUROCRIS2013, Porto, 2013-11-14/15 Trust n Expertise and care design, install and operate the experiment to clean the raw data n Resulting data sets + metadata publishable authored documents n Further reuse, especially in the long term Depend on trust in the data sets Metadata to assert meaning and authenticity

6. EUROCRIS2013, Porto, 2013-11-14/15 Main project goals n Specify a metadata model for monitoring data n Design and build a digital archive Primary raw data and processed results of on-going projects n Improve data reliability integrated backup strategy n Create a Web interface browse and search metadata; visualize and download data n Simple user management and access control n Automate data files ingestion

7. EUROCRIS2013, Porto, 2013-11-14/15 Data life cycle COMPONENTS 1 2 3 3 5 4 structure 1 – data acquisition 2 – metadata ingestion 3 – query and visualization 4 – DB conversion to SIARD format 5 – data backup backup repository database application vibest.fe.up.pt/shm

8. EUROCRIS2013, Porto, 2013-11-14/15 Global metadata model

9. EUROCRIS2013, Porto, 2013-11-14/15 Context level

10. EUROCRIS2013, Porto, 2013-11-14/15 Project and structure info

11. EUROCRIS2013, Porto, 2013-11-14/15 System level

12. EUROCRIS2013, Porto, 2013-11-14/15 Monitoring system

13. EUROCRIS2013, Porto, 2013-11-14/15 Data level

14. EUROCRIS2013, Porto, 2013-11-14/15 File structure

15. EUROCRIS2013, Porto, 2013-11-14/15 Digital archive n Repository Files coming every 30 min (67000 since 2009-04-01 on the example dataset) Hierarchy of folders by year, month and day n Conventional descriptive name for each file PROJSUByyyyMMdd_hhmmss[mili]_typ.ext n Technology Postgres database management system Vaadin framework for Java Web applications Apache http server running on Ubuntu operating system A few libraries for specific operations

16. EUROCRIS2013, Porto, 2013-11-14/15 Implemented services n Simple user management system Access control at project level and specific file level n Automatic ingestion of new data files n Web interface with compact design n Data files Search by interval; search by event OAI-PMH for metadata (combined with authorization) Download of set of files -Zipping facility for grouping Visualization of set of files (graph and data) -20000 points maximum displayed; interpolation if larger

17. EUROCRIS2013, Porto, 2013-11-14/15 Graph of accelerations

18. EUROCRIS2013, Porto, 2013-11-14/15 Conclusions n Filling in metadata is a bit demanding for researchers Less than in the Core Scientific Metadata Model Regarding data as an authored scientific outcome assigns responsibility and improves data quality n Access control mechanism is a must n Generality of the model for monitoring data Relatively few concepts specific to Civil Engineering n Open problem How to visualize processed results? (may be MATLAB matrices)