TRLN High Performance Data Storage System 21 Sep 2006 Jim Porto Ken Galluppi

2 Not Many Answers We do not have a large disaster database—no one really does—that’s the problem. Data exists in many formats and is distributed across 100 counties and numerous state agencies. I think we are all open to using metadata tags, at a minimum, on the data that we do compile.

3 Not Many Answers We are open to “standard” metadata formats. Size—we estimate that the size of all the databases that we want to compile will be on the order of 100 TB. Depends on what we add, but we could be adding up to 10% per year. A sizeable portion of the data could be permanent and will require long term storage. We hope to develop longitudinal data sets for time analysis. Don’t know how much data will be generated per month or per year.

4 Not Many Answers Don’t know yet either the granularity or the frequency of access. Software is not complex at this time, but development of very large database algorithms to tie spatial features together, to provide 3d spatial analysis, and to provide time series analysis for visual image displays and comparisons could become fairly complex.

5 The DataBase NCOneMap Data Element Types (Example) Economy Health Environment Socio-Demographic Bio-Habitat Geophysical Weather Land Cover Utility-Telecom Imagery-Basemap Elevation Location-Geodetic Structures Transportation Cadastral Boundaries Hydrography

6 Problems With NCOne Map DB Display only—can not manipulate data once displayed. Distributed across 100 counties and multiple state agencies. Multiple formats for databases. Access and security concerns. Not all data is captured.

7 A Disaster GEO-Informatics Database County Databases WMS NC Map 1000 TB TRLN Database Research Modeling Operations Planning State Databases

8 Characteristics of Database Could have a significant archive component. Could have a significant transient component requiring periodic flushing (ex. weather radar,) Builds relationships between permanent and transient data. One model is to link to distributed data bases (local, state, and federal), rapidly build transient databases both for mapping and for data manipulation, AND to partner with Triangle Universities to archive static data in data depositories.

9 Computer Science Issues How to maintain assembled data for transient use once maps have been rendered. Management of large data sets assembled on the fly. Time sampling and flushing of unused pieces. Data ownership. Data security.

10 Additional Computer Science Issues Data integrity compared to archived data. Bandwidth to move data around the network. Visualization of large datasets. How to build applications on top of large data sets (eg spatial analysis in standard GIS packages is not feasible)

11 Uses of Database Development of new modeling and display software that can handle 3d, time series, and very large databases. Substantive research into disaster vulnerabilities through large scale modeling. Source of regional and state operational information. Source of regional and state planning information.

12 Benefits of Database Aid to Disaster Decision Makers (General Assembly is increasingly interested in the State’s Disaster Response). Aid for Scenario construction and testing to improve preparedness. Cross discipline approach bringing in many professions. (Environmental quality, meteorology, public health, economics, planning and land use, agriculture, forestry, for example.) Regional economic analysis more feasible.

13 Benefits of Database Gain experience in working with ultra large databases. Develop new software tools to integrate, to visualize, and to analyze very large, diverse databases. Develop solutions for security and working agreements for data ownership.

14 Final Thoughts We want to advance the state of knowledge in the use if large datasets. We want to give the Triangle Universities a distinctive project that will garner national recognition. AND we want to provide relevant research that aids the citizens of NC (especially since this project if being funded by tax dollars.)