Long-Term Preservation of At-Risk Digital Geospatial Data: The North Carolina Geospatial Data Archiving Project Steve Morris NCSU Libraries

Risks to Digital Geospatial Data .shp .mif .gml .e00 .dwg .dgn .bsb .bil .sid Note: Percentages based on the actual number of respondents to each question

Risks to Digital Geospatial Data Producer focus on current data Also, archiving data does not guarantee “permanent access” Future support of data formats in question Need to migrate formats or allow for emulation Data failure “Bit rot”, media failure Preservation metadata requirements Descriptive, administrative, technical, DRM Shift to “streaming data” for access Note: Percentages based on the actual number of respondents to each question

Note: Percentages based on the actual number of respondents to each question

Note: Percentages based on the actual number of respondents to each question

Note: Percentages based on the actual number of respondents to each question

Note: Percentages based on the actual number of respondents to each question

Today’s geospatial data as tomorrow’s cultural heritage Note: Percentages based on the actual number of respondents to each question

Time series – vector data Parcel Boundary Changes 2001-2004, North Raleigh, NC Note: Percentages based on the actual number of respondents to each question

Time series – Ortho imagery Vicinity of Raleigh-Durham International Airport 1993-2002 Note: Percentages based on the actual number of respondents to each question

NC Geospatial Data Archiving Project Partnership between university library (NCSU) and state GIS agency (NCCGIA) Focus on state and local geospatial content in North Carolina (state demonstration) Tied to NC OneMap initiative, which provides for seamless access to data, metadata, and inventory information Objective: engage existing state/federal geospatial data infrastructures in preservation Note: Percentages based on the actual number of respondents to each question

Local Government GIS in NC Data resources are highly distributed and subject to frequent update More detailed, current, accurate than federal/state data resources North Carolina local agency GIS environment 100 counties, 95 with GIS 85 counties with high resolution orthophotography Growing number of municipal systems Hundreds of millions of dollars investment Note: Percentages based on the actual number of respondents to each question

NCGDAP Targeted Content Resource Types GIS “vector” (point/line/polygon) data Digital orthophotography Digital maps Tabular data (e.g. assessment data) Content Producers Mostly state, local, regional agencies Some university, not-for-profit, commercial Selected local federal projects Note: Percentages based on the actual number of respondents to each question

Work plan in a Nutshell Work from existing data inventories NC OneMap Data Sharing Agreements as the “blanket”, individual agreements as the “quilt” Partnership: work with existing geospatial data infrastructures (state and federal) Technical approach: blend emerging digital library technologies with geospatial technologies Metadata: METS, FGDC, PREMIS?, GeoDRM? Repository: Dspace and others Note: Percentages based on the actual number of respondents to each question

Big Challenges Format migration paths Management of data versions over time Preservation metadata Harnessing geospatial web services Preserving cartographic representation Keeping content repository-agnostic Preserving spatial databases More … Note: Percentages based on the actual number of respondents to each question

Vector Data Format Issues Vector data much more complicated than image data ‘Archiving’ vs. ‘Permanent access’ An ‘open’ pile of XML might make an archive, but if using it requires a team of programmers to do digital archaeology then it does not provide permanent access Piles of XML need to be widely understood piles GML: need widely accepted application schemas (like OSMM?) The spatial database conundrum Export feature classes, and lose topology, annotation, relationships, etc. … or use the spatial database as the primary archival platform (some are now thinking this way) Note: Percentages based on the actual number of respondents to each question

Managing Time-versioned Content Many local agency data layers continuously updated E.g., some county cadastral data updated daily—older versions not generally available Individual versioned datasets will wander off from the archive How do users “get current metadata/DRM/object” from a versioned dataset found “in the wild”? How do we certify concurrency and agreement between the metadata and the data? Note: Percentages based on the actual number of respondents to each question

Preservation Metadata Issues FGDC Metadata Many flavors, incoming metadata needs processing Cross-walk elements to PREMIS, MODS? Metadata wrapper METS (Metadata Encoding and Transmission Standard) vs. other industry solutions Need a geospatial industry solution for the ‘METS-like problem’ GeoDRM a likely trigger—wrapper to enforce licensing (MPEG 21 references in OGIS Web Services 3) Note: Percentages based on the actual number of respondents to each question

Harnessing Geospatial Web Services Note: Percentages based on the actual number of respondents to each question

Harnessing Geospatial Web Services Automated content identification ‘capabilities files,’ registries, catalog services WMS (Web Map Service) for batch extraction of image atlases? last ditch capture option preserve cartographic representation retain records of decision-making process … feature services (WFS) later. Rights issues in the web services space are ambiguous … GeoDRM in development Note: Percentages based on the actual number of respondents to each question

Preserving Cartographic Representation Note: Percentages based on the actual number of respondents to each question

Preserving Cartographic Representation The true counterpart of the old map is not the GIS dataset, but rather the cartographic representation that builds on that data: Intellectual choices about symbolization, layer combinations Data models, analysis, annotations Cartographic representation typically encoded in proprietary files (.avl, .lyr, .apr, .mxd) that do not lend themselves well to migration Symbologies have meaning to particular communities at particular points in time, preserving information about symbol sets and their meaning is a different problem Note: Percentages based on the actual number of respondents to each question

Repository Architecture Issues Interest in how geospatial content interacts with widely available digital repository software Focus on salient, domain-specific issues Challenge: remain repository agnostic Avoid “imprinting” on repository software environment Preservation package should not be the same as the ingest object of the first environment Tension between exploiting repository software features vs. becoming software dependent Note: Percentages based on the actual number of respondents to each question

Preserving Spatial Databases Spatial databases in general vs. ESRI Geodatabase “format” Not just data layers and attributes—also topology, annotation, relationships, behaviors ESRI Geodatabase archival issues XML Export, Geodatabase History, File Geodatabase, Geodatabase Replication Growing use of geodatabases by municipal, county agencies Some looking to Geodatabase as archival platform (in addition to feature class export) Note: Percentages based on the actual number of respondents to each question

NCGDAP Philosophy of Engagement Provide feedback to producer organizations/ inform state geospatial infrastructure Take the data as in the manner In which it can be obtained “Wrangle” and archive data Note the ‘Project’ in ‘North Carolina Geospatial Data Archiving Project’– the process, the learning experience, and the engagement with geospatial data infrastructures are more important than the archive Note: Percentages based on the actual number of respondents to each question

Questions? Contact: Steve Morris Head of Digital Library Initiatives NCSU Libraries Steven_Morris@ncsu.edu Phone: (919) 515-1361 NCGDAP website: http://www.lib.ncsu.edu/ncgdap/ Note: Percentages based on the actual number of respondents to each question