Metadata Taking the Next Step Michael Moeller Metadata Specialist NOAA Coastal Services Center

Outline A quick review Making the case–the value of metadata Establishing a metadata program Tools for metadata creation and validation Questions and discussion

A Quick Review

Simply put, metadata is information about your data. What is Metadata?

Metadata is that component of data which describes it. Environmental Sensitivity Index Data A Component of Data RARNUM - unique combination of species, concentration, and seasonality CONC (concentration) - density species is found at location Season_ID - seasonality code like to the seasonal table Element - biology group Metadata

Metadata describes… CONTENT CONDITION QUALITY Characteristics of the data Characteristics of the data A Component of Data

Because metadata provides vital information about a data set, it should never be viewed or treated as a separate entity. Metadata Non-spatial or attributes Spatial Take Home Message Metadata is a critical and integral component of any complete data set. Metadata is a critical and integral component of any complete data set.

Assumptions For our discussion, we assume all metadata is well written and fully developed. We also assume that metadata is considered at the beginning of a project, and that it is written throughout the life of the project (not just at the end).

Making the Case: The Value of Metadata

The Old Model Primary focus is on external value Discovery Assessment Access Use

The Value of Metadata Primary internal value “Inheritance” “Properly documenting a data set is the key to preserving its usefulness through time.” The Old Model

The Value of Metadata The New Model “An aid to data management” Main focus shifts to internal value Discovery Assessment Access Use

Additional data management benefits An Emerging Concept Data Currency Date of last edit/update Age of source files Data Utility Track source file usage Track distribution frequency

Monitoring Data Development Data processing steps Status of development Estimating Development Costs Data processing–time and extent Source file availability Additional data management benefits An Emerging Concept

To realize the full potential of metadata under this new concept, metadata creation must become integral to the data development process. The question is “How?” Making the case for metadata

Preserves data investment Limits liability Helps manage data resources Aids in external data acquisition Facilitates data access and transfer Provides for efficient data distribution Making the case for metadata Approach metadata development from a business perspective Build administrative support

Stress the individual benefits of metadata Build technical support Reduces workload over the long term Field fewer data inquiries Provides a means of documenting personal contributions Facilitates sharing of reliable information Making the case for metadata

Develop strong staff support Incorporate metadata expectations into job descriptions and performance standards Build technical support Provide staff development opportunities; the three “T’s”  Training  Tools  Time Making the case for metadata

Develop templates to facilitate efficient and consistent metadata creation Build organizational support Identify pertinent fields within the metadata structure Populate fixed fields  Use standardized language  Define distribution methods  Cite standards used Build source and contact libraries Making the case for metadata

Map metadata fields to the workflow Distribute the effort  Technicians–lineage  Analysts–process and methodology  Field Scientists–accuracy assessments  I.T. Managers–tools, automated collection methods, information management Establish and assign responsibilities Making the case for metadata

Mandate the use of standards and templates Develop boilerplate metadata-deliverable language for data contractors Require publication of metadata Create and publish standard operating procedures to document metadata policies and procedures Establish standard policies Making the case for metadata

Establishing a Metadata Program

Daunting as it may seem at first, developing a metadata program is really not that difficult. As an added bonus, it provides an opportunity for you to step back from your existing data management efforts and look at how they can be improved.

Developing a metadata business plan As you begin to think about your metadata program, consider the following: What data needs to be documented and at what level? How will it be managed? Who will do the work? How much will it cost? How long will it take? How will it be used? Can the workload be distributed? Decisions, decisions, decisions

A laser beam, or an umbrella? One big issue that pops up when putting together a metadata program is the level at which you document your data. Options may range from documenting every data layer to documenting at the project level, or some combination therein. What data needs to be documented?

What are your organization’s goals for the metadata program? Will you be using custom tools or off-the-shelf tools to write your metadata? Will you be storing your metadata as flat text files, or will they be incorporated into a database? Will you put all of your metadata on a clearinghouse node, or only certain versions? Who will be responsible for managing the metadata? How will it be managed? Define your data management plan early in the process.

One or many, in-house or outsourced? Ideally, each data creator would document their own data. However, in the real world, that’s not always possible. Will you need to hire extra people to help document your data collections, or will you increase the workload of existing personnel? Will it require sending the work to an outside contractor? Who will do the work?

It always comes down to money! As with many things in the operation of your organization, money is king. Metadata has upfront costs associated with it, but compared to not documenting the data, the costs are minimal over the long run. Perhaps the biggest cost will be labor, as personnel need training and time to write the metadata. How much will it cost?

Time is on your side. Or is it? If you do an informal survey of employees, time is the one thing they need more of. Making metadata creation part of the workflow sounds good in theory, but can be difficult to implement. Although there is a learning curve, metadata creation becomes much less time-consuming the more it’s done. In some instances, it may even be possible to automate some or all of the metadata creation. How long will it take?

If you write it, they will use it. Considering your target audience will help you decide what information is required in your metadata, and how best to present that information. The question is, who will be using your metadata? How will it be used?

Can the work load be distributed? To share, or not to share. To help reduce costs, an organization or agency may be able to partner with other internal groups, or with outside groups, to help share the metadata creation tasks. Suggestion: Make sure you have agreed- upon protocols in place to ensure consistency in the documentation.

Metadata Creation and Validation Tool Time

Tools for metadata creation NOAA Coastal Services Center MetaScribe Allows you to create a template record that can be used to create large numbers of similar records. NOAA CSC ArcView Metadata Collector Extension for ArcView 3.x that allows user to capture metadata specific to a particular ArcView data set. TKME Straightforward text editor for metadata. ESRI ArcGIS metadata tool Found within the new ArcGIS application, this tool allows you to generate metadata for any ArcGIS data set.

Metadata Creation Tools TKME An editor for formal metadata, TKME is intended to simplify the process of creating metadata that conform to the standard.

NOAA CSC ArcView ® Metadata Collector The ArcView® Metadata Collection Tool was developed by the National Oceanic and Atmospheric Administration’s (NOAA) Coastal Services Center in ArcView using the Avenue® scripting language. This tool collects and compiles Federal Geographic Data Committee (FGDC)-compliant metadata for ARC/INFO coverage's, shapefiles, grids, and supported image formats. Metadata Creation Tools

MetaScribe This new tool was also developed by the National Oceanic and Atmospheric Administration’s (NOAA) Coastal Services Center to aid in the creation of multiple sets of metadata that exhibit a high degree of redundancy. Metadata Creation Tools

ArcGIS metadata collector Found in ArcCatalog, this tool allows the user to write metadata within the Arc environment. Metadata Creation Tools

CNS (“Chew ‘n Spit”) A pre-parser for formal metadata designed to help metadata managers convert records that cannot be parsed by mp into records that can be parsed by mp. Metadata Validation Tools MP (Metadata Parser) A compiler to parse formal metadata that checks the syntax against the FGDC Content Standard for Digital Geospatial Metadata and generates output suitable for viewing with a Web browser or text editor. MPBatch This tool, developed by Intergraph, takes CNS and MP and wraps them in a graphical user interface (GUI). It makes using CNS and MP much easier, and allows for batching process multiple metadata records.

Tool Demos

Questions and Discussion

Metadata Don’t Duck Michael Moeller