1. Understanding the Value and Importance of Proper Data Documentation Introduction to Metadata Table of Contents Presentation Overview 1 – 2 – 3 – 4 – 5 – X X-X Section Name Page

2. Understanding the Value and Importance of Proper Data Documentation At the conclusion of this module the participant will be able to Describe the concept of metadata in general terms Describe metadata as it relates to geospatial data Give at least 2 examples of common, non-geospatial metadata Objectives What Is Metadata?

3. Understanding the Value and Importance of Proper Data Documentation What is Metadata? First things first Before we begin, let’s make sure we are on the same page with some definitions. Metadata – When referring to metadata in this presentation, we make the assumption that the metadata is well-written and fully developed according to the rules of the Federal Geographic Data Committee’s (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM). Geospatial – This term will be used to refer to any data set tied to a location on the Earth’s surface. 1-2

4. Understanding the Value and Importance of Proper Data Documentation 1-3 Emily and Madison What is Metadata? So, what is metadata, anyway? Simply put, metadata is information about data. A plain and simple example of what we are talking about can be seen in a photograph. Do you have a box of pictures at home with nothing written down about them? Why should you document them? You took the picture; you know what the picture shows. But imagine that your grandkids inherit that box of pictures. Do you think they will know anything about the pictures? Look at the example to the right. We are given a couple of names, written on the back of the photo, but what other information could we provide that would enhance the photo? ______________________________________________ Plain and simple

5. Understanding the Value and Importance of Proper Data Documentation Finding geospatial data is just like finding books. More Examples Perhaps the best common example of metadata is the library card catalog. Imagine going to the library and trying to find a book without this system. The card catalog entry for a book contains information about that book, such as title, author, publish date, number of pages, etc. It also includes keywords that can be used by the search system to help you locate it. 1-5 What is Metadata? Rodale's illustrated encyclopedia of herbs ISBN:087596964x (pbk.) : $17.95 ISBN:0878576991 : $24.95 Title:Rodale's illustrated encyclopedia of herbs / Claire Kowalchik & William H. Hylton, editors ; writers, Anna Carr... [et al.]. Publication info: Emmaus, Pa. : Rodale Press, c1987. Physical descrip: vi, 545 p. : ill. (some col.) ; 24 cm. General Note: Includes index. Subject term: Herbs. Subject term: Herbs--Utilization. Subject term: Herb gardening. Subject term: Herbs--History. Subject term: Herbals. Added author: Kowalchik, Claire. Added author: Hylton, William H. Added author: Carr, Anna, 1955- Added author: Rodale Press. Rodale's illustrated encyclopedia of herbs ISBN:087596964x (pbk.) : $17.95 ISBN:0878576991 : $24.95 Title:Rodale's illustrated encyclopedia of herbs / Claire Kowalchik & William H. Hylton, editors ; writers, Anna Carr... [et al.]. Publication info: Emmaus, Pa. : Rodale Press, c1987. Physical descrip: vi, 545 p. : ill. (some col.) ; 24 cm. General Note: Includes index. Subject term: Herbs. Subject term: Herbs--Utilization. Subject term: Herb gardening. Subject term: Herbs--History. Subject term: Herbals. Added author: Kowalchik, Claire. Added author: Hylton, William H. Added author: Carr, Anna, 1955- Added author: Rodale Press. Geospatial metadata is very similar to the card catalog information, but it is much more robust. It includes information such as quality, accuracy, and scale.

6. Understanding the Value and Importance of Proper Data Documentation At the conclusion of this module the participant will be able to Describe the relationship between data and metadata List 3 types of data that can benefit from metadata List the 2 most common ways used to search data Describe the concept of granularity and how it applies to metadata Objectives Metadata–A Component of Data

7. Understanding the Value and Importance of Proper Data Documentation Rethinking the relationship between data and metadata. Metadata as a Component of Data Properly documenting your data provides vital information to interested parties. These interested parties may be internal or external to your organization, but either way, well-written metadata is giving them the opportunity to find out details about the data that will help them decide whether they want the data, how to access the data, and how to use the data. Look at the example to the right. If you are a scientist that works with environmental sensitivity index (ESI) data on a regular basis, then you might understand what the columns in the data set represent. But if you have acquired this data set to include in a project and have never seen the data before, then you will need information that will help you understand the data, and more importantly, incorporate that data into your project. Metadata serves that function. Environmental Sensitivity Index Data RARNUM = unique combination of species, concentration, and seasonality CONC (concentration) = Density species is found at location SEASON_ID = seasonality code like to the seasonal table ELEMENT - Biology group Metadata A Component of Data Metadata is that component of data which describes it. It documents characteristics of the data such as content, condition, and quality. This latter characteristic should not be overlooked. New federal regulations require federal agencies to document the quality of their data under the Data Quality Act (Public Law 106-554 Section 515). Metadata can help agencies meet the requirements of this new regulation by documenting their data quality within their metadata.

8. Understanding the Value and Importance of Proper Data Documentation Metadata – It’s not just for GIS! Title Scale Source Content Location Publication Access Title Scale Source Content Location Publication Access GIS files Imagery Geodatabases GPS data Biological data In situ data GIS files Imagery Geodatabases GPS data Biological data In situ data Metadata Data set Metadata is most commonly associated with digital spatial data; however, it can be used to describe any data, such as remotely-sensed imagery, GPS data, biological data, or in situ data. Think for a moment about a data set that does not have a tie to a location on the earth’s surface. Can you think of one? When considering metadata for your data, whatever they may be, why start from scratch? Using the current standard can save time and money, and is flexible enough to allow for changes down the road, as the situation arises. Use metadata to describe a wide variety of data types. SpeciesSp_IDSite_IDCount Desmognathus apalachicolaeA25-955BRP14 Gyrinophilus porphyriticus danielsiD33-009BRP4a7 Plethodon chattahoochee HightonT93-193Sugar223 Plethodon cinereus Z73-244Elk8b5 A Component of Data

9. Understanding the Value and Importance of Proper Data Documentation Drawing the Box When you write metadata, one important aspect to capture in the metadata is the geographic region. This is done by either describing a bounding coordinate box that encompasses the location where the data was collected, or by using place keywords within the metadata. Geographic locations and keywords are two of the most common ways metadata is searched. The standard allows for the addition of theme, place, temporal, and stratum keywords. Choose keywords wisely, and consider different variations for the same word. Metadata describes the geographic boundaries of your data. Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box Acme Data Company Coordinate Box A Component of Data

10. Understanding the Value and Importance of Proper Data Documentation Defining the Data Set As you consider metadata for a project, a common question that often arises is “At what level of granularity should the metadata be written?” In other words, can one project level record suffice, or do each individual data elements need to be documented? The answer is that it depends. It may be possible to capture all the appropriate information in one project level record, but more often than not, additional records may have to be written to describe various components of the data set. Metadata can be used to document at various levels of granularity. Project Data Set Data Layer Data Element One course of action is to begin a project level record at the beginning of the project, and use that to build templates for other metadata within the project as it progresses. Remember, there is no right or wrong answer to the question of granularity. It is organization and project specific, but what ever level of metadata is written, make sure it is fully- developed and well-written. A Component of Data

11. Understanding the Value and Importance of Proper Data Documentation Metadata Non-spatial or attributes Spatial Without accompanying metadata, the fitness for use of any given data set is unknown, and its usefulness may be significantly impaired. Metadata is written and maintained as an integrated part of the data creation process as opposed to a process that might occur at end of project, if time allows. Metadata is a critical component of a complete data set. As such, it should never be viewed as a separate entity. It is and should remain an integral component of the data. No data set should be considered complete without a fully-developed, well- written metadata record or records. Metadata is an integral component of a data set. A Component of Data

12. Understanding the Value and Importance of Proper Data Documentation At the conclusion of this module the participant will be able to Describe the current idea of internal and external value of metadata List 3 types of data that can benefit from metadata List at least 3 key functions of metadata Objectives The Value of Metadata

13. Understanding the Value and Importance of Proper Data Documentation The Value of Metadata The Traditional View of Metadata The Primary External Value One of the main traditional roles of metadata is it’s use in data discovery. Much like the card catalog system at the library, metadata can be searched to find data of interest. In addition, the metadata contains other information that can be helpful for a seeker to determine whether or not the data is fit for their particular use, and if so, how the data can be accessed and used. The Current Model The Primary Internal Value Within an organization, the traditional role of metadata was one of ‘inheritance.’ This simply refers to metadata’s ability to help preserve a data set’s usefulness through time. Imagine an individual who has worked with a particular type of data for years, and then suddenly leaves that organization. If that individual did not adequately document the data, then it’s possible that some, if not all, of that data’s usefulness will be lost.

14. Understanding the Value and Importance of Proper Data Documentation The New View of Metadata The New Internal Value As an organization’s data holdings increase, data management becomes critical. Metadata can assist in building an efficient data management program. Now what was once an external value of metadata becomes an internal value. Organizations can use metadata to document their data holdings, and then use this metadata internally for data discovery, assessment, access, and use. This has several benefits, including a reduction in labor and/or data duplication. These types of benefits translate to more economic efficiency. An Emerging Concept – An Aid to Data Management To realize the full potential of metadata under this new concept, metadata creation must become integral to the data development process. The question is “How?” The Value of Metadata The ability to maintain a record of a data set’s currency will help ensure that project source data is not out of date. Metadata can aid in tracking editing and update frequencies, as well as the usage for a data set’s source files, along with where and how often a data set is being distributed. Data managers are always looking for help in managing costs and resources. Metadata can be used as an aid to monitoring the data development process. In addition, metadata can be a resource that data managers use to make more accurate cost predictions for future projects based on past experiences

15. Understanding the Value and Importance of Proper Data Documentation Building support within your organization To help sell metadata in your organization, pitch these key functions of metadata: Shift the focus on the primary value from external to internal. Data Archive Data are the most expensive components of a GIS. Metadata is a means of preserving the value of data investments. This is of particular significance to local and regional governments experiencing rapid staff changes. Data Assessment GIS data development has shifted from data producers to data consumers. From a consumer perspective, metadata is the truth in labeling required to assess available data products. From the producer’s perspective, metadata is a means of declaring data limitations and serves as a form of liability insurance. Data Management Metadata enables organizations to retrieve in-house data resources by specific criteria for global edits and annual updates Data Discovery Metadata is the primary means of locating available geospatial data resources via the Internet. Metadata is a primary public information resource as it is a non-technical means of presenting technical information. Data Transfer Metadata is increasingly used by software systems as a means of properly ingesting data and by analysts as a means of properly displaying data. Data Distribution By building metadata in compliance with national standards, you can participate in the Global Spatial Data Clearinghouse. Participation promotes your agency and frees staff from answering data inquiries. The Value of Metadata

16. Understanding the Value and Importance of Proper Data Documentation Metadata preserves data investment. The cost of data acquisition and processing is a large portion of a project’s budget. It would be a shame to lose that investment because the data were not properly documented. Metadata limits liability. One of the most often overlooked features of a well-written metadata record is it’s use as a legal document. The various constraints of a data set should be clearly defined within a metadata record. This does not mean it will keep an organization out of court, but it does add one more layer of documentation to support its case. Metadata helps manage data resources. We’ve touched on a few ways metadata can help with data management, such as data currency, data utility, monitoring of the development process, and estimating development costs. Metadata aids in external data acquisition. Metadata can be searched from existing external and/or internal clearinghouse systems to find data of interest. Metadata facilitates data access and transfer. Once a data set is found, metadata should contain information on access and transferring that data. Metadata provides for efficient data distribution. Writing metadata in accordance with current federal standards allows organizations to participate in the National and Global Spatial Data Clearinghouse. To build administrative support, focus on these ideas: Incorporating metadata into the data development process begins with a fundamental shift in the way an organization looks at life. Given the reality that government agencies operate differently than private industry, the operational paradigm shift that must occur within these agencies is often difficult. To make this shift, begin by building administrative, technical, and organizational support. Build support from within The Value of Metadata

17. Understanding the Value and Importance of Proper Data Documentation Making the Business Case – Technical Support The primary responsibility for metadata creation will most likely fall to your technical staff. As such, it is vital that you are able to stress the individual benefits of metadata creation along with the institutional benefits. Reduces the workload over the long term - Metadata contains information that can aid in quickly locating and retrieving data resources. The metadata does this through the use of specific criteria such as keywords, bounding coordinates, time period, data type, entities and attributes, etc. Field fewer data inquiries - Well-written metadata contains within it the information most people will need to evaluate, access, and use your data. Providing this information will help cut down on staff response time to data inquiries. Document individual contributions - The initial metadata record produced for a given data set establishes the core content that will persist, with updates, for the life of the data set. This provides data developers an opportunity to document their efforts and contributions and serves as a tangible performance indicator that may be incorporated into organizational and individual evaluations. To build technical support, stress the individual benefits of metadata. It’s also important to include your technical staff in the decision-making process as it pertains to metadata program design and feasibility analysis. They may bring unique insight into this process, and it will foster a spirit of ownership in your metadata program, thereby increasing the chances it will be successful. The Value of Metadata

18. Understanding the Value and Importance of Proper Data Documentation Building Technical Support Incorporate metadata expectations into job descriptions and performance standards. By establishing the ground rules up front, you’ll avoid ambiguities later on. This works well for new hires, but existing staff will have to be approached a bit differently. Training Provide staff development opportunities by providing the three ‘T’s’ – training, tools, and time. In most situations, your staff will be carrying a full workload. To ensure that they will be able to become contributors to the metadata program, it’s important to make sure they have the time to get trained in metadata creation, as well as time to become familiar with the metadata tools available. There are several other things to consider when it comes to building strong staff support, including: The Value of Metadata

19. Understanding the Value and Importance of Proper Data Documentation Develop Metadata Templates Organizational templates facilitate consistent and efficient metadata creation. Much of the angst regarding metadata is associated with related standards including the Content Standard for Digital Geospatial Metadata (CSDGM) and the forthcoming International Organization for Standardization (ISO) Geographic Information – Metadata standard. The standards are extensive and somewhat overwhelming because they are written to address a wide range of geospatial data types (imagery, GIS files, GPS data, geocoded databases) developed by a wide range of organizations. Individual organizations can address the problem by building custom metadata templates. Templates are built by extracting those metadata fields pertinent to the organization and the specific data types and geographies of the organization. In addition, libraries can be built to provide information about contacts, sources, and methodologies common to the organization. A suggested method for building such templates includes: 1. Adopt all mandatory fields specified by the national metadata standard 2. Adopt all ‘mandatory if applicable fields’ pertinent to the data type or organization 3. Identify ‘optional fields’ of interest to the organization 4. Create a pilot record from the draft template 5. Have the pilot reviewed and revised by administrators, analysts, technicians, and contributing scientists 6. Identify those fields that tend to remain consistent. This may include: access and use constraint statements data distribution methods and contacts contact information north, south, west, east bounding coordinates coordinate system and datum place keywords native data set environment source citations The Value of Metadata

20. Understanding the Value and Importance of Proper Data Documentation After the template or templates have been designed, map those fields included in the template to the individuals or groups within your organization that are responsible for that information. This mapping has the effect of destroying the perception that metadata is a huge, onerous task, and turns it into a series of manageable steps within the data development process. These smaller steps are also easier to incorporate into the process as it proceeds, writing as you go if you will, and eliminates the need for a much larger and more involved effort at the end of the project. Here is an example of how this mapping might occur: Distribute the effort Map metadata fields to the existing data development workflow. Data Development StageMetadata Information Data Planning Identification Information Title, Originator, Abstract, Purpose, Keywords, Time Period Data Organization Point, Raster, Vector Spatial Referencing Coordinate System and Datum Entity and Attributes (planned) Data Processing Data Quality Completeness, Positional Accuracy, geoprocessing steps Data Analysis Data Quality Attribute Accuracy, analysis steps Entity and Attributes (results) Metadata Reference The Value of Metadata

21. Understanding the Value and Importance of Proper Data Documentation Collecting Metadata Develop/Integrate Metadata Collection Tools To effectively implement your metadata program, you will need to use a tool or tools to collect the metadata. In its simplest form, metadata can be collected using a pencil and pad of paper. In fact, some of your metadata may be sitting on your desk in just such format. However, it is much more likely that you will employ something a bit higher on the technology ladder. Let’s take a look at some options. Form Documents – Whether hardcopy or digital, these are perhaps the most versatile means of capturing information at various stages of the data development process. These forms can be customized in such a way as to facilitate an efficient collection effort, and may even be done in such a way that limits the exposure of most personnel to the metadata standard. Customized forms give you control over content, and allows you to guide the contributor through the information collection process. Database / Spreadsheets - These tools allow you to store metadata information in data tables. Column headings relate to specific metadata fields, and you can develop records for each data set. Within the relational database realm, you can build in the relationship between the workflow discussed earlier to specific data tables. One big advantage of working with a database is the ability to integrate production rules such as population of mandatory fields, the use of ‘pick lists’ for sources and contacts, and automated QA/QC routines. Shareware Metadata Products - In order to comply with Executive Order 12906, government agencies began developing in-house tools for metadata creation. Most of these are available to the public. Some were developed for very specific data, others were more broad in their approach. For a description of several that are available, see the section on metadata tools. Commercial Metadata Products - Commercial metadata products fall into two categories: stand-alone and GIS-internal. Stand-alone products generally allow the user to ‘harvest’ some metadata information directly from the geospatial data set and provide a user-interface for additional data entry. These programs are typically robust production tools that facilitate the building of templates and libraries and enable interaction with a range of data types. As such, they are particularly useful to organizations that produce and manage data using multiple data development software. GIS-internal products also provide a data entry interface but, due to their proprietary nature, are able to harvest more information directly from the data set. When considering the purchase of metadata production software, some things to consider include: - Is the software easy to understand and use?- Is it built to existing and applicable metadata standards? - Does it automatically capture and update metadata from the data set?- Does it allow for the use of digital forms for customized data entry? - Does it support global updates and edits?- Does it provide a means for optional and/or custom viewing formats? - Does it package the metadata with the data?- Does it support required metadata import and export formats? The Value of Metadata

22. Understanding the Value and Importance of Proper Data Documentation Targeting Success Develop Metadata Policies and Procedures Success should be the ultimate goal of your metadata program. To help reach that goal, consider putting in place procedures and policies that will help guide participation, streamline operations, and encourage compliance. Develop these policies and procedures with the following in mind: Assign Responsibilities – Once you have your workflow mapped and you have decided on a toolset, begin assigning responsibilities for collection and management of the metadata. Managers can be responsible for documenting metadata information mapped to the data planning stage, coordinating overall collection efforts, and enforcing policies. Technicians can be responsible for documenting metadata information mapped to the data processing stage, and can build data source citations and contact information libraries. Scientists and field staff can be responsible for reviewing and revising metadata information pertinent to data collection methods and findings. Analysts can be responsible for documenting metadata information mapped to the data analysis stage and assisting technicians in metadata documentation. Information technology / system managers can be responsible for developing and maintaining the metadata collection ‘tool’, managing and updating metadata records, and within and external to the organization. Assign Priorities - Trying to decide which data set to document first? If possible, start with current products, and then work back through historical data sets. Develop a plan that considers: What is the core, or framework, value of the data set to the organization? What is the utility of the data set within the organization? How many external requests do you receive for the data set? What historical significance is the data set to your organization? Establish Administrative Guidelines - Guidelines assist in understanding and implementing the program objectives. Some options for these guidelines include: Organizational compliance is defined by the use standards and templates. Standardized language should be established for metadata distribution liability and access/use constraint statements. (Check with your legal staff on this one.) When using contractors, develop boilerplate language with respect to metadata creation as a criteria for contract deliverables of external data development. Develop job descriptions for remote sensing/GIS positions that include metadata skills. Require units within your organization to publish their metadata holdings. Publish a metadata ‘Standard Operating Procedure’ (SOP) document that outlines and specifies the policies and procedures your organization adopts. Implement and Advocate - Advocating participation will help integrate metadata into the data development process. Here are some suggestions to help increase participation: Provide your staff with appropriate metadata training and time to learn how to use the metadata tools you’ve decided on. Encourage your staff to publish their efforts in professional and research journals. Provide incentives such as prizes and awards. Utilize project ‘punch lists’ that indicate that the work is not complete until the metadata is done. Present metadata as a management priority during staff and management meetings, memo’s, and presentations. The Value of Metadata

23. Understanding the Value and Importance of Proper Data Documentation At the conclusion of this module the participant will be able to Describe the relationship between data and metadata List 3 types of data that can benefit from metadata List the 2 most common ways used to search data Objectives Metadata–A Component of Data

24. Understanding the Value and Importance of Proper Data Documentation Discovering Data Through Metadata We’ve come full circle. User http FGDC Gateway Z39.50 Clearinghouse Servers Industry & NGO’s Universities State agencies Federal agencies Web user interface Gateway using standard protocols Multiple organizations providing metadata Remember discussing the external value of data discovery? That value, whether internal or external, is one of the most important functions of metadata. In the external world of metadata, the FGDC has established the Clearinghouse system to help users search for and find data of interest, much the way a library catalog helps you find a book you are interested in. “The Clearinghouse Activity, sponsored by the FGDC, is a decentralized system of servers located on the Internet which contain field-level descriptions of available digital spatial data. This descriptive information, known as metadata, are collected in a standard format to facilitate query and consistent presentation across multiple participating sites.” www.fgdc.gov/clearinghouse/background.html

25. Understanding the Value and Importance of Proper Data Documentation Discovering Data Through Metadata Pick a door, any door. The Clearinghouse has six gateways that allow a user to access the entire collection of internet servers, or nodes. Imagine a room with six doors. Inside that room is over 425 file cabinets, each containing a unique set of records. It doesn’t matter which door you choose to enter through, you have access to the same file cabinets. The gateways function in just this manner, giving the user access to the same collection of clearinghouse nodes, irregardless of which gateway is selected for entry. The gateways are distrubuted to help regulate the flow of requests. The Clearinghouse system works well, as long as it is fueled by well-written and fully-developed metadata!

26. Understanding the Value and Importance of Proper Data Documentation Using the FGDC Clearinghouse Variety is the spice of life! Once inside the gateways, a user can search any of the available servers using a variety of criteria, including bounding coordinates, time, and keywords. Searches return a list of metadata records by title. Search options include: NSDI Search Wizard – This method bins the servers into topic categories based on the type of metadata they house. Map interface with place names – This method allows the user to select an area of interest from a map. Place names – This option allows the user the option of selecting an area of interest from a pull-down list.

27. Understanding the Value and Importance of Proper Data Documentation Using the FGDC Clearinghouse This Wizard is no Harry Potter! When using the NSDI Search Wizard option, you select a maximum of 4 thematic areas to search, followed by any time period, keyword, or bounding coordinate information. What is returned to you is a list of available nodes which house metadata in one of the thematic areas selected, and which also meets the other criteria specified in the search. This list of nodes comes pre-selected. If you decide to search a subset of those, you can deselect all nodes, then reselect only those nodes of interest. While this method helps whittle down the list of available nodes, keep in mind that the node selection process employed by the search wizard is wholly dependant upon the node administrator’s description of the metadata records housed on that node.

28. Understanding the Value and Importance of Proper Data Documentation Using the FGDC Clearinghouse Successful searches provides links to the nodes. Once a search is complete, the nodes with metadata that met the search criteria will become active links. Clicking on the name of the server will return a list of metadata records by their title. Because of this, it is important to consider the title when creating your metadata. We’ll talk more about this later. To see the full metadata record, simply click on the “Show Metadata” link.

29. Understanding the Value and Importance of Proper Data Documentation Using the FGDC Clearinghouse Viewing the metadata record. If available, the returned record is displayed in html format, with links to each of the 7 main sections of the record, if available. Other viewing formats possible are SGML, XML, DIF (a NASA format), FAQ, and plain text. Within the first section of the metadata is an element called “Online_Linkage” and is used to provide a URL for citation information. Many metadata creators use this link to point to either a project-specific Web site, a general Web site about the organization, or in some cases, an actual ftp site which initiates a download process. An FGDC metadata record has an indented hierarchal structure.

30. Understanding the Value and Importance of Proper Data Documentation Customized Search Interfaces The NOAA Coastal Services Center has developed a search interface featuring those nodes that house coastal metadata. This custom interface can be used for a wide-variety of applications, both external and internal. It offers an organization the ability to customize a search interface to their collection of metadata or associated nodes. It allows specific communities to be targeted, thereby making the search process more streamlined and efficient. The search mechanism offers the same functionality of the FGDC clearinghouse interface, without the heavy overhead of having over 400 possible nodes to search. For more information about the code for this interface, contact the NOAA Coastal Services Center at metadata@csc.noaa.gov.

31. Understanding the Value and Importance of Proper Data Documentation Exercise 1 – The Hunt for Good Metadata For this exercise, we will be using the Federal Geographic Data Committee (FGDC) Geospatial Data Clearinghouse to search for data of interest. The Geospatial Data Clearinghouse is a collection of over 400 metadata servers (called nodes) that house metadata describing data, primarily for use in Geographic Information Systems (GIS), image processing systems, and other modeling software. These data collections can be searched through a single interface based on their metadata. To access the clearinghouse, go to http://www.fgdc.gov. This is the main page for the FGDC. On the left side of the page, click on the clearinghouse link. This takes you to the clearinghouse web page. Clicking on the top link in the center of the page, "Search for Geospatial Data", will allow access to the search system. You should see a picture of the globe showing six gateways. Choose one of these gateways (they all lead to the same collections of metadata). On the next page, you will be presented with four options: Use the NSDI Search Wizard to "smart select" servers and data Search Clearinghouse sites using map interface with place names Search Clearinghouse sites using place names Check status of participating Clearinghouse collection servers The top 3 are search options. The first option allows you to search for metadata by topic area. The NSDI Search Wizard bins the metadata nodes into 21 topic areas based on the 'flavor' of metadata each node houses. The next two search interfaces are fairly similar; the main difference is that the second option allows the user to define a geographic bounding box of the area of interest on a map. The last option allows the user to check on the operational status of the individual clearinghouse nodes. Take a few minutes to examine each of these 4 options. As you are experimenting with the search interfaces, observe how well each different interface works. Test each for speed in returning results, and think about how intuitive each option is for constructing a search query. When you have finished exploring these different search interfaces, choose one and begin a search for data that you are interested in. You may have a particular need related to a current project you are involved with. If you are not currently working on a project that requires you to find data, then search for data that is of interest to you.

32. Understanding the Value and Importance of Proper Data Documentation Exercise 2 – Reading a Metadata File In this exercise, we will use a metadata file that you found from the clearinghouse search system exercise. Your record may or may not provide all the information asked for here. What you should focus on while working through this exercise is whether or not the record you have chosen provides an adequate level of information to be of use to a user. What's the title of the data set? What are the bounding coordinates? What is the presentation form of the geospatial data? Why was this data set created? Find and list three place keywords. Who created the data set? How complete is the data? How reliable are the data? What problems remain in the data set? What spatial reference method is used to represent the data? List three data set attributes and the definition of each. How can someone get a copy of the data? Who distributes it? What sources were used to develop this data set?

33. Understanding the Value and Importance of Proper Data Documentation Metadata is a document that describes your data’s content, condition, and quality. Metadata is an integral component of your data and should never be viewed as a separate entity. Metadata’s traditional external value includes data discovery, assessment, access, and use. Metadata is also a powerful tool that an organization can utilize to build a strong internal data management structure. In order to be successful, this internal management structure requires building support at the administrative, technical, and organizational levels. A Quick Review

34. Understanding the Value and Importance of Proper Data Documentation At the conclusion of this module the participant will be able to List the seven main and three supporting sections of the content standard Describe the difference between a compound element and a data element Describe the 3 levels of conditionality as defined in the content standard Given an element name, be able to use the graphical production rules to find the definition of that element within the structure of the content standard Objectives Using the FGDC Content Standard

35. Understanding the Value and Importance of Proper Data Documentation Standardized Metadata What is Training? It’s easier when we all play on the same field with the same rules. When you think about it, our we interact with standards on a daily basis. Think about putting gas in your car, measuring ingredients to bake bread, or plugging a lamp into a wall socket. Each of these is controlled by standards. Metadata should be no different. Standardized metadata ensures consistency. It serves as a uniform summary description of the data set. The agency responsible for developing the metadata standard is the Federal Geographic Data Committee, a 19 member interagency committee composed of representatives from the Executive Office of the President, Cabinet-level and independent agencies. The FGDC is responsible for developing the National Spatial Data Infrastructure (NSDI) in cooperation with organizations from State, local and tribal governments, the academic community, and the private sector. The NSDI encompasses policies, standards, and procedures for organizations to cooperatively produce and share geographic data.

36. Understanding the Value and Importance of Proper Data Documentation The Content Standard for Digital Geospatial Metadata Whew! That’s a mouthful! Let’s just call it ‘The Standard.’ The FGDC created the Content Standard for Digital Geospatial Metadata (CSDGM, or “the standard”) to address the requirements found in Executive Order 12906. This Presidential mandate states “…each agency shall document all new geospatial data it collects or produces, either directly or indirectly, using the standard under development by the FGDC, and make that standardized documentation electronically accessible to the Clearinghouse network.” To accomplish this, the CSDGM utilizes common terms, common definitions, and a common language and structure. It establishes the names and definitions of the various elements in the standard. The primary objectives of the standard are to help users determine if a particular data set is available, to assess the data for their intended use, access and transfer the data, and how to use the data. Who? Who collected the data? Who processed the data? Who wrote the metadata? Who to contact for questions? Who to contact to order? Who owns the data? Where? Where were the data collected? Where were the data processed? Where are the data located? What? What are the data about? What project were they collected under? What are the constraints on their use? What is the quality? What are appropriate uses? What parameters were measured? What format are the data in? When? When were the data collected? When were the data processed? How? How were the data collected? How were the data processed? How do I access the data? How do I order the data? How much do the data cost? How was the quality assessed? Why? Why were the data collected? Metadata, written in the CSDGM format, can answer questions such as these:

37. Understanding the Value and Importance of Proper Data Documentation Using The CSDGM The FGDC’s Content Standard for Digital Geospatial Metadata is a collection of 334 compound and data elements. It is arranged into numbered chapters called “sections.” Each section is organized into a series of elements that define the information content for metadata to document a data set. Each section provides: Section name and definition Compound element names and definitions Data element names and definitions Production rules, or conditionality, for each element

38. Understanding the Value and Importance of Proper Data Documentation The Structure of the Content Standard Data Quality Information Data Quality Information Spatial Data Organization Information Spatial Data Organization Information Spatial Reference Information Spatial Reference Information Entity & Attribute Information Entity & Attribute Information Metadata Distribution Information Distribution Information Metadata Reference Information Metadata Reference Information Identification Information Identification Information Seven Main Sections 9 Time Period Information 9 Time Period Information 10 Contact Information 10 Contact Information 8 Citation Information 8 Citation Information Three Supporting Sections 1 2 3 4 5 6 7 The FGDC has also produced a workbook version of the standard which presents the production rules in a graphical format. These graphics illustrate: The structure of the standard Element groupings Element conditionality Element repeatability

39. Understanding the Value and Importance of Proper Data Documentation Section Compound Element Data Element Deciphering the Graphics The standard is organized using numbered chapters called "sections." Each section begins with the name and definition of the section. These are followed by the component elements of the section. Each section provides the names and definitions of the component elements, information about the types of values that can be provided for the elements, and information about the elements that are mandatory or repeatable. A compound element is a group of data elements and other compound elements. All compound elements are described by data elements, either directly or through intermediate compound elements. Compound elements represent higher-level concepts that cannot be represented by an individual data element. A data element is a logically primitive item of data. The entry for a data element includes the name of the data element, the definition of the data element, and a description of the values that can be assigned to the data element.

40. Understanding the Value and Importance of Proper Data Documentation Data Elements – The Things You “Fill In” Data Element The form for the definition of a data element is: Data element name -- definition. Type: (choice of "integer", "real", "text", "date", or "time") Domain: (describes valid values that can be assigned) The information about the values for that data elements include a description of the type of value, and a description of the domain of values. The type of the data element describes the kind of value to be provided. The choices are "integer" for integer numbers, "real" for real numbers, "text" for ASCII characters, "date" for day of the year, and "time" for time of the day. The domain describes valid values that can be assigned to the data element. The domain may specify a list of valid values, or restrictions on the range of values that can be assigned to a data element. An example of the definition of a data element is: Abstract -- a brief narrative summary of the data set. Type: text Domain: free text

41. Understanding the Value and Importance of Proper Data Documentation Defining Conditionality The standard defines sections, compound elements, and data elements as being mandatory, mandatory if applicable, or optional. The graphical productions rules uses colors to depict these different levels of conditionality. Mandatory Mandatory elements are colored yellow. Information for these elements must be provided. If the information is not know for a mandatory data element, the entry "Unknown" or a similar statement should be given. An example is compound element 1.2, Description, which is composed of the three data elements, Abstract, Purpose, and Supplemental Information. The first two, Abstract and Purpose, are mandatory and information must be provided for these elements. The third data element, Supplemental Information, is optional. Mandatory Mandatory elements are colored yellow. Information for these elements must be provided. If the information is not know for a mandatory data element, the entry "Unknown" or a similar statement should be given. An example is compound element 1.2, Description, which is composed of the three data elements, Abstract, Purpose, and Supplemental Information. The first two, Abstract and Purpose, are mandatory and information must be provided for these elements. The third data element, Supplemental Information, is optional. Mandatory Data Element Mandatory Compound Element

42. Understanding the Value and Importance of Proper Data Documentation Mandatory if Applicable Mandatory if applicable elements are colored green. Information for these elements must be provided if the data set exhibits the characteristic defined by the element. An example is compound element 4.2, Vertical Coordinate System Definition, from Section 4, Spatial Reference Information. This compound element becomes mandatory if a data set has vertical (altitude/elevation or depth) information. If the data set does not have vertical information, the element is not applicable and is not provided. Mandatory if Applicable Mandatory if applicable elements are colored green. Information for these elements must be provided if the data set exhibits the characteristic defined by the element. An example is compound element 4.2, Vertical Coordinate System Definition, from Section 4, Spatial Reference Information. This compound element becomes mandatory if a data set has vertical (altitude/elevation or depth) information. If the data set does not have vertical information, the element is not applicable and is not provided. Defining Conditionality Mandatory if Applicable Data Element Mandatory if Applicable Compound Element

43. Understanding the Value and Importance of Proper Data Documentation Optional Optional elements are colored blue. Information for these elements is provided at the discretion of the data set producer. An example is data element 1.2.3, Supplemental Information. This particular element can be used to provide a variety of information not otherwise captured in other elements of the standard. Users can provide a variety of Web site URL’s, lists, tables, or other information. Optional Optional elements are colored blue. Information for these elements is provided at the discretion of the data set producer. An example is data element 1.2.3, Supplemental Information. This particular element can be used to provide a variety of information not otherwise captured in other elements of the standard. Users can provide a variety of Web site URL’s, lists, tables, or other information. Defining Conditionality Optional Data Element Optional Compound Element

44. Understanding the Value and Importance of Proper Data Documentation Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 Compound elements are composed of other compound and data elements. The composition is represented by nested boxes. In this example, Compound Element 1 is composed of Compound Element 1.1 and Data Element 1.2. Nested within Compound Element 1, Compound Element 1.1 is composed of Data Element 1.1.1 and Data Element 1.1.2. How Elements Are Grouped

45. Understanding the Value and Importance of Proper Data Documentation Repeating Elements Compound Element 1 (can be repeated unlimited times) Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 If an element can be repeated independently from other elements, it will be indicated as such below the element name. Using the example to the left, this group of elements can be repeated unlimited times. If all elements were used, the structure of these elements would look like this: Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2

46. Understanding the Value and Importance of Proper Data Documentation Examples of Conditionality Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2 In example A, all elements are mandatory, so three pieces of information would be required. For example B, Data Element 1.2 is required. The other two data elements are found within the optional Compound Element 1.1. If it was decided these elements were important, data elements 1.1.1 and 1.1.2 would need to be included. Example AExample BExample CExample D Example C shows only Data Element 1.1.1. being mandatory. However, if Data Element 1.1.2 applies, then it must be provided. The remaining data element is optional, and is left to the discretion of the provider. In example D, the decision for inclusion occurs at the Compound Element 1 level. If the definition of Compound Element 1 does not apply, nothing is provided. However, if it is applicable then the rules apply as in Example C.

47. Understanding the Value and Importance of Proper Data Documentation Exercise 3 – Using the Graphical Production Rules In this exercise, we will interpret a metadata file that you've found using the clearinghouse search system in exercise 1. Using the graphical representation of the production rules and the CSDGM workbook, locate the answers to the following questions. Where is the definition for the data element West Bounding Coordinate? Where is the definition for the compound element Description? What data elements make up the compound element Description? What is the domain for the data element Process Date? What is the domain for the data element Address Type? Name one compound element from Section 2 that can be repeated. Name one data element from Section 8 that can be repeated. Is the data element Abstract mandatory? Is the data element Ellipsoid Name mandatory? What is the short name for the compound element Direct Spatial Reference Method? What is the data type for the data element Abscissa Resolution? __________________________________________________