1. Laura Russell (larussell@vertnet.org) Programmer VertNet Buenos Aires (Argentina) 28 September 2011 Training course on biodiversity data publishing and fitness-for-use in the GBIF Network, 2011 edition How to publish (data set) meta data

2. Data publishing process

3. Outline - Why metadata? - The GBIF EML profile - Metadata standards - Preparation of metadata - Where does the metadata go? - Preparing metadata (examples)

4. Outline - Why metadata? - The GBIF EML profile - Metadata standards - Preparation of metadata - Where does the metadata go? - Preparing metadata (examples)

5. ”Data Intensive Science” ”Fourth Science Paradigm” e-Infrastructure Reflection Group (European Strategy Forum on Research Infrastructures). Report on Data Management, November 2009. http://www.e-irg.eu/images/stories/publ/task_force_reports/dmtfjointreport.pdf ”Digital Data Deluge” The Fourth Paradigm: Data-Intensive Scientific Discovery http://research.microsoft.com/en-us/collaboration/fourthparadigm/contents.aspx high quality metadata for long-term curation and use of data sets Key requirement: Why metadata?

6. William K. Michener, Meta-information concepts for ecological data management, Ecological Informatics, Volume 1, Issue 1, January 2006, Pages 3-7, ISSN 1574-9541, DOI: 10.1016/j.ecoinf.2005.08.004. (http://www.sciencedirect.com/science/article/B7W63-4HJRS57-3/2/ea2e08412c6776456f540e66983546c0) Information about data sets deteriorates over time!

7. Why metadata? Metadata supports: - Discovery - Interpretation/Evaluation - Provenance - Quality - Fitness-for-use - Analytical re-use

8. Outline - Why metadata? - The GBIF EML profile - Metadata standards - Preparation of metadata - Where does the metadata go? - Preparing metadata (examples)

9. Metadata Standards Ecological Metadata Language (EML) v2.1.1 http://knb.ecoinformatics.org/software/eml/ Dublin Core http://dublincore.org/documents/dcmi-terms/ Directory Interchange Format (DIF) http://gcmd.nasa.gov/User/difguide/difman.html ISO 19115/19139 Geographic Metadata ISO 19115: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=26020 ISO 19139: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=32557

10. Metadata Standards Natural Collections Descriptions (NCD) http://www.tdwg.org/standards/312/ Federal Geographic Data Committee (FGDC) Biological Profile* http://www.fgdc.gov/standards/projects/FGDC-standards-projects/metadata/biometadata/ *An extension of the FGDC CSDGM (Content Standard for Digital Geospatial Metadata) Multimedia Resources Metadata Schema http://www.tdwg.org/charters/article/view/448/36

11. ISO 19115/19139 North American Profile of ISO 19139 http://www.fgdc.gov/standards/projects/incits-l1-standards- projects/NAP-Metadata/napMetadataProfileV101.pdf/view Several Resources available for crosswalk; transform; view EML to FGDC Biological Profile https://code.ecoinformatics.org/code/eml/trunk/lib/eml2tonbii/ # FGDC CSDGM to ISO Transform # FGDC CSDGM to ISO Crosswalk # ISO XML to HTML View: # FGDC BIO to ISO Transform # FGDC BIO to ISO Crosswalk http://www.ncddc.noaa.gov/technology/metadataandxml/view FGDC CSDGM ISO 19139 EML to ISO 19139 http://code.google.com/p/gbif-metadata/source/browse/trunk/metadata/src/main/resources/eml2iso19139.xsl Open source INSPIRE- compliant MD editor (multilingual functionality) http://www.inspire-geoportal.eu/EUOSME/

12. Metadata and Languages A Multilingual Metadata Catalog for the ILTER: Issues and Approaches. Vanderbilt, K.L., et al., Ecological Informatics, Volume 5, Issue 3, May 2010, Pages 187-193, doi:10.1016/j.ecoinf.2010.02.002 Adopt a lingua franca, e.g., English -data publishers provide discovery level metadata in English; -full metadata in local language. Just use local language with keywords from multilingual thesauri, e.g., GEMET, AGROVOC -GEMET, the GEneral Multilingual Environmental Thesaurus; 27 languages. http://www.eionet.europa.eu/gemet/ http://www.eionet.europa.eu/gemet/ -AGROVOC; agriculture, forestry, fisheries, food and related domains; 20 languages. http://www4.fao.org/agrovoc/default.htm http://www4.fao.org/agrovoc/default.htm Long term solution: multilingual ontologies Issues? -additional burden; tools, metadata standards

13. Outline - Why metadata? - The GBIF EML profile - Metadata standards - Preparation of metadata - Where does the metadata go? - Preparing metadata (examples)

14. GBIF EML Profile - Requirements gathering -GBIF Metadata Task Group http://www2.gbif.org/GBIF-MIFTG-Report.pdf -EML; ISO 19115; NCD; INSPIRE Directive http://community.gbif.org/mod/file/download.php?file_guid=10915http://community.gbif.org/mod/file/download.php?file_guid=10915; http://community.gbif.org/mod/file/download.php?file_guid=5656 http://community.gbif.org/mod/file/download.php?file_guid=5656 http://rs.gbif.org/schema/eml-gbif-profile/1.0/eml-gbif-profile.xsd - GBIF EML schema http://community.gbif.org/pg/groups/5258/gbif-metadata-networkhttp://community.gbif.org/pg/groups/5258/gbif-metadata-network/ - GBIF community site: metadata network - GBIF profile documentation http://links.gbif.org/gbif_metadata_profile_how-to_en_v1 http://links.gbif.org/gbif_metadata_profile_guide_en_v1

15. Outline - Why metadata? - The GBIF EML profile - Metadata standards - Preparation of metadata - Where does the metadata go? - Preparing metadata (examples)

16. Preparing metadata - Metadata editors e.g., IPT; Spreadsheet template; Morpho; EUOSME - Scripting -Output directly from existing metadata database -Transform from another metadata specification to EML - Editing XML directly -Validation essential

17. Outline - Why metadata? - The GBIF EML profile - Metadata standards - Preparation of metadata - Where does the metadata go? - Preparing metadata (examples)

18. Where does the metadata go? http://metadata.gbif.org

19. Sources of Metadata GBIF Data Cache - Registered IPT installations - National/regional/organisation level catalogues - Thematic catalogues, e.g., OBIS GBIF approach: -no imposed metadata standard or preferred catalogue implementation for participants; -avoidance of lossy conversions in submitting metadata GBIF Participants External networks e.g., Knowledge Network for Biocomplexity (KNB)

20. GBIF metadata architecture GBIF Catalogue GBIF Registry EuroGEOSS Catalogue e.g., GBIF Node IPT Instance Catalogue e.g.,KNB GBIF Data Cache OAI-PMH Direct payload GBIF metadata catalogue specification: http://links.gbif.org/gbif_metadata_catalogue_specification.pdf

21. OAI-PMH Open Archives Initiative Protocol for Metadata Harvesting Providing a low-barrier mechanism for interoperability across distributed metadata repositories Data providers expose metadata Service providers consume metadata through a client application known as a harvester that issues OAI-PMH service requests over HTTP. http://www.openarchives.org/pmh/ GBIF: role as harvester and provider

22. Outline - Why metadata? - The GBIF EML profile - Metadata standards - Preparation of metadata - Where does the metadata go? - Preparing metadata (examples)

23. Spreadsheet processor 1.Download the spreadsheet from the site http://tools.gbif.org/spreadsheet-processor/templates/metadata/metadata- 1_v1.xls http://tools.gbif.org/spreadsheet-processor/templates/metadata/metadata- 1_v1.xls 2.Complete the spreadsheet 3.Transform it as a GBIF metadata profile file by using the spreadsheet processor http://tools.gbif.org/spreadsheet-processor/ http://tools.gbif.org/spreadsheet-processor/ Note: the processor doesn’t publish a file to GBIF, it provides a publication-ready file.

24. IPT metadata editor 1.Create a new resource in the IPT 1.Complete the metadata -Dataset (Resource) -Project -People and Organisations -Keyword Set (General Keywords) -Coverage -Taxonomic Coverage -Geographic Coverage -Temporal Coverage -Intellectual Property Rights -Methods -Additional Metadata and Natural Collections Descriptions Data 2.Publish it -Metadata for published and unpublished data sets -Output as part of DwC-A zip file (EML.xml)

25. Presenter (email) Role Organization Buenos Aires (Argentina) 28 September 2011 Training course on biodiversity data publishing and fitness-for-use in the GBIF Network, 2011 edition How to publish (data set) meta data