UMass Seminar Presentation October 7, 2009 Data and Data Management: Publish (your data) or Perish Presented at the UMass Seminar Series October 7, 2009.

Slides:

Advertisements

Similar presentations

BEER Workshop November 9, 2008 Has Data Management Gone Mainstream? Presented at the BEER Workshop Coconut Grove (Miami), Florida November 9, 2008 Robert.

Advertisements

DSpace: the MIT Libraries Institutional Repository MacKenzie Smith, MIT EDUCAUSE 2003, November 5 th Copyright MacKenzie Smith, This work is the.

NERACOOS: Unlocking the power of data Riley Young Morse | Gulf of Maine Research Institute.

Integrated Ocean Observing System (IOOS) Data Management and Communication (DMAC) Standards Process Julie Bosch NOAA National Coastal Data Development.

Visualizing Fitness for Purpose Bob Groman and Dicky Allison Biological and Chemical Oceanography Data Management Office Woods Hole Oceanographic Institution.

StatCat Building a Statistical Data Finder ssrs.yale.edu/statcat Steven Citron-Pousty Ann Green Julie Linden Yale University.

Caro-COOPS Data Management: Metadata. Cast-Net addresses the need for improved connectivity among coastal observing systems by creating a regional framework.

Rome (26 Nov 2009) D4Science World User Meeting Moi Khim TAN The WorldFish Center Strengthening Information Access and Dissemination in Support of Effective.

The SouthEast Coastal Ocean Observing SECOORA Meeting Regional Association (SECOORA) June 11-12, 2007 SECOORA Data Management and Communications SECOORA.

Integrated Ocean Observing System Data Management and Communications March 2004 The US Integrated Ocean Observing System (IOOS) Plan for Data Management.

Exchanging Environmental Data for the Gulf of Maine 2007 Exchange Network Collaboration Grant Prep webinar for development meeting October 28th-29th, 2008.

Biological and Chemical Oceanography Data Management Office 1 of 12 An Introduction to the Biological and Chemical Oceanography Data Management Office.

Report on National activities, 2012 Poland Waldemar Walczowski SAON Board Meeting, 1-2 October 2012 Potsdam.

U.S./European Partnerships in Coastal Atlases and Coastal/Ocean Informatics Coastal Zone 2007 – Portland, Oregon National Coastal Zone Management Community.

NOAA Metadata Update Ted Habermann. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International.

ODINCARSA Planning Meeting II, December 7-10, 2009.

From Data Integration to Integrated Solutions Tom Shyka, GMRI Paul Currier, NH DES Water Quality Workshop MACOORA - NERACOOS -SECOORA IOOS, USGS, EPA,

Data Management Practices: BCO-DMO’s Successes and Challenges Bob Groman BCO-DMO Woods Hole Oceanographic Institution NERACOOS/NeCODP Data Management Workshop.

IOOS DMAC 2015 Jon “OTN is a biological observing system of the UN’s Global Ocean Observing System.” Following on from the Census.

Metadata Guides for Smarties Marine Metadata Initiative URL:

Agenda: DMWG SM policy status ESIP meeting recap Reminder - DM Webinar Series New and updated web pages on DM website Metadata Training Sessions CDI meeting.

ADC Meeting ICEO Standards Working Group Steven F. Browdy, Co-Chair ADC Workshop Washington, D.C. September, 2007.

The Marine Metadata Interoperability Project A Model for Community Collaboration September 23, 2010 Nan Galbraith WHOI.

The Multipurpose Marine Cadastre A Tool for Planning & Decision Making in the Marine Environment Christine Taylor Minerals Management Service (MMS) David.

Public Participation and the Advisory Committee Process A Collaborative Partnership For Water Resources Toni M. Johnson, Chief Water Information Coordination.

1 Systems Development Cheryl Itkin SIMCorB Meeting RTP, NC June 29-30, 2000 SIMCorB Organization Policy Systems Outreach.

ENEON first workshop Observing Europe: Networking the Earth Observation Networks in Europe September, Paris Summary on data availability, sharing,

Larger GIS Community Can answer: –Local questions at small extents Spatial and temporal extents limited –Global questions at low resolution (while ignoring.

Peter H. Wiebe and Nancy Copley Woods Hole Oceanographic Institution How does CMarZ Work? CMarZ Information System / Database /OBIS/ Species Pages.

Planning for Arctic GIS and Geographic Information Infrastructure Sponsored by the Arctic Research Support and Logistics Program 30 October 2003 Seattle,

U.S. Department of the Interior U.S. Geological Survey Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center.

U.S. GLOBEC Pan-Regional Synthesis Workshop 1 Presentation to the U.S. GLOBEC Pan-Regional Workshop 29 November 2006 Bob Groman Data Access and Associated.

Design central EMODnet portal Objectives, Technical Proposal and Consultation Process.

IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal MINCyT,

1 Global Systems Division (GSD) Earth System Research Laboratory (ESRL) NextGen Weather Data Cube Chris MacDermaid October, 2010.

Building a Web-based GIS Portal For the Great Lakes Observing System (GLOS) Pete Giencke Program Specialist Data and Information Management.

Increasing Momentum in the Formation of State and Regional Monitoring Councils Linda Green, co-chair, Collaboration and Outreach Workgroup, National Water.

Bob Keeley Marine Environmental Data Service Dept. of Fisheries and Oceans Ottawa, Canada Jun, 2006 SeaDataNet Meeting.

Biological and Chemical Oceanography Data Management Office slide 1 of 19 CAMEO Data Management Bob Groman Biological and Chemical Oceanography Data Management.

IODE Ocean Data Portal - ODP  The objective of the IODE Ocean Data Portal (ODP) is to facilitate and promote the exchange and dissemination of marine.

SCOR Meeting on Coordination of International Marine Research Projects The Scientific Committee on Oceanic Research (SCOR) convened a meeting on Coordination.

1 NOAA Priorities for an Ecosystem Approach to Management A Presentation to the NOAA Science Advisory Board John H. Dunnigan NOAA Ecosystem Goal Team Lead.

November 16, 2009 Page 1 of 28 Data and Data Management: Introduction to the BCO-DMO Presented to Professor Keiichi Uchida November 16, 2009 Robert C.

U.S. GLOBEC Georges Bank 2007 Phase 4B SI Meeting April 23, 2007 GoMODP, Data Interoperability and the MapServer Interface to U.S. GLOBEC Data Presented.

Canada’s National Pollutant Release Inventory (NPRI) and Data Visualization PRTR Global Round-Table Session on Next Generation PRTRs Spain, November 24.

1 Not So Strange Bedfellows: Information Standards For Librarians AND Publishers November 6, 2015.

IPY International Polar Year Progress report to STG 2.

OOI-CYBERINFRASTRUCTURE OOI Cyberinfrastructure Education and Public Awareness Plan Cyberinfrastructure Design Workshop October 17-19, 2007 University.

Jeff de La Beaujardière, PhD NOAA Sr System Architect, IOOS Program Office DAARWG Meeting Silver Spring, Integrated Ocean Observing System (IOOS)

Institutional Repositories: the DSpace Experience Ann J. Wolpert Director of Libraries Massachusetts Institute of Technology.

U.S. Environmental Protection Agency Central Data Exchange Pilot Project Promoting Geospatial Data Exchange Between EPA and State Partners. April 25, 2007.

IOOS Sea Surface Temperatures: Realizing Truly Distributed Data with Open Standards Presenter Kyle Draganov.

Greater Visibility, Greater Access QSpace QSpace Queen’s University Research & Learning Repository.

International Oceanographic Data and Information Exchange - Ocean Data Portal (IODE ODP) Enabling science through seamless and open access to marine data.

Distributed Data Servers and Web Interface in the Climate Data Portal Willa H. Zhu Joint Institute for the Study of Ocean and Atmosphere University of.

The Earth Information Exchange. Portal Structure Portal Functions/Capabilities Portal Content ESIP Portal and Geospatial One-Stop ESIP Portal and NOAA.

Future needs and plans for ocean observing in the Arctic AOOS Arctic Town Hall Futur Zdenka Willis Integrated Ocean Observing System National Program Office.

US GLOBEC Georges Bank Phase 4B Scientific Investigators’ Meeting 1 Presentation to the US GLOBEC Georges Bank Phase 4B Scientific Investigators October.

Biological and Chemical Oceanography Data Management Office slide 1 of 10 U.S. GEOTRACES Data Management Cyndy Chandler BCO-DMO ~ WHOI 23 September 2008.

FGDC Coordination Group Meeting – December 5, 2006 Tony LaVoi, NOAA December 5, 2006 FGDC Coordination Group Meeting Marine & Coastal Spatial Data Subcommittee.

Biological and Chemical Oceanography Data Management Office slide 1 of 10 The Biological and Chemical Oceanography Data Management Office (BCO-DMO) Cyndy.

The Role of the International Hydrographic Organization in Marine SDI Implementation Maureen Kenny NOAA National Ocean Service Office of Coast Survey.

NOS DataExplorer Enterprise GIS Efforts within NOAA's Ocean Service Jason Marshall (PSGS) NOAA Coastal Services Center.

IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal.

Data Browsing/Mining/Metadata

Digital Collection Development Policy

Inventory Database Participation for SECOORA Members

Flanders Marine Institute (VLIZ)

Data and Data Management: Introduction to the BCO-DMO

Design central EMODnet portal Objectives and Technical description Initial draft prepared by the Flanders Marine Institute.

Presentation transcript:

UMass Seminar Presentation October 7, 2009 Data and Data Management: Publish (your data) or Perish Presented at the UMass Seminar Series October 7, 2009 Robert C. Groman

UMass Seminar Presentation October 7, 2009 Topics to Cover Has data management gone mainstream? NSF now says: Your data or your funding “Data” is a plural noun – facts, statistics, or items of information; and metadata Accessing data: Is a picture worth a thousand bytes? Data Interoperability

UMass Seminar Presentation October 7, 2009 Points to make (somewhere) Permanent archive of data Benefits of early open access to data (with minimum/no restrictions)

UMass Seminar Presentation October 7, 2009 Purpose Metadata are data and critical for data reuse Raise level of awareness (appreciation?) for data management Want to use some formulas Difference between an engineer and a mathematician

UMass Seminar Presentation October 7, 2009 Venn Diagram: Data and Metadata All data and information (D) necessary to use the data. Data (d) Metadata (m) D ≠ m + d facts, statistics, or items of information Set Theory

UMass Seminar Presentation October 7, 2009 Probability Inversely Proportional to Time Second order effects: Length of cruise Success of cruise Participants Immediate activity following the cruise

UMass Seminar Presentation October 7, 2009 Theorems † Theorem 1: The probability that all the necessary data and information are collected and preserved to allow another researcher to properly use your data is inversely proportional to time since the data were collected. Corollary: Unless data and information are collected and preserved during the experiment (e.g. cruise), subsequent researchers will have a difficult time using your data. Theorem 2: The longer the time since the data were collected the less likely the data will be considered “final”. † Left to the reader as an exercise.

UMass Seminar Presentation October 7, 2009 Seeing Versus Using Someone’s Data Maybe you don’t want others to use your data. Hard to believe, but this does happen. For example: –I’m not done publishing my papers based on the data –My graduate student is almost done analyzing the data –It’s not final yet –My dog ate it (No, I haven’t heard this one yet, but there was a case where the data were erased.) Old/current policies and practices about data archiving New policies about data publishing and data archiving –Web accessible –NSF mandate (for real this time)

UMass Seminar Presentation October 7, 2009 Quantum Mechanics Revisited Heisenberg Uncertainty Principal (HUP) does NOT seem to apply If Δx and Δp are the uncertainties in the measurements of the position and momentum, then the product ΔxΔp is at least on the order of Planck's constant, h.Planck's constant When measuring conjugate quantities, the product of their standard deviations must be at least h / 4πconjugate quantitiesstandard deviations Not to be confused with the term observer effect (OE) which refers to changes that the act of observing will make on the phenomenon being observed. HUP does not seem to apply, but observer effect (OE) does. The more people look at the data the higher their quality.

UMass Seminar Presentation October 7, 2009 Ocean Observing → Sharing Data Northeast Coastal and Ocean Data Partnership (née Gulf of Maine Ocean Data Partnership) –“… to promote and coordinate the sharing, linking, electronic dissemination, and use of data in the Gulf of Maine region. “ –“… linking databases that are created and individually maintained by Participants ….” –“… develops the web-based, visualization, and other information technologies needed for the seamless exchange ….” –24 member organizations consisting of research, educational, non-profit, commercial, and local, state, and federal agencies. Ocean observing systems –Oceans.us: National Office for Integrated and Sustained Ocean ObservationsNational Office for Integrated and Sustained Ocean Observations –NFRA: National Federation of Regional AssociationsNational Federation of Regional Associations –NERACOOS, MACOORA, ….NERACOOSMACOORA –ORION: Ocean Research Interactive Observatory NetworksOcean Research Interactive Observatory Networks –GOOS: Global Ocean Observing SystemGlobal Ocean Observing System

UMass Seminar Presentation October 7, 2009 NERACOOS Northeast Regional Coastal Ocean Observing System (NERACOOS) efforts Rivaling the difficulties of the First and Second Continental Congresses, but NERACOOS did prevail.

UMass Seminar Presentation October 7, 2009 Northeast Coastal and Ocean Data Partnership Technical Committee Activities (2008 Report from Chair) 1)Partner table of expertise - S. Most has been gathering completed surveys from the partners. Bob G. developed a web site to add, query and review the partner records. 2) Dataset accessibility survey - An accessibility survey format has been created by the subcommittee. Many of the partner’s data links identified through a previous survey and through the GoMODP portal have been reviewed. This is still a work in progress. 3) Update technical guidance - Thanks to Anne and Lou, a section on registering metadata records with the GeoSpatial One-Stop was added to the technical guidance. In the first version, we only had a placeholder for this info. The revised version of the technical guidance is on the GoMODP web site: 4) Participate in pilot projects - We may be taking another look at the monitoring location project in light of the IOOS Regional Observation Registry ( Stay tuned for details. Modification of the EPA’S Data Exchange Template. [But is this the way to go?] 5) Other - Are we interested in NOAA’s Data Transport Library (DTL) - Anne Ball will discuss this when we next have a conference call.

UMass Seminar Presentation October 7, 2009 Biological and Chemical Oceanography Data Management Office BCO-DMO NSF funded 3 year project to provide short and medium term data management, including web based access, to all NSF funded projects from the biological and chemical oceanographic programs Large NSF projects are expected to have their own data management offices – a person Web site:

UMass Seminar Presentation October 7, 2009 MapServer interface and interoperability enhancements Provides access to geo-referenced scientific data and metadata Presents distributed data sets in a unified way Uses MapServer as the visualization applicationMapServer Visualize data with graphics generated on-the-fly Request custom subsets of data in a variety of file formats – flat file, Matlab, netCDF, WFS. Compare data from different sources

UMass Seminar Presentation October 7, 2009 JGOFS/GLOBEC Data Management System

UMass Seminar Presentation October 7,

UMass Seminar Presentation October 7, 2009 Cruise Tracks

UMass Seminar Presentation October 7, 2009 Select 5 Cruises

UMass Seminar Presentation October 7, 2009 Click on “Show Data” Button

UMass Seminar Presentation October 7, 2009 Select CD data in EN307

UMass Seminar Presentation October 7, 2009 Shows stations

UMass Seminar Presentation October 7, 2009 EN307 graph it options

UMass Seminar Presentation October 7, 2009 Depth versus salinity and versus temperature

UMass Seminar Presentation October 7, 2009 Select another cruise: AL9906

UMass Seminar Presentation October 7, 2009 Map it options for abundances

UMass Seminar Presentation October 7, 2009 Graph it option for AL9906

UMass Seminar Presentation October 7, 2009 AL9906 Nutrient/Phytoplankton Plot

UMass Seminar Presentation October 7, 2009 Interoperability features (for free)

UMass Seminar Presentation October 7, 2009 MapServer Supports Interoperability Features Open Geospatial Consortium standards –Web Mapping Service (WMS), and –Show me the data –Web Feature Service (WFS) –Get me the data Retains the functionality of the JGOFS/GLOBEC Data Management System –Download data as ASCII, CSV, Matlab, netCDF Will be adding Google Earth output file option

UMass Seminar Presentation October 7, 2009 Related Activities MMI – Marine Metadata Interoperability –“Promoting the exchange, integration and use of marine data through enhanced data publishing, discovery, documentation and accessibility." UNOLS Subcommittee to Report on Best Practices for the Collection of Data and Metadata at Sea to Promote Public DisseminationUNOLS –Too new to even have its own web site The Working Group on Zooplankton Ecology (WGZE), with guidance from the Working Group on Marine Data Management (WGMDM), is providing these general metadata guidelines for plankton data collected and submitted to ICES. (2003) Sensor Interoperability Metadata Workshop (2006) ICES ASC 2006 Theme session M "Environmental and fisheries data management, access, and integration" NOAA Coastal Services Center Data Transport Laboratory (DTL) –Integrated Ocean Observing System (IOOS) –Ocean.US data management and communications (DMAC) strategy Etc. NEEDS UPDATING

UMass Seminar Presentation October 7, 2009 Metadata Schema The print size is small to protect the innocent and guilty.

UMass Seminar Presentation October 7, 2009 What is the difference between an engineer and a mathematician?

UMass Seminar Presentation October 7, 2009

UMass Seminar Presentation October 7, 2009 NERACOOS Evan Richert (chair), Philip Bogden (GoMOOS), Janet Cambell (UNH), David Mountain, Neal Pettigrew (UMaine), John Trowbridge (WHOI), Robert Weller (WHOI) Purpose: “… formation of a Regional Association (RA) for the Northeast region “ Advisory Committee created (20 members) and others to address governance issues, etc.