LaMNA Incoming Dataset Processing Flow.

Slides:



Advertisements
Similar presentations
SDMX in the Vietnam Ministry of Planning and Investment - A Data Model to Manage Metadata and Data ETV2 Component 5 – Facilitating better decision-making.
Advertisements

Indian Institute of Remote Sensing Indian Space Research Organisation Dehradun Challenges in Capacity Building in Remote Sensing & GIS P. S. Roy
Emission Inventory System Reports Course Sally Dombrowski
The North American Carbon Program Google Earth Collection Peter C. Griffith, NACP Coordinator; Lisa E. Wilcox; Amy L. Morrell, NACP Web Group Organization:
Publish or perish? Linking Scratchpads and the new Biodiversity Data Journal for streamlining publication of botanical data D.N Koureas 1, L. Penev 2 &
The state of the art: DHS and MICS
Arctic Observing Viewer a web mapping application for AON data collection sites
Common Ground Project San Diego Bay Watershed Monitoring ( Multimedia GIS (
16 months…. The Visibility Information Exchange Web System is a database system and set of online tools originally designed to support the Regional Haze.
Depositing and Disseminating Digital Resources Alan Morrison Collections Manager AHDS Subject Centre for Literature, Linguistics and Languages.
How to use Student Voice Training Session Gary Ratcliff, AVC - Student Life.
Broadcast service Core tools. Agenda 1.Introduction – tool and its main features 2.Setting up and sending a simple broadcast 3.Achieving.
Rebecca Boger Earth and Environmental Sciences Brooklyn College.
Thinking about GIS applications and your projects.
THE DATA CITATION INDEX AN INNOVATIVE SOLUTION TO EASE THE DISCOVERY, USE AND ATTRIBUTION OF RESEARCH DATA MEGAN FORCE 22 FEBRUARY 2014.
IWC Database Overview of technology and application 13 th July 2010.
What’s Important Is Information … and We Have Specimens, Too! Neftali Camacho and Darolyn Striley Natural History Museum of Los Angeles County We use databases.
© Paradigm Publishing, Inc. 5-1 Chapter 5 Application Software Chapter 5 Application Software.
University of Waterloo Library : Using GIS for Managing Information Using GIS for Managing Information : Keeping Track of your Records Eva Dodsworth, Geospatial.
VOCABULARIES A data management presentation. Data management best practices Inventory of resources/datasets – Database level or series of datasets/collections.
AON Data Questionnaire Results 21 Respondents Last Updated 27 March 2007 First AON PI Meeting Scot Loehrer, Jim Moore.
Peter H. Wiebe and Nancy Copley Woods Hole Oceanographic Institution How does CMarZ Work? CMarZ Information System / Database /OBIS/ Species Pages.
1 Martyn Jessop, King’s College London.
Web services at TRFIC TRFIC has developed the Access Technologies to achieve its goals of interoperability and provide access to data and information on.
Bonanza Creek LTER Information Management Information Management at BNZ Information Management Data & Metadata Website & Communication Education.
Metadata and Geographical Information Systems Adrian Moss KINDS project, Manchester Metropolitan University, UK
JCOMM in situ Observing Platform Support Centre JCOMMOPS
The Value of Geospatial Metadata Metadata has tremendous value to Individuals within your organization, as well as to individuals outside of your organization.
1 Integrated Services Program The Virginia Metadata Training Workshop Summer, 2006 Lyle Hornbaker Integrated Services Program
Preserving Digital Culture: Tools & Strategies for Building Web Archives : Tools and Strategies for Building Web Archives Internet Librarian 2009 Tracy.
Relational Databases Melton, Beth “Databases: Access Terminology and Relational Database Concepts.” 09/LPMArticle.asp?ID=73http://pubs.logicalexpressions.com/Pub00.
Why Archiving and Preserving GIS Data Is Important Maps tell a compelling story of change over time. They document movement, progress, and change to the.
Data documentation and metadata for data archiving and sharing Managing research data well workshop London, 30 June 2009 Manchester, 1 July 2009.
EASI a free web database application for collecting and managing monitoring records.
SE Coastal Network Water Quality Inventory & Monitoring Program Database Development Wade Sheldon & John Carpenter Dept. of Marine Sciences University.
Data discovery and data processing for environmental research infrastructures Roberto Cossu ENVRI WP4 leader ESA.
Full and Open Access GEOSS: Are we there yet? Massimo Craglia, Elena Roglia, Alessandro Sorichetta European Commission Joint Research Centre
International Seminary on Digitisation: Experience and Technology 11 th May 2004 | National Library | Lisbon – Portugal DIGITAL ARCHIVE OF PORTUGUESE ART.
INTRODUCTION TO GEOGRAPHICAL INFORMATION SCIENCE RSG620 Week 1, Lecture 2 April 11, 2012 Department of RS and GISc Institute of Space Technology, Karachi.
ORNL DAAC MODIS Subsetting and Visualization tools Tools and services to access subsets of MODIS data Suresh K. Santhana Vannan National Aeronautics and.
Future Perspectives of Ocean Observatories in Germany [Name of the infrastructure / site / time series…] Contact person: [name, ] [Institution(s)
MOJN – June 2007 DM Presentation 1 Data Management; Yesterday and Today The Mojave Network from the Data Management Perspective –Data Management; Tabular.
UNIZULU INSTITUTIONAL REPOSITORY GATEWAY TO LOCAL CONTENT.
Recommendation 1 The IGS shall develop a standard protocol for exchanging information about IGS stations. The associated machine-readable database should.
November 16, 2009 Page 1 of 28 Data and Data Management: Introduction to the BCO-DMO Presented to Professor Keiichi Uchida November 16, 2009 Robert C.
September 2012 Developed by Agricultural and Biological Engineering Department at Purdue University and Department of Regional Infrastructures Engineering.
The Avian Knowledge Network and some of the lessons learned from the birding community Denis Lepage Senior Scientist.
Ontario Data Documentation, Extraction Service and Infrastructure.
Scratchpads and the new Biodiversity Data Journal Biodiversity Data Publishing made… easier Dimitris Koureas Natural History Museum London.
Hellenic Centre for Marine Research (HCMR) MedOBIS - Ocean Biogeographic Information System for the Eastern Mediterranean and Black Sea.
09/12/2003CCAQS Data Management System1 Central California Air Quality Studies Data Management System Status Report to the Study Agency Policy Committee.
U.S. Environmental Protection Agency Central Data Exchange Pilot Project Promoting Geospatial Data Exchange Between EPA and State Partners. April 25, 2007.
Find Research Data b2find.eudat.eu B2FIND User Training How to find data objects and collections using EUDAT’s B2FIND This work is licensed.
Sharing Digital Scores: Will the Open Archives Initiative Protocol for Metadata Harvesting Provide the Key? Constance Mayer, Harvard University Peter Munstedt,
11 Researcher practice in data management Margaret Henty.
Using the Exchange Network One State’s Perspective Deb Soule New Hampshire Department of Environmental Services NERACOOS/NECOSP Data Management Workshop,
Data Visualization as a Tool for Communicating Ocean Science Rob Bochenek Information Architect Axiom Consulting & Design.
STREAMLINING STORM WATER PERMIT IMPLEMENTATION THROUGH USE OF AN INFORMATION MANAGEMENT SYSTEM April 8, 2004 Presented by John W. Hart 30 th Environmental.
NVS New Zealand National Vegetation Survey. What is NVS? NVS (National Vegetation Survey) – New Zealand’s largest archive facility for plot-based vegetation.
Managing ITS Assets using GIS in Silicon-Valley City of San Jose, CA Kenneth Salvail, P.E. ITE District 6 Annual Meeting July 2007, Portland Oregon.
How to Create an Essential Metadata Record Using an Online Tool aka ‘ Now You Have No Excuse For Not Creating.
The Challenges of Digital Preservation in a Changing Environment Andrew Pitt Pfizer eArchive Service Team Global Records Management Services DPC Digital.
The Earth System Curator Metadata Infrastructure for Climate Modeling Rocky Dunlap Georgia Tech.
IAEA International Atomic Energy Agency Implementing SDMX for Energy Domain: From Discussion to Actual Implementation and Testing Andrii Gritsevskyi Oslo.
In Idaho Progress and Prospects Charles Swift
Northeast Coordinated Bird Monitoring
MERRA Data Access and Services
Five basic objectives of ISTM
Using the Checklist for SDMX Data Providers
Underground Injection Control
Presentation transcript:

LaMNA Incoming Dataset Processing Flow

Contents Justification What we do How we do it The challenges ahead How you can help

Contents Justification

New challenges... Global climate change means local phenomena affected by regional or continent-wide processes Management challenge: solutions that make sense at various scales (esp. regionally) Need datasets that span geographic AND time scales The Avian Knowledge Network: gather and compile avian datasets under a single format

New challenges... Typical avian monitoring data: presence/absence or counts......expect where species may go extinct......but we don’t know WHY populations are changing. Bird banding data include detailed metrics: sex, age, body condition, reproductive status Bird banding data provide information critical to UNDERSTANDING WHY bird populations will change

LaMNA joined the Avian Knowledge Network...we aim to compile, preserve and make widely accessible banding datasets through: Proper documentation (data about data, or “metadata”) Safe archiving Eased data accessibility and use (“use it or lose it”) - esp. for multi-scale analyses One of LaMNA’s goals LaMNA estimate: Very short half-life of banding data: 5-8 years (due to personnel changes, misplaced files, lack of interest, improper databasing methods) Loss amount: $5 million (10% of annual monitoring budget) WE ARE LOSING POTENTIALLY VERY IMPORTANT DATA VERY RAPIDLY!

Contents What we do

LaMNA and the AKN LaMNA curates and “advertises” datasets via the AKN locally and globally Dataset LaMNA documents & curates AKN curates & advertises National Biodiversity Information Infrastructure Global Biodiversity Information Facility Data owner controls how data are shared and used Data sharing policy:

Contents How we do it

Data flow through LaMNA Cooperator contacts LaMNA with interest in documenting, preserving and advertising data LaMNA collects and maintains information about cooperator in “metadata” database Metadata documentation begins...

LaMNA metadata documentation Simple but thorough process: LaMNA requests all or sample dataset LaMNA experts review data contents LaMNA follows up with brief questionnaire Only data on cooperator and essential additional information requested

LaMNA metadata documentation Institution & contact information Station locations Years operated Data access and sharing level All correspondence tracked, permits, sampling protocols...

Cooperator sends full dataset in any format LaMNA data processing Dataset (dBase) Dataset (Excel) Dataset (hard copy) Dataset (Paradox) Dataset (Word) Dataset (FoxPro) Dataset (Simple text) LaMNA documents & curates Each “dataset” should comprise 4 data files: Banding data Effort data Location data Sampling protocol Banding: Species Sex Age... Effort: Station Date Bander... Location: Latitude Longitude Datum Sampling protocol: Narrative of methods

LaMNA data processing LaMNA tracks each file received

LaMNA data processing Effort data example: Humboldt Bay Bird Observatory Effort data example: Fort Hunter Leggett Environmental Location data in GIS database (display: Google Earth) Sample protocol text: Sekercioglu Las Cruces 2007 Protocol

LaMNA data processing LaMNA experts process datasets Output: single flat table holding cooperator data in AKN data exchange format (515 fields!) web accessible LaMNA database tools Single dataset in AKN’s data exchange format (515 fields!) Sampling protocol Location data Effort data Bird data Cooperator dataset

LaMNA data processing LaMNA developed tools to track progress of data processing

LaMNA data processing Follow status per cooperator & protocol

Contents The challenges ahead

LaMNA current status 4,800,000 6,700,000 5,500,000 7,300,000 4,900,000 “Banding universe”: ~790 stations run avg years avg. ~2,000 records/year

LaMNA current status Data Universe (30 x 10 6 banding records) Data files received: effort, birds, location curated “as is” (518,789 Bird records) Data reviewed by LaMNA (296,103 Bird records) Data processed: all data properly linked into single table “view” (273,815 Bird records) Data processed: formatted to fit AKN’s data exchange format (203,621 Bird records) Dataset fully documented, curated and posted in web-based access system (180,600 Bird records)

Contents How you can help

OUR GOAL: PROPERLY DOCUMENT PRESERVE AND MAKE AVAILABLE EVERY BANDING DATASET Our work has just begun... We need YOUR HELP Contact LaMNA to see how you can help: LaMNA current status Click here