Where/how could we change the overall process of field project implementation to improve in our mission of answering key science questions? Are we open enough to new ideas from PIs? Are we paying enough attention to the needs during the analysis phase and do we have the appropriate balance between our support of current and future projects versus support of past projects? How are we doing with regard to our support of projects with a more global and/or long- term scope? Are we keeping up with logistical and staffing challenges, especially in the more complex projects? Are our communications and interactions with NSF and the PIs during the project planning phase efficient, clear and productive? How effectively are we managing interactions with other agencies during field projects as they increasingly partner with us? Are we tapping into the operational data sources (e.g. NEXRAD, model output, satellite, mesonets, soundings, etc.) effectively? Are our visualization and analysis tools used during field projects and beyond meeting the needs of the science community? How are we doing with regard to remote participation in the field phase? Is this important? How are we doing with regard to our data delivery in terms of both timeliness and quality? What can we do to increase student involvement in field projects in a meaningful way? WORKSHOP DISCUSSION QUESTIONS

NCAR Community Data Portal (CDP) Architecture disk NCAR MSS XML/THREDDS data catalogs XML/THREDDS data catalogs disk Community Data Portal CDP Data Node Data Search & Discovery Data DownloadData Aggregation & Subsetting Data Analysis & Visualization OAI Metadata Publishing OPeNDAP/GDS/LAS access

EOL PLATFORM FINAL DATA FORMATS AircraftNetCDF, GenPro RadarDORADE, NetCDF LidarBSCAN, NetCDF GAUS, DropsondeASCII, NetCDF ProfilerNetCDF ISSASCII, NetCDF ISFFNetCDF SatelliteTDF, NetCDF, HDF, McIDAS, 1-B

“EXTERNAL” SUPPORTING PROJECT FINAL DATA FORMATS Upper Air (Individual Networks and Composites) ASCII, NetCDF Surface (Individual Networks and Composites) ASCII, NetCDF AircraftNetCDF, Binary, ASCII Model OutputGRIB, Binary, NetCDF OceanographyASCII, Imagery RadarBinary, Imagery Field CatalogImagery, Text MapserverBinary

Composite Data Sets at NCAR/EOL A composite dataset is a collection (over some time period and region) of similar data (e.g. surface meteorological) from a variety of sources, put into a common format, and passed through a uniform quality control. Why does NCAR/EOL develop composites? - Provides data in a uniform format with QC. - Allows determination of network/site problems. - Useful for model applications. - Prevents duplication of effort.

Hourly Surface Meteorological Data Composite (2991 stations) 1-min sites (* 385) AWOS (+ 335) RAWS (* 220) MesoWest (+ 94) HPCN (o 138) RWIS (+ 279) GPSMET (o 153) CO CoAgMet (* 17) FL FAWN (+ 5) IA IEM (+ 88) IL ICN (o 19) IN PAAWS (* 7) KS GWMD5 (* 10) MI MAWN (o 33) MO CAWS (* 21) OH OARDC (o 11) OK ARS Micro (o 42) OK Mesonet (+ 119) TX LCRA (o 102) TX TNRCC (+ 47) West TX Meso (o 39) Texas ET (o 23) 15 Other Networks (o 804)

FORMATS Advantages of Common Data Formats and Common Metadata Data Discovery and Mining Data Portal and Catalog Connectivity Data Interoperability and Tools Data Manipulation Data Analysis and Visualization Data Integration

“ The beauty about standards is that there are so many to choose from ” - Anonymous

FORMATS Issues for Consideration Must be easy for data providers (with incentive to submit) Need additional documentation in addition to standard metadata (e.g. calibrations, processing, QA, etc) Must be “living” format for long-term retrieval Build on external supported development Interactive translators? CF Conventions? Substantial Effort and Coordination

FORMAT DISCUSSION NetCDF is not NetCDF! Work required to establish CF conventions to allow data integration Migrating data to existing supported formats when applicable (flexibility) Software development based on field campaign priority rather than long-term commitment Accelerate development of format standards such as radars and other types EOL should be proactive rather than reactive in working with the community re: format standards

WHY DATA STEWARDSHIP? “The formalization of accountability for the management of organizational data” Data is the legacy of a Field Project (Result of Millions of $$ and large effort spent!) Responsibility to funders and taxpayers! Provide long-term benefit for future generations of scientists Data needed for Climatological Research

DATA STEWARDSHIP & LONG- TERM ARCHIVE ISSUES How long should data be made available? What Data should be archived? Archive back-up (Remote site?) Data Media Migration Software Migration and Compatibility Need Complete Metadata Maintain Data Discovery Capabilities Strong Support and Effort = $$$ !

DATA STEWARDSHIP & LONG- TERM ARCHIVE DISCUSSION NSF should provide longer term archive funding based on community data needs and reality

RESULTS OF BAD OR NO DATA MANAGEMENT PLANNING