START-08/pre-HIPPO FIELD CATALOG AND DATA MANAGEMENT Steve Williams NCAR Earth Observing Laboratory (EOL) Boulder, Colorado START-08/pre-HIPPO Planning Workshop Boulder, Colorado 8-9 January 2008

Project Data Management Strategy  Early involvement in project planning  Close coordination with PIs to develop data management strategy (e.g., plan, policy, format, special collection and processing)  Consistent implementation of strategy for lifetime of project and beyond  Reliable and efficient archive and distribution system  Easy and efficient long-term access to datasets by broader community including stakeholders, educators, and students

Project Data Management Considerations Develop Data Management Plan Data Types Data Formats and Documentation Data Collection Real-time Data Requirements Data Quality Control Data Archival Data Distribution Coordination with other Programs

NAME Data Management Timeline Finalize Operations Plan Complete Operations Summary (WWW) Operational Data Staged to CODIAC On-line Catalog Operational (CODIAC) Final Specs from PIs Field Ops and Data Collection QC’d data from PIs Final Data Staged Finalize Data Management Plan OPS Planning Meeting Planning Meeting Preliminary data stagedIntegrated Data Staged Draft DM Plan 2004 Data Analysis Workshop

TYPICAL DATA POLICY SUMMARY All investigators must agree to promptly submit their data to the archive All data shall be provided to other Project Investigators upon request During the initial 1-year data analysis period, data may be provided to a third party only with the permission of the investigator(s) who collected the data All data will be considered public domain not more than 1-year following the end of the field phase Any use of the data will, at a minimum, include acknowledgment. Co-authorship TBD with the investigator(s) who collected the data

EOL FIELD CATALOG TOOL In-field tool to ingest and display operational and preliminary research data and project documentation for making real-time decisions and evaluating project progress Features: Daily Mission Reports Operations Summary Facility Status Reports Data Analysis Products Authoring Tools Web-based access

FIELD CATALOG SAMPLE PRODUCTS

UCAR Office of Programs University Corporation for Atmospheric Research

T-REX Field Catalog Statistics Reports/Summaries (Status, Mission, and Operations) 340 documents and 1555 image files (0.26 GB) Research Platform Products (Aircraft, Surface, Lidar, Upper Air) 124,150 image files (8.01 GB) Operational Products (Satellite, Surface, Radar, Upper Air) 85,357 image files (7.81 GB) Model Output Imagery (Analysis and Forecast Fields) 143,252 image files (10.22 GB) TOTALS: 354,654 Files (26.30 GB)

Daily reports HIAPER flight summary HIAPER status Ops plan of the day Weather summary Observational products (archive imagery in FC) Satellite products GOES East and West imagery (VIS, IR, WV) - hourly Radar products - hourly U.S. national radar mosaic Surface products North American surface analysis Operational model analyses and forecasts - 00Z and 12Z NCEP GFS 00Z and 12Z forecasts (graphics at 12-hour intervals out to 2 days) 200, 300, 500, 700 hPa PCPN, MSLP, 850 hPa Z and wind - PCPN, MSLP, 850 hPa T - MSLP and thickness Tropopause Height - START08 team will generate these figures Pressure - START08 team will generate these figures PV Selected PV surfaces - START08 team will generate these figures Cross sections of T, wind, PV, TROP - START08 team will generate these figures Research products WRF CLaMS PROPOSED START08 FIELD CATALOG PRODUCTS

NAME Data Flow Operational Data Ingest GTS Satellite Radar Model Special Research Data Ingest Aircraft Ships Profiler ISS NAME PI Data and Products NAME Online Field Catalog NAME Data Archive Center NCAR (EOL) SMEX-04 Data Center (NSIDC) Distributed Data Centers CICESE IMTA NASA/DAACs NASA/GSFC NASA/JPL NASA/LaRC NOAA/ETL NOAA/NCDC NOAA/NODC NOAA/NSSL NSIDC SMN Univ. of AZ Delayed Operational Data “Final” PI Research Data Catalog Information “Preliminary” Data Sets “Preliminary” Products