1. In-situ Data and obs4MIPs
Opportunities and Challenges
Jim Biard

2. Background The obs4MIPs goal
Make observational products more accessible for climate model intercomparisons
Initial focus on observed variables that are near-exact matches to model variables
Measurement type is found in list of requested CMIP5 model variables
Has standard CMIP5 frequency, grid, pressure levels, etc

3. Current Situation obs4MIPs submissions (from obs4MIPs master list):
>10 different institutions
43 complete (6 from NCEI-Asheville)
103 in work (6 from NCEI-Asheville)
Initial set primarily satellite retrievals
In-situ data becoming more prominent among newer proposed submissions

4. Where to next?
Bringing more observational data sets into the ESGF/obs4MIPs system opens up new opportunities for innovative science
Such an effort can help foster the goal of greater interaction between the modeling, observation, and re-analysis communities
There are a number of challenges to be faced as we move forward

5. Examples from NCEI
NCEI holdings include numerous observational datasets that are valuable for climate change research
Not all of them are easy to fit into the current obs4MIPs framework
Two examples:
GridSat-B1
GHCN-M Station Data

6. GridSat-B1
Gridded Satellite Data from ISCCP B1 Infrared Channel Brightness Temperature
Regular lat/lon grid with 0.25° monthly resolution (0.07° 3-hourly resolution available)
The brightness temperature measurement is not a “requested” variable, although modelers are interested in using it
Requires an addition to the standard output table

7. GridSat-B1 Questions
How should observations be handled that aren’t in the standard output tables?
How should observations be handled that aren’t directly connected to model variables?
Do data sets with significantly different spatial resolutions pose a problem?
How to update ongoing data sets?

8. GHCN-M Station Data
Global Historical Climate Network - Monthly Station Data
Corrected and uncorrected observations of monthly mean, max, and min temperatures from 1701 to present for 7280 unique stations
Number of stations varies from year to year
Requires extensive accompanying metadata for the observations to be interpreted properly

9. GHCN-M Station Data Further details regarding station data
Collections of individual point measurements
Positions may change over time
Instrument, method, and quality at a location changes over time
Environment of location changes over time
Capturing this information requires auxiliary data (metadata) that may change slowly

10. GHCN-M Station Data Questions
Should station data be included in obs4MIPs?
What file form/structure should be used for station data?
Should support for intercomparison of station data with grids be provided?

11. Possible Solutions
How should observations be handled that aren’t in the standard output tables?
Don’t allow them
Case-by-case basis handling by the obs4MIPs team
Producer-driven additions to the tables
Community-driven model (e.g. CF standard names)

12. Possible Solutions
How should observations be handled that aren’t directly connected to model variables?
Leave this to be resolved by consumers
Require/provide “forward transform” modules to produce observation estimates from model outputs

13. Possible Solutions
Do data sets with significantly different spatial resolutions pose a problem? (Storage issues?)
Leave this to be resolved by consumers
Provide or require regridder modules

14. Possible Solutions
Should support for intercomparison of station data with grids be provided?
Provide/require station-to-grid and/or grid-to-station transform modules

15. Possible Solutions
What file form/structure should be used for station data?
netCDF Climate & Forecast (CF) Metadata Conventions have station and trajectory data forms
No widespread consensus on how to store the metadata, but efforts are underway
The CF community is open to influence

16. Fin

17. LAI CDR Leaf Area Index Climate Data Record
Regular lat/lon grid with 0.05° daily resolution
Current period of record will consume ~70 GB (with netCDF-4 compression)
Incomplete coverage due to clouds, etc
Needs to be accompanied by its QA flags variable (flags for clouds, surface type, etc) for the observations to be interpreted properly

18. LAI CDR Questions
Do data sets with significantly different spatial resolutions pose a problem?
Do higher volume data sets pose a problem?
How well will missing data be handled?
How can needed auxiliary variables be included?
The data set is in ongoing production for the foreseeable future. How should updates be handled?

19. Possible solutions
Broaden the obs4MIPs auxiliary data rules to allow data beyond number of observations, standard deviation, and standard error

20. Possible solutions How to update ongoing data sets?
Each file but the first and last should contain the same number of time intervals
One approach is to keep extending the last file until it is full, then start a new file
How should the metadata for the extended file be handled? (uuid, dates and versions, etc)