1. Eric Freeman2, Catherine Marzin3,
What are the data needs for ACRE-facilitated historical reanalysis and reconstructions? Part I: Marine Data and Metadata
Scott Woodruff1,
Eric Freeman2, Catherine Marzin3,
Clive Wilkinson4, Steve Worley5 1) NOAA Earth System Research Laboratory, USA
2) Sourcecorp & CDMP/NCDC, USA
3) NOAA National Marine Sanctuaries Program, USA
4) CRU/Univ. of East Anglia, UK & CDMP/NCDC, USA
5) National Center for Atmospheric Research (NCAR), USA
ACRE Data and Visualization Meeting
15-17 September 2009, UK Met Office, Exeter

2. Topics Effective measures for data rescue/archival
RECLAIM
International Comprehensive Ocean-Atmosphere Data Set (ICOADS)
Release 2.5: recently completed major update
Release 2.6: proposed by Oct (reanalysis goal)
RECovery of Logbooks and International Marine data project (RECLAIM)
Effective measures for data rescue/archival
Processing “pipelining” (proposed)
International Maritime Met. Archive (IMMA) format
Priority work areas for R2.6
Ongoing data rescue: a multidisciplinary example
a variety of additional ACRE-related projects discussed at this meeting

3. Evolution of ICOADS platform composition
buoy/ODAS
WMO logbook exchange

4. Release 2.5: 1662-2007 Completed in July 2009
Also: monthly GTS updates now operational 2007-forward
Woodruff, S.D., S.J. Worley, S.J. Lubker, Z. Ji, J.E. Freeman, D.I. Berry, P. Brohan, E.C. Kent, R.W. Reynolds, S.R. Smith, and C. Wilkinson, 2009: ICOADS Release 2.5: Extensions and Enhancements to the Surface Marine Meteorological Archive. Int. J. Climatol. (submitted).
R2.5
R2.4

5. Improvements in ocean area coverage
Avail.?

6. Early period (1792-1879): data mixture changes additions/deletions, separately by deck
ACRE
CLIWOC
DWD
Kobe
UK RN
DWD
US MMJ
HSST

7. Recent period (1937-2007): data mixture changes
Increases largely from blending higher quality delayed-mode Voluntary Observing Ship (VOS) data
R2.5

8. RECLAIM Project
Support from NOAA Climate Database Modernization Pgm (CDMP)
Closely linked with ACRE and other international projects
UK archive surveys (e.g. WW2, WW1)
Published data and literature inventory
Locating early platform and instrumental metadata
Wilkinson, C., S.D. Woodruff, P. Brohan, S. Claesson, E. Freeman, F. Koek, S.J. Lubker, C. Marzin, D. Wheeler, 2009: RECovery of Logbooks And International Marine Data: The RECLAIM Project. Int. J. Climatol. (submitted)
Woodruff, S., E. Freeman, C. Wilkinson, et al., 2009: ICOADS Marine Data Rescue: Status and Future CDMP Priorities (draft technical report available from

9. Proposed processing “pipelining”
As practical, initiate concurrent processing:
(b) prior to completing (a)
(c) prior to completing (b)
Can be helpful to explore data quality/characteristics in advance (e.g. dups)
Translation into common International Maritime Meteorological Archive (IMMA) format
(a) Imaging
(b) Digitizing (keying)
(c) IMMA translation

10. IMMA: flexible structure core + optional attachments
icoads
immt
meta
model
. . .
suppl
Emerging international standard (JCOMM)
Planned development of new attms for:
“historical” attm (e.g. Beaufort wind)
reanalysis feedback information
Contribution of data in IMMA format:
expedite blending into ICOADS
Key requirement: attm of original (“suppl.”) data
Experience demonstrates format translations frequently:
contain small (or large) errors
or are incomplete

11. Selected Data Rescue & Blending Candidates
Green =digitized
Yellow =partially
Red =undigitized
Note: all require translation
Including HISTOR

12. German Maury (1845-67) 850 logbooks loaned by DWD
Imaged and digitized by CDMP
~544K reports
Awaits translation into IMMA format

13. US Maury (1784-1863; 1.3M) SLP biases
Identified by Todd Mitchell (2002 “MARCDAT” workshop)
Report near completion by KNMI on US Maury data
not anticipated to offer a specific solution to this problem
More research work appears needed
early barometers characteristics/procedures not well understood
sympiesometers (or air barometers)
mercurial barometer
aneroid barometer

14. 11 Feb 1899 ~12Z Sea Level Pressure (SLP)
C20th Reanalysis:
colors = range of uncertainty
red dots = obs locations

15. Daily ship obs only for 1877-84
Bulletin of Simultaneous Observations also pubs. by: Deutsche Seewarte and Danske Meteorologiske Institut
Daily ship obs only for
land obs (covering complete period) proposed for future digitization
Imaging complete
Digitization
by late 2009
~150K global reports
Pipelining candidate

16. Greenwich Mean Noon (GMN) Obs 1910-47; Daily and Monthly Forms
Imaging complete
Digitization underway
Goal: late 2010, early 2011
2.6M global reports
Pipelining candidate
R.M.S. Laconia (1912) , courtesy of the Steamship Historical Society of America, Inc.

17. US Fish Commission Scientific Survey Logbooks
(imaged, to be digitized)
Date
Position
Depth
“Obtained; Sea Urchins, 1. Starfish, Rock Crabs, Hermit Crabs, and shells. A few skates and flounders a number of Pectens (scallops)”
Nature of Bottom
Air
Temperature
Sea Surface Temperature
Dredging Instrument
Bottom
Temperature
Atmospheric and Oceanographic Data
Species
Richness
Salinity
Barometer
Cloud Cover
Wind
Ecological Data 17

18. Conclusions Pipelining planned (imaging, keying, translation)
IMMA format: critical foundation for later work
translation support (e.g. CLIWOC and ACRE) very helpful
How to decide data rescue priorities?
how to quantify data rescue benefits (e.g. SST, reanalyses, ecology)?
DWD Historical Archive (~ ; incl. “HISTOR”)
additional blending likely very valuable for historical reanalyses
NOAA resources needed for ICOADS historical updates
more frequency desirable for many science applications
Plans for eventual “Climate ICOADS” initiative
bias adjustments -- value-added products

19. 19