1. RECLAIM and ICOADS Scott Woodruff 1, Philip Brohan 2, Eric Freeman 3, Elizabeth Kent 4, Sandy Lubker 1, Clive Wilkinson 5, and Steve Worley 6 1) NOAA Earth System Research Laboratory, USA 2) Met Office Hadley Centre, UK 3) Sourcecorp and CDMP/NCDC, USA 4) National Oceanography Centre, Southampton, UK 5) CDMP/NCDC, USA; & CRU/Univ. of East Anglia, UK 6) National Center for Atmospheric Research, USA

2. 1) RECLAIM RECovery of Logbooks And International Marine data project Initiated in 2004, building on EU-funded CLIWOC (1750-1850) project (ES, NL, UK, and some FR) Website: http://icoads.noaa.gov/reclaim/http://icoads.noaa.gov/reclaim/ –populated with imaged US and UK publications –detailed UK data inventories, etc.

3. Recent Accomplishments Imaging by KNMI in 2006 of C19th Dutch logbooks; planned for future digitization by CDMP Jointly funded UK & CDMP effort to image (268K pp) and digitize (1.5M obs) selected UK Royal Navy (RN) Ship’s Logs for data sparse period (1938-47) ~WW II

4. Ongoing and Planned UK Projects Together with ACRE, The UK National Archives (TNA), other partners English East India Co. Logbooks (>1K; 1790-1834) at British Library; many recorded SLP and AT RN Ship’s Logs from ~WW I (1914-23; ~376K days of obs) H.O. “Remarks Books” (6K, 1800-1909), Met. Registers (1850-), early balloon data from ships, publications, etc. Many more logbooks (1669-) untapped at TNA and elsewhere

5. 2) ICOADS International Comprehensive Ocean- Atmosphere Data Set Original COADS project initiated in 1981 –Joint in US between NOAA (ESRL and NCDC) & NCAR Plus extensive international contributions such as: –DWD, JMA, KNMI, UK Met Office and National Oceanography Center, Southampton Website including data and metadata access: http://icoads.noaa.gov/ http://icoads.noaa.gov/ Formal links to JCOMM under consideration

6. Major update in progress: Release 2.5 (~1662-2000) red1662-1949 processed (red) – 1950-69 being checked 1970-2000 (approx.) by late 2008 Green Green spans periods of new sources for R2.5 R2.0

7. R2.5 (bars): increases in data density: 1800-1969 US MMJ Japan Kobe UK RN UK MDB R2.0

8. Evolving national (+ HSST) data mix R1 1985 R2.0 2002 R2.5 2008 Thompson et al. 2008, Nature: A large discontinuity in the mid-C20th in observed global- mean surface temp HSST data (abbrev. fmt.)

9. R2.5: Decadal changes in spatial density; focus on 1930s R2.5: Decadal changes in spatial density; focus on 1930s (red= +, grey= 0, blue= –) HSST deck 156: Previously undetected dups w/ deck 193 (due to reversed sign of gravity correction in HSST) removed – but no fix available for empty box UK MDB deck 215 (German data): additional previously undetected dups removed, or replacement by better quality data from DWD Dutch deck 193: empty box (data never received?)

10. Future Directions: a) Update Frequency Aim for monthly extensions –GTS plus some delayed-mode (DM) data –replace NCEP Real-time data Challenges: –NOAA funding pressures – larger role for NCDC Historical (CDR) updates not suitable for “operations” –VOS call sign masking Stemming from security and commercial concerns NCEP GTS Dec 2007: all ships masked –Mandated WMO BUFR transition could be disruptive to data quality and continuity

11. DM ship/buoy/oceanographic data through 1997/1997/1996 DM buoy data GTS WMO Pub. 47 metadata Recent data mixture complexity: platform types; plus DM v. GTS data > GTS drifter reports consolidated in DM Potential for artificial signals from DM vs. GTS data mixture changes R2.5

12. Future Directions: b) Improve Data Quality Control Existing ICOADS QC –Out-of-date algorithms and QC limits –Can be insufficiently responsive to genuine climate signals (“trimming” problem) Improvements could be tied into: –Enhanced internationalization (JCOMM): E.g., improved VOS data flow: TT-DMVOS –Proposed “Climate ICOADS” Project For bias-adjusted/valued-added products

13. R2.5 ad hoc QC improvement: Trimming limits (July) for RH: Used 1910-49 for pre-1910 data R2.5 ad hoc QC improvement: Trimming limits (July) for RH: Used 1910-49 for pre-1910 data (End years of the three original trimming data periods (1854-1909, 1910-49, and 1950-79) shown) QC gaps in other variables e.g. SLP and SST

14. DM buoy/ODAS data (e.g., Canadian): 1980-date Chinese/GODAR Ships: 1968-93; 424K Arctic Drift Stations: 1952-76 US Navy Hourlies: 1952-64; 3M (deck 117) US Lightship Data: 1891-1986; 430K (~2009) GMN Obs: 1886-1939 DWD Historical Archive: 1850-1939; 11.2M Arctic Norwegian Logbook Data: 1867-1912; 125K German Maury Collection: 1845-67; 544K English East India Co.: 1790-1834; 285K to <6.8M (~2009) c) Future Directions: Additional Data (Metadata) Rescue & Blending Green Green =digitized Yellow Yellow =partially Red Red =undigitized But costly format translations w/ careful validation required: International Maritime Met. Archive (IMMA) format: has extensibility & flexibility to preserve original data

15. Conclusions Much useful RECLAIM & ICOADS work remains, e.g.: –HSST replacement desirable (as feasible; e.g., Dutch not?) –negotiate exchange of more DWD historical marine archive –improve SST indicator information (e.g., 1969-73 US data) IMMA observations: critical foundation for all later work –intensive translation validations costly, but often worthwhile –pre-evaluation of data quality of newly available sources? –possible role for additional international cooperation (e.g., UK) CLIMAR-III (6-9 May, Gdynia, Poland) –3rd JCOMM Workshop on Advances in Marine Climatology –summary being finalized; WMO Bulletin article planned –recommendations relevant to RECLAIM, ICOADS, and ACRE