Publication of a large-scale hydrologic data set using the SDSC SRB NPACI All Hands Meeting March 19, 2003 Edwin P. Maurer University of Washington Departments of Civil and Environmental Engineering and Atmospheric Sciences

Public Interest in Hydrologic Variability Droughts Floods Extreme Events Source: NOAA, Hydrologic Info. Ctr. $6-8 billion/yr $5 billion/yr

Prospects for Future Forecast for Future: Deluge or Drought “floods and droughts…will become increasingly common as the world grows warmer” – NYT, 8/28/2002 “effects…include increases in rainfall rates and increased susceptibility of semiarid regions to drought.” – U.S. Climate Action Report, 2002

Motivation for Development of Hydrologic Dataset 1)Large and growing costs imply potential societal benefit to predictability 2)Better observation and prediction of climate signals and their teleconnections to land areas increases predictive skill 3)Improved understanding of continental-scale hydrologic variability through data collection and modeling

Understanding the Land-Surface Water Budget EP W Q (Near Surface) Water Balance Equation Examine variability in water budget components Need long records of observations to define variability and predictability

Precipitation and Evaporation Observations Precipitation appears well defined, generally since 1948 U.S. Station density: 1 per 700 km 2 Ameriflux (flux towers) provides measurements of E, since mid 1990’s U.S. Station Density: 1 per 130,000 km 2

Snow Water and Soil Moisture Observations About 600 SNOTEL sites in western US Snow water content measured since 1977 Spatial coverage poor at continental scale Source: A. Robock, Rutgers U.

Runoff (Streamflow) Observations Streamflow in the U.S. measured at roughly 7,000 active gauging stations. Stations represent regulated flow conditions Streamflow is a spatially integrated quantity Source: U.S.G.S.

Need for Modeling  To derive W and E, use P (and T) to drive a hydrologic model  Reproduce observed Q  By water balance, E must be close  Physically-based land surface representation can provide information on variability Variability of water budget components cannot be determined with observations

Hydrologic Model VIC Model Features: Developed over 10 years Energy and water budget closure at each time step Multiple vegetation classes in each cell Sub-grid elevation band definition (for snow) Subgrid infiltration/runoff variability

Use of Existing GIS Products Land Cover/Vegetation: from the University of Maryland 1-km Global Land Cover product (derived from AVHRR) Soil parameters: derived from Penn State State STATSGO in the U.S., FAO global soil map elsewhere.

Production of the Hydrologic Dataset Using VIC model, simulation run for 50 years at 3-hour time step Input Time series of spatial data One terabyte of output archived

Derived Data Set Characteristics Variables include all water and energy budget components Long term spatial data set allows characterization of variability 50-years 3-hour time step 1/8 degree (~12 km) resolution 77,000 grid cells through domain

Use and Sharing of the Dataset The database allowed the in-house investigation of predictability of runoff, resulting in several journal articles. Currently updating dataset. Ongoing use at the University of Washington, using SRB as permanent storage site. This unique dataset is of interest to many others in Climate modeling, Forecasting, and Hydrologic Modeling communities.

Data Storage and Access Storage challenge: Each File (3-hourly) ~500MB (compressed)Each File (3-hourly) ~500MB (compressed) 19 variables x 51 years=969 files19 variables x 51 years=969 files  ~500GB space needed for 3-hourly files (+ 80GB for daily).  ~500GB space needed for 3-hourly files (+ 80GB for daily). netCDF format used (compression to 2-D)netCDF format used (compression to 2-D) Desired ftp-type storageDesired ftp-type storage Discovered (by word of mouth) that SDSC SRB could provide a storage system for: Large storage space for archiving data Permanent space allocation Accessible location for other users

Storing the Dataset Uploading files – first began investigating SDSC SRB in late Our local system limitations required postprocessing in 4 stages. PC-based software problematic (SRB Browser, later replaced by InQ), esp. for multiple file transfers and using PC/UNIX network interface. Installation and use of UNIX utilities worked better. Understanding “containers” “collections” etc. outside of “ftp” mentality – learning curve. Final uploading required assistance, generously provided by SDSC personnel. Subsequent updating has gone smoothly with minimal outside assistance.

Retrieving Data from the Collection As with uploading, early PC-based software problematic Later InQ better, but PC-based retrieval restricted usability (large file sizes, saving across networks) Browser-based mySRB good for examining archive but not functional for downloading files UNIX utilities allow scripting for accessing multiple variables and years. This is how the data is almost always used.

Geographic Distribution of Users On-line (voluntary) registration form records many users Since the publication of the journal article in November 2002, over 40 unique users have registered.

Precipitation/Land Cover Experiment Differing VIC soil moisture fields used to initialize a mesoscale atmospheric model over Florida Shows changes in precipitation due to soil moisture and vegetation (Marshall et al. 2002, Colorado State Univ.)

Regional Application of the dataset Paucity of land surface observations for validating weather prediction and climate models. VIC model output used as “pseudo-observations” This graphic from a study headed at UCSC/Scripps (Roads et al., 2002)

Use in Diagnosing Climate Model Bias Betts at al (2003) studied the annual cycle of the land surface water and energy balance over the Mississippi River basin. Assessed the systematic biases in the surface energy and water budgets of NASA-DAO atmospheric general circulation model and ECMWF reanalysis (ERA-40). Used the current dataset as benchmark measurements for assessing model biases.

Summary and Future Plans  The SDSC SRB has provided a valuable service to this project and the hydrologic community, allowing the permanent storage and sharing of our data set with other researchers.  As software improved and we gained experience in retrieving data, users from many disciplines have been able to successfully access the data.  Future data set development will benefit from this experience, to provide:  Better file formats for selective downloading  Smaller files for ease of use

THANK YOU!