IPWG Validation current status and future directions Chris Kidd Beth Ebert John Janowiak The University of Birmingham, Birmingham, UK Bureau of Meteorology, Melbourne, Australia CPC/NCEP/NWS/NOAA, USA 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

IPWG validation sites Addresses IPWG objectives: #2) Establish standards for validation and independent verification of precipitation measurements #3) Foster the exchange of data on intercomparisons of operational precipitation measurements from satellites Main regions (initiated 2002) are: Australia, United States, Europe and South America, with Japan, South Africa and others for selected periods Common graphical, descriptive and statistical analysis of daily, 0.25° resolution products. 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

GV-IPWG synergies CriteriaGV programIPWG Type of validationPriority on physical, also statistical Has focused on descriptive and statistical Source of validation dataArranged for and collected by principle investigators Doesn't request. IPWG participants free to contribute Source of observational dataSpecific satellite-based products participants provide products directly to validation groups Types of Validation dataGauge, radars and specialist instrumentation, diverse in specific locations Conventional gauge and/or radar networks, usually part of a national network Types of observational datasingle-sensor, instantaneous, full-resolution datasets Blended satellite sensor products, time/area averaged. After Turk & Arkin, BAMS 2008 Both approaches are complementary 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China GV=Ground Validation

Global precipitation data sets 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

IPWG Intercomparison regions Near real-time intercomparison of model & satellite estimates vs radar/gauge 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

Processing system Initial setup: Setting of dates Cleaning out old/decayed data Acquiring data: Searching existing data Listing missing data Creation of.netrc file ftp data sources Remapping of data: … to regional grid or 5 km PSG projection… Results generation: Statistical analysis Graphical output Web pages: Generate HTML files Copying to server

Processing checks foreach day (d 0 -d 0-31 ) d n =d n+1 set d 0 =today foreach day (d 0 -d 0-31 ) foreach product (p 1 -p n ) if (product for day) !exist add to.netrc file foreach datasource (s 0 -s n ) ftp datasource (4k) N Y foreach product & day remap to PSG using LUTs & standardise format standardise filename foreach product & day Generate statistics Generate plots foreach product & day generate HTML files

Processing checks foreach day (d 0 -d 0-31 ) d n =d n+1 set d 0 =today foreach day (d 0 -d 0-31 ) foreach product (p 1 -p n ) if (product for day) !exist add to.netrc file foreach datasource (s 0 -s n ) ftp datasource (4k) Y N foreach product & day remap to PSG using LUTs & standardise format standardise filename foreach product & day Generate statistics Generate plots foreach product & day generate HTML files Set up list of past dates/days Usually okay: sometimes needs tweaking Checks for a products results: Okay if no results, but not if bad data Prepares products into common format Usually okay… Generates outputs: Okay if there is rain… Generates raw HTML: Occasional issues with server FTP runs several times: 4K buffer limit on macros Automated systems they are NOT!

Validation products Both gauge and radar are used – depending upon the real-time accessibility to these data sets Australia: gauge analysis United States: gauge and radar analysis Europe: radar analysis South America: Gauge analysis 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China European radar data: European composite with different radar and calibration and QC strategies; nominal 5km 15-minute product; available within ~1 hour; errors include range effects and anaprop (shipping, etc) Gauge data: 09-09Z data (dated at end of period); ~6000 daily gauges with mean 10km spacing; available 1 month in arrears

Radar vs gauge data 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China Radar (daily integrated) Gauge data

Overall results 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China There is a clear seasonal dependence of satellite product performance (poorer in winter) Models overestimate rain area/extent Satellite products underestimate rain area/extent Significant day-to-day variations Surface contamination of satellite products noticeable in winter Performance dependent upon precipitation extent and intensity, spatial and temporal resolution of product – implications for high-resolution precipitation products

The future of IPWG validation regions 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China What are the key requirements in the future to make best of our own limited resources? What are the requirements of the user community? What are the requirements of the algorithm/product developers? What new sources of data are available? Should we go beyond daily regional comparisons (local- global, instantaneous-seasonal)?

The future of IPWG validation regions New sites: BALTEX region – limited number of algorithms (and need to read HDF5!); also Canada (limited coverage) High-latitude oceanic location (Iceland; Shetlands?) Broaden existing regions (e.g. Europe) New comparison techniques: high-resolution products will need to rely upon descriptive statistics implementation of fuzzy-logic statistical techniques New performance results: Seasonal and global comparisons 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

International framework Current European radar network – used in IPWG EUMETNET radar coverage 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China The future of IPWG validation regions

Latitude profiles: rainfall occurrence

Monthly and seasonal validation Monthly and seasonal diagnostic validation summaries 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

The future of IPWG validation regions Suggested analysis levels: real time analysis: 'correctness' of near real-time products: should these be easily available (cics?) 'historical' analysis: 'authorised' release of quality- controlled products high-resolution analysis: similar set-up to daily/0.25 degree products (including instantaneous) 'training' data sets: example data sets from a comprehensive range of precipitation products – common naming convention and format 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China

Personal recommendations Evaluation of combined techniques: individual component products for combined algorithm actual combination techniques Standardisation of precipitation product format simple 2D arrays with accompanying text files common format –variable size quantitative resolution – 0.1 mm/day (WMO standard) error fields can be accompanying files Naming conventions standardised 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China The IPWG gauntlet: …. to generate global 1km, 1 minute rainfall estimates….

Anagnostou & Hossain: Satellite error propagation in flood prediction Northern Italy: Basin scales from 116 to 1200 km 2, storm induced flash flood event (Nov 1996) Satellite products: Kidd, 1h/4km, and 3B42, 3h/25km: reference rainfall: radar at 1h/1km Ensembles generated based on SREM2D; Hydrological model simulations based on tRIBS distributed rainfall-runoff model Bacchiglione (1200 km 2 ) Posina (116 km 2 ) PMIR: 4km/30min 3B42RT: 1deg/3hr High resolution products do work!

Recommendations To establish guidelines for satellite precipitation product data: establish common criteria that can and will be used to promote long-term data sets Investigate statistical tests that are more relevant to the data sets we are dealing with, and that be applied over a range of spatial and temporal scales To promote near real-time high resolution (sub-daily, <0.25 degree) precipitation inter-comparisons 4 th International Precipitation Working Group (IPWG) Workshop October 2008 Beijing, China To widen our validation regions to provide globally representative (inter)comparisons: at daily scales, but also monthly and seasonal scales.