G O D D A R D S P A C E F L I G H T C E N T E R 1 Global Precipitation Measurement (GPM) GV Data Exchange Protocol Mathew Schwaller GPM Formulation Project Ground Validation Manager

2Ground Validation Workshop — September 2005 Objectives for this Presentation Examine the assumptions and issues surrounding a data exchange protocol for international global precipitation measurement (GPM) ground validation Identify many problems associated with defining and maintaining a standard data exchange protocol Propose an “applications-based approach” to interoperability among international GPM ground validation sites

3Ground Validation Workshop — September 2005 Some Assumptions Assumptions Each GV site will have its own unique set of instrumentation optimized for its local users – Not possible or desirable to standardize site instrumentation Each GV site will have its own data system for archive and distribution – These data systems will be optimized for local users All GV sites make voluntary contributions of local data and information, with the goal of validating Global Precipitation Measurement instrumentation, algorithms and products – There is no central funding authority (definition of voluntary) – There is no central authority to manage voluntary contributions Issues There will be a relatively small number of applications that will utilize the data from an international consortium of GV sites – At present, there are no applications that fall into this category Is it possible for a band of volunteers to adopt a common data exchange format leading to global validation products?

4Ground Validation Workshop — September 2005 Graphic Illustration of the Assumptions Each site will have its own unique instrumentation Each site will have its own unique data system Only a few shared/global applications are anticipated Is it possible to adopt common data model, data exchange attributes, formats, protocols, and middleware for generating global validation products? instrumentation data system data exchange protocol middleware application(s) GV site-1 GV site-nGV site-2

5Ground Validation Workshop — September 2005 The Scaling Problem Coordination costs rise exponentially as partners are added Any n partners can form up to (2 n -1) possible subsets Specifying GV site attributes, and coordinating shared attributes among varying sites quickly becomes a logistical nightmare! The more successful you are (the larger the number of GV participants) the time and $$ spent on coordination rises exponentially If unchecked, data coordination costs could eat into the GV budget for instrumentation, measurement and analysis

6Ground Validation Workshop — September 2005 The Problem of Site Bias Each site will have its own set of instruments, measurement protocols, calibration procedures, analysis methods and data products – There is no way to coordinate or dictate instrumentation and operations commonality among all GV sites It should be possible, but may be very difficult, to agree on a common data product: there may be many interpretations and representations of “instantaneous rainfall” for example Even if common data products are found (e.g., reflectivity), it will be difficult to agree on a common method for measuring and reporting the errors associated with the product We need to assume that the measurement error of any data product will vary from one GV site to another, and that there will be (unknown) bias from site to site

7Ground Validation Workshop — September 2005 The Problem of Incentive There are a number of costs that each GV site must consider before entering into an international GV consortium – Data conversion costs : product content and format will likely vary from site to site – Data reliability costs: product measurement error and bias vary at each site, and they may not be well characterized – Utilization costs: no global applications at present that can use the data even if it were available There is certainly value in generating a globally consistent data set for precipitation validation, but… The value of participating in a GV consortium must be greater than the cost

8Ground Validation Workshop — September 2005 An Ideal Exchange Protocol Maximizes incentives for voluntary participation – Provides some value to each participant Minimizes cost for participation – maximizes re-use of existing resources at local sites Minimizes coordination costs Compensates for or otherwise quantifies within-site error and site-to-site bias

9Ground Validation Workshop — September 2005 Typical 3-Tier Architecture In many data system architectures, applications are separated from data sources by “middleware” Middleware strengths: – Common interface for applications programmers to access data objects and services – Resolution of location/access information about data objects and services – May provide workflow services for complex tasks Weaknesses: – Middleware needs to be designed, developed, tested, maintained – “Longer term, the project could still falter if the International Virtual Observatory middleware standards make it too expensive for institutions to prepare their survey data” applications middleware data sources

10Ground Validation Workshop — September 2005 Space Sciences Middleware Data Sources Applications Example from space sciences: the International Virtual Observatory

11Ground Validation Workshop — September 2005 Recommendation At this stage: focus on the applications, let the protocol follow the applications Define applications/algorithms for validating global precipitation that are interesting and useful Implement the application(s) at cooperating partner sites in the international GV consortium – Initially, this will require developing a custom interface for each site – As the number of global GV applications grows (beyond 2 or 3), work on a common data exchange protocol should be reconsidered data source-1 application-A data source-ndata source-2 Replicated application Custom interface for each data source application-A

12Ground Validation Workshop — September 2005 Example GV Application Matching ground and space-based (PR, DPR) data for statistical validation Algorithm resamples coincident ground and space- based radar observations (Bolen and Chandresekar, 2000; Liao et al., 2001) GPM DPR TRMM PR ground radar

13Ground Validation Workshop — September 2005 Prototype in Development Resampling prototype provides statistical comparison of ground and space radar reflectivity – Good agreement high in the storm (where PR/DPR attenuation effects are minimal) indicates good relative agreement between PR/DPR and the GV radars – Good agreement near the surface indicates that the PR/DPR attenuation-correction algorithms are working well – Can be extended to comparison of precipitation rate, DSD and other variables Prototype will focus on TRMM PR comparison with NOAA NSR-88D radars – Starting with one or two NSR-88D sites – Will evaluate the possibility of scaling the prototype to all 158 WRS- 88D radars – Will evaluate the possibility of scaling to other S-band radars from other US and international sites

14Ground Validation Workshop — September 2005 Conclusions Focus on global applications that can exploit GV data from an international consortium of providers – These applications will help define the requirements for a GV data protocol Once the applications requirements are understood a number of possible frameworks for data exchange may be considered, for example: – CEOS/GEOS/CEOP: focus on catalog services – Open Geographic Information System (OGIS): focus on web services – National/International Virtual Observatory (and others): focus on Grid storage and computing Whatever the framework, the GV data exchange protocol must address practical issues of scalability, site bias, and incentives for participation

15Ground Validation Workshop — September 2005