Stocker - Page 1Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Evolution of a Measurement-Based Data Processing System for Precipitation Erich Franz Stocker NASA/GSFC Code Mary Cleave NASA Headquarters

Stocker - Page 2Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Topics Determining measurements Architectural characteristics of a measurement- based system Evolving TRMM mission data processing system to measurement based Precipitation Processing System (PPS) Architecture-based process in use Summary

Stocker - Page 3Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Establish of a Measurement NASA Earth Science is evolving –Measurement-based programs for mature measurements –Away from single point missions. NASA intends to evolve to measurements –The measurement has been made via remote sensing on previous missions –When a program has matured to the point that follow-on flights with a number of appropriate instruments have been identified. –Flights and instruments need not all be provided by NASA Measurement programs are warranted when NASA needs –A parameter or set of related parameters needed for the measurement have emerged as key to a climate research area with an NASA Earth Sciences research program –To support climate research with a seamless data set from mission to mission. –Consensus of the research community as to the importance of the parameter to that research The selection of parameters that constitute a climate data record (measurement) is being developed within NASA Earth Sciences by –No fixed one-size fits all process but focused on an identified research need –Science Divisions lead and –Managed by the science program managers –in conjunction with the science community

Stocker - Page 4Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Characteristics of Measurement Approach Evolution from a science team per mission to a science team per measurement –TRMM science team has become the Precipitation Measurement Missions (PMM) team –Funding and research announcements are focused on the single mission science team Programmatic direction from a single Program Scientist within NASA Hq Earth Sciences. Measurement based data processing systems may have evolved from a single mission system but support multiple missions (instruments) for processing the measurement –Build on existing relationship with the measurement science team –Maintain processing capabilities from one mission to next –Handle new missions, parameter data, or related instrument data as needed with minimum incremental cost for the processing

Stocker - Page 5Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Advantages of the Measurement Approach Ensures continuity of measurements and data products key to NASA Earth Science research focus Helps ensures that missions are focused on the science measurement needs rather than on the hardware being developed Places missions within the overall direction of the measurement science community Ensures that measurement based science data processing systems –Are under science umbrella rather than the computer science one –Have the science expertise to produce measurement products correctly, efficiently and with continuity –Provides infrastructure support from mission to mission to Keep costs low – through incremental expansion costs Focus on the needs of the measurement data collection and processing Provide measurement expertise and continuity Ensure that processing not so general that all focus on the measurement is lost Facilitates science research on the measurement even when no mission is immediately available.

Stocker - Page 6Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Precipitation Processing System Precipitation identified as ready for evolution to measurement –Many missions –Long history of producing the measurement –Consensus of the science community TRMM Data and Information System (TSDIS) designated as a prototype measurement based data processing system Basic requirements –Be able to handle data from a number of different sources –Be able to able to support TRMM, existing precipitation data products and the Global Precipitation Measurement mission –Be able to add or remove data streams (with a known incremental cost for additions) –Be able to transition the data processing system to another provider when desired (i.e. move processing ) –Provide seamless consistent data set of precipitation through many missions

Stocker - Page 7Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Single Point to Measurement Based

Stocker - Page 8Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 PPS Evolution Approach Totally architectural approach –Analysis –Requirements –Design –Implementation –Testing –Configuration Management Start with the TSDIS architecture and reuse as many architectural components –Identify all architectural components –Identify those that are fixed in single source approach –Determine all dependencies (language, design, and code) From this starting part evolve PPS by –Identifying the characteristics needed because of the given charge –Identify all the components required –Identify all the properties and constraints on processes, data flows, and connectors –Multiple architectural walkthroughs before first design/implementation and walkthroughs throughout the process

Stocker - Page 9Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Configurable Flexible Architecture Characteristics for Measurement Based GPM Req for Algorithms Simplified deletions Simplified modifications Simplified additions Run in different modes Ease of maintenance Expand functionality beyond initial evolution Adaptable to new and different situations/uses Usable in unanticipated manner Support running of special or injected code in plug-in mode Run on small Equipment Run on large equipment No hard software restrictions No breakage on scaling No lock-up on scaling Operations settable via data Data streams addable or changeable via data Process interactions settable via data DB and other tools settable via data Easily moved to different hardware platforms Easily moved to different operating system platforms Ability to use with different supporting COTS Minimum human interaction Recover automatically for predetermined anomalies Able to call for help Recover automatically from failure Portable Extensible Portable ExtensibleScalableRecoverable Scalable Configurable Portable Recoverable FlexibleExtensibleScalable

Stocker - Page 10Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Evolving for Measurement Based Processing Architecture Processing Infrastructure Algorithms Expertise

Stocker - Page 11Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 PPS Architecture Overview Project Specific Existing Core

Stocker - Page 12Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Some PPS Aspects Tied to the ongoing NASA REASON CAN community standards efforts –Support ESML (GHCC) –Support OpenDaPS (URI) –Data storage in HDF4 and HDF5 –Evaluating other standards as identified Working to ensure availability in generic GIS data formats Allow dynamic subsetting online (in mulitple formats) –Geographic –dataset parameter subsetting Flexible toolkit –Easily add formats (currently binary, HDF4, HDF5, NetCDF) –Add languages (currently C, C++, and F77) –Dynamically produce read/write routines for subsetted data Use of SOAP to provide access to special services (e.g. user code within PPS) Close working with other precipitation centers

Stocker - Page 13Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Examples Working with Others Working with CSU to prototype –Intercalibrated 1C radiometer data from multiple sensors –A appropriate logical format for the representation of such data (this also with a wider community using radiometer data Working with Univ of Utah and MSFC on precipitation features dataset Working with UAH/GHCC on prototyping aspects important for precipitation –XML based data ingest formats –ESML for HDF4 and F77 Working with several different groups on GIS versions of precipitation data Testing dynamic subsetting capabilities –To satisfy user requests on TSDIS –To satisfy study needs for GPM Driven by the needs of the precipitation community and the work done through REASON CAN

Stocker - Page 14Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Prototype Science Discipline Center

Stocker - Page 15Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 PPS/GPM Operations Concept