1. Interoperable Information System of Systems for HTAP
Rudolf B. Husar and Rich Scheffe
With Erin Robinson
Presented at
HTAP Workshop, WMO, Geneva, January 26, 2006

2. Intercontinental Transport Events have
Influenced AQ Regulations ..and now Policy? –> Air Quality Management Web-Based Collab. in Virtual Workgroups –> Science, the Process Near-Real-Time Data Access & Integration –>Technologies and Tools
US-Daily Avg. PM25 Dust: Seasonal + Events

3. HTAP Characteristics
Stakeholders are autonomous countries, organizations
Diverse perspectives on the issues, solutions
Pollution sources and impacts are hemispheric
Information integration is key to success
Observations and models are ready for integration

4. The HTAP - GEOSS GEOSS is a good system model for HTAP
Stakeholders are autonomous countries, organizations
Diverse perspectives on the issues, solutions
Recognition of the need to understand, act
Pollution sources and impacts are hemispheric
Observations and models are ready for integration
Information integration is key to success
Hence
GEOSS is a good system model for HTAP
HTAP is a suitable use case for GEOSS
Some Characteristics of System of Systems (SoS)
Autonomous constituents managed independently
Independent evolution of each constituent
Displays emergent (novel, unpredictable) behavior

5. Summary
There is an ‘alignment of stars’ for integrating heterogeneous HTAP data
System of Systems architecture is suitable for integrating HTAP data
Standard data access is a key interoperability protocol
Heterogeneous data can be non-intrusively standardized by mediators
Service-based software architecture delivers tailored products to diverse uses
Federated data and shared web-based tools are in use
DataFed already includes over 100 datasets (emissions, ground, satellite)
The system has been applied to EPA regulatory development
However,
DataFed is just one of the many mediator nodes, but these need to be connected
Much more data would need to be federated
HTAP model-data comparison would be an attractive use case
WMO-GAW could be an effective neutral facilitator for global data integration using GEOSS (system of systems) principles

6. GEOSS Architecture Framework - DSS for HTAP
Models
HTAP Reports: Model Estimates, Obs. Evidence
Knowledge into the Minds of AQ Analysts
HTAP Knowledge into the Minds of Policy Makers
HTAP Knowledge into the Minds of Policy Analysts
Observations
Decisions
Decision Support System
The decision support system for hemispheric transport assessment performs both model to model comparison as well as model to observation comparison.
The reports of analyses are submitted to policy analysts in the hemispheric transport Task Force.
The policy makers decide whether and in what way hemispheric pollutant transport should be included in International Long-Range Transport Protocols.
GEOSS Architecture Framework

7. System of Systems: Interoperability Connecting Machines and People
Interoperability Stack: Key concept of the Web System components have to be interoperable at each layer
Amplify Individuals Connect Minds
Open Architecture Data Standards Service Based
IP – Internet Protocol

8. How do you agree on Interoperability?

9. Are WCS/WMS Space-Time-Parameter queries Convergence Protocols
What few things must be the same so that everything else can be different?
Are WCS/WMS Space-Time-Parameter queries
GetCapabilities
GetData
Capabilities, ‘Profile’
Data
Where? When? What? Which Format?
Server
Back End
Std. Interface
Client
Front End
WCS/WMS
App
Query
GetData
Standards
Where?
BBOX
OGC, ISO
When?
Time
What?
Temperature CF
Format
netCDF, HDF..
CF, EOS, OGC T1 T2
Convergence Protocols

10. Lets agree on Space-Time-Parameter Data Access Query Protocol

11. From ‘Stovepipe’ to Federated Architectures
Current info systems are project/program oriented and provide end-to-end solutions
Info Users
Data Providers
Info System
AIRNow
Public
Each project/program can be augmented by Federation data and services
Part of the data resources of any project can be shared for re-use through DataFed
Model
Compliance
Manager
Scientist
Science
DAACs
Data processing and rendering can then be performed through web services
Through the Federation, the data are homogenized into multi-dimensional cubes
Traditional air quality information systems provide end-to-end solutions through dedicated ‘Stovepipe’ processing streams
However, many formerly closed data systems are now opening up and they share their data – through DataFed or other mediators
In a federated approach data are homogenized and augmented with services and tools streams…
The federated resources in turn can be fed back into individual project processing streams
The transition from the stovepipe to the networked, service oriented architecture can be gradual and non-disruptive.

12. Web Services and Workflow for Loose Coupling
Obs. & Models
Decision Support System
Control
Std. Interface
Std. Interface
Data
Reports
Datasets
Data Views
Integrated Data
Reports
Web Service Interaction
Service Chaining & Workflow
Service
Broker
Emergent behavior in a network that is only created by the combination, not by the individual members.
Emergent behavior: novel and coherent (logically connected, purposeful, meaningful) structures, patterns and properties arising from self-organization (mashers are self-organizers) in complex systems Wikipedia
Emergent behavior is created by mashing
Mashing occurs when open on both ends of network.- diffusion
Input side – grow by assimilation
Output side – Harvest (i.e. DataFed harvests datasets)
Find
Publish
Bind
Service
User
Service
Provider

13. Collaborative Reporting and Dynamic Delivery
Obs. & Models
Decision Support System
Control
Std. Interface
Std. Interface
Data
Reports
Datasets
Data Views
Integrated Data
Reports
Analysis Reports:
Information supplied by many Needs continuous program feedback Report needs many authors Wiki technologies are for collaborative writing
Co Writing - Wiki
Dynamic Delivery:
Much of the content is dynamic Animated presentations are compelling Movies and screencasts are for dynamic delivery
Screencast

14. Model-Data Comparison Workflow Software
VIEWS Chemical Data
Ft. Collins, CO
Std I/O
Model- Data Comparison
Workflow
AeroCom
Chemical Models
Paris, FR
Std I/O
Workflow Flow Program Lego-like assembly of component
WMS, WCS OGC Services
READ T

15. Sample of Federated Datasets

16. HTAP Model-Obs “Test/Benchmark”ed
A virtual data center design to meet the needs of the TF HTAP model intercomparison and evaluation
Archives of key observations and model data
Tools for model intercomaprison

17. Tools.. Analyst Console User Layout

18. Imagine…More Shared Global Obs & Models
2007
Global Data & Models
2007++
More Global Data & Models

19. The Network Effect: Less Cost, More Benefits through Data Reuse
Programs ask/get Data
Orgs Develop Programs
Public sets up Orgs
Data
Data
Program
Data
Organization
Program
Data Re-Use Network Effect
Public
Program
Organization
Data
Data
Program
Data
Pay only once Richer content
Less Prog. Cost More Knowledge
Less Soc. Cost More Soc. Benefit
VIEWS
AeroCom
Federated Information System
Observations, Models, Services
CAPITA
Other Federations
Obs
Models
Obs & Services
Services
Obs & Model
Models & Services

20. Agile Information System Design Principles
Guiding Idea: System of Systems GEOSS
Domain of Interaction
WMO/GAW & Stakeholders
Infrastructure: Internet, Middleware, Web 2.0
Methods, Tools: Standards, Use Cases
Possible DataFed Roles:
Guiding Idea: Refine, solidify, evangelize the System of Systems idea
Methods and Tools: Develop, promote, implement, verify standards; Cooperatively develop use cases
Infrastructure: Maintain the DataFed middleware for distributed data access; Supply Agency champions with SoS ‘sales’ material
Based on P. Senge et. al, 1994: Architecture of Learning Organizations(Link)

21. Organizations have different missions
R. Scheffe
Apparent divergences?
Organizations have different missions
NPS
Protect ecosystems,
AQ WQ
NASA
Explore fundamental
Earth System Properties
USDA
Protect/optimize Ag
and forest resources
EPA (Protect human health & envi.
Improve air, water, ecosystem
Not necessarily. The various organizations have very different missions from protecting human health to understand Earth’s environment to the exploration of the Earth System. In the past, these different missions have lead to divergences.
CDC
To promote health and quality of life ….
NOAA
To understand and predict changes in the Earth’s environment and …

22. Stars aligned? Confluence again.. Alignment R. Scheffe
Observation technologies
{e.g., satellites}
Computational power
Collaboration (culture)
empire building?
Alignment
Science, talent
{embodied in AQ models
and young geniuses}
Budgets, agency collaboration
resource/program accountability
We know that we are in the midst of a sensing revolution, there are major improvements in data sharing technologies and that cultural changes are taking place toward increased collaboration. So, are the stars aligned to better integration?
Information technologies
{e.g., data sharing protocols}
Accountability, ↓ regulatory
assessments {e.g., NAS, CASAC}

23. Summary
There is an ‘alignment of stars’ for integrating heterogeneous HTAP data
System of Systems architecture is suitable for integrating HTAP data
Standard data access is a key interoperability protocol
Heterogeneous data can be non-intrusively standardized by mediators
Service-based software architecture delivers tailored products to diverse uses
Federated data and shared web-based tools are in use
DataFed already includes over 100 datasets (emissions, ground, satellite)
The system has been applied to EPA regulatory development
However,
DataFed is just one of the many mediator nodes, but these need to be connected
Much more data would need to be federated
HTAP model-data comparison would be an attractive use case
WMO-GAW could be an effective neutral facilitator for global data integration using GEOSS (system of systems) principles