From Earth Science Observations to GI Coverages: Towards an harmonization framework for coverages Stefano Nativi Italian National Research Council (Institute of Methodologies for Environmental Analysis) and University of Florence Scientific Data Type and Feature modeling Rutherford Appleton Lab 7-9 Mar 07

Outline Rationale From events to datasets –An harmonization framework for the coverage domain netCDF Coverage implementation –Abstract, content and encoding models –ncML-Gml O&M Coverage implementation Scientific Feature Coverage implementation Future work Conclusions

Rationale Provide Information Society with an effective, NRT and easy-to-use fruition of multidimensional Earth Sciences datasets (e.g. 4/5-D) Geospatial datasets Acquisition and Encoding Knowledge Extraction and Harmonization Standard Models and Interfaces Explicit Semantic level / Interoperability level SOCIETY INFRASTRUCTURES, PLATFORMS and SYSTEMS

Over-simplified Worldviews To the Geographic Information community, the world is: features –A collection of features (e.g., roads, lakes, plots of land) with geographic footprints on the Earth (surface). features discrete objects shape/geometry –The features are discrete objects described by a set of characteristics such as a shape/geometry To the Earth Science community, the world is: observationsparameters continuous functions –A set of event observations described by parameters (e.g., pressure, temperature, wind speed) which vary as continuous functions in 3-dimensional space and time. parameters equations. –The behavior of the parameters in space and time is governed by a set of equations. [from Ben Domenico]

ES and GI Data Models Diverse semantic and portability levels Diverse attention to domain multi-dimensionality Diverse aggregation structures and Data types [from J. Caron]

Observ.s Vs. Features: Value-added Chaining (Event) Observation –estimate of value of a property for a single specimen/station/location –data-capture, with metadata concerning procedure, operator, etc Feature –object having geometry & values of several different properties 1.classified object –snapshot for transport geological map elements 2.object created by human activity –artefact of investigation borehole, mine, specimen [from S.Cox Information Standards for EON] Coverage –compilation of values of a single property across the domain of interest –data prepared for analysis/pattern detection

An Harmonization Framework A unified framework with: O&M model, GF Model and IGCD Model Harmonization –O&M classes implementations for the Coverage domain –Scientific feature implementations for the Coverage domain GI:coverage is a GI:feature type > OGC model <<General Feature Model>> ISO 19107,… > ISO O&M Coverages > CF- NetCDF Grid Model CF-NetCDF Coverage > CF- NetCDF Observation Model …. This allows to develop crosswalks towards well- accepted ES data models - Example: CF-netCDF community ES GI > ISO > CF- NetCDF Grid Model

multidimensional Observation dataset (e.g. 4/5D hypercube) N-Dimension Coordinate Systems ES Dataset content, explicit/semi-implicit/implicit Geometry, … … Scalar measured quantities IGCD CF- netCDF

2D+elev+time dataset 2D Spatial Coordinate System + elev + time Range set GI coverage content, <coordinateSystem Axis> explicit/implicit Geometry Spatial Reference System (SRS), <rectifiedGrid Domain>, IGCD CF- netCDF

2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) 2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) 2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) 2 Dimension Coordinate System Implicit/explicit Geometry Range set Spatial Reference System (SRS) The Mediation Process ES hyperspace dataset (3/4/5D) 2D + elev + time Coverages 2D+elev+time dataset 2D SCS + elev + time Implicit/explicit Geometry Range set Spatial Reference System (SRS) a Coverage … … N-Dimension Coordinate Systems explicit/semi-implicit/implicit Geometry Scalar measured quantities s s S S S S S IGCD CF- netCDF

The Implementation ES data model –netCDF –Extra metadata: CF conventions GI Coverage model – ISO 19123: DiscreteGridPointCoverage Harmonization implementation-style –Declarative style Mediation Markup Language Rule-based procedure IGCD CF- netCDF

CF-netCDF Model NetCDF data model was extended adding a set of conventions –One of the most popular convention is the Climate and Forecasting metadata convention (CF) –Introduce more specific semantic elements (i.e. metadata) required by different communities to fully describe their datasets netCDF Model

DiscreteGridPointCoverage

0…1 1…n Mapping Rules

Concept typeDefinitionNotes An observation is a function from a given multidimensional real domain (  d ) to a multidimensional real co-domain (  c ). Note: a netCDF variable is a special case of Observation (with domain in  d and c=1). d = {b 1, b 2, …, b n } A dataset is a set of observation data. Note: a netCDF file is a special case of Dataset. S: {  3, SCS} A Spatial Domain is  3 with a law from  3 to a location in the physical universe (Spatial Coordinate System). A 2D Spatial (Planar) Domain is the restriction of S to  2. c: {S, T}   n n   C = {c} A coverage is a function defined from a Spatio- Temporal Domain (e.g. Lat, Lon, Height, Time) to a multidimensional real co-domain (  n). Note: if a set of CF-netCDF coordinate variables is a Spatio-Temporal Domain, then CF-netCDF variables defined over the corresponding dimensions can be mapped to Coverages Domain and Functional Definitions Observation Data/ Observation b:  d   c d, c   B= {b} Dataset Spatial Domain Coverage

Concept typeDefinitionNotes g(b) =c g: B  C Given an observation data, the Observation to Coverage operator generates a coverage. s = {g 1, g 2, …, g n } A Dataset to Coverages operator consists of a set of Observation to Coverage operators. Hence, Given an dataset element, the Dataset to Coverages operator generates a set of coverage elements. (Another task is the metadata elements mapping from dataset to the whole set of coverages). p(c) = m p: C  M A Coverage Portrayal operator transforms a coverage to a map, by means of a combination of the following operations: –Domain restriction (to a certain Z 0 and T 0 ); –Co-domain restriction (to a scalar quantity). Domain and Functional Definitions Observation to Coverage Operator Dataset to Coverage Operator Coverage Portrayal Operator

Data model harmonization: Implementation style Abstract model level Hyperspatial Dataset Coverage/Feature netCDF + CF Content model level ISO Coverage Model GIS Information Community Earth Sciences Information Community Encoding level ncML GML Mapping rules IGCD CF- netCDF

Data model harmonization ISO Data Model GI Information Community Earth Sciences Information Community Information Society (e.g. Spatial Data Infrastructure) ncML-G ML ncML Encoding Model GML 3.x Encoding Model WCS 1.x Content Model WFS Content Model Data Models Mediation netCDF Data Model CF Metadata IGCD CF- netCDF

ncML-G ML Mediation Markup Language An extension of ncML (netCDF Markup Language) based on GML (Geography Markup Language) grammar IGCD CF- netCDF Content Feature of interest Content Models Encoding Models

Available Language specification and Tools The ncML-GML markup language implements the presented reconciliation model It is a Mediation Markup Language between ncML (netCDF Markup Language) and GML –An extension of ncML core schema, based on GML grammar NcML-GML version –based on GML N2G version 0.8 –Java API for ncML-GML ver WCS-G –WCS 1.0 server which supports ncML-GML/netCDF documents Subsetting (domain and range-set) –netCDF –ncML-GML Client for WCS-G –GI-go thick client IGCD CF- netCDF

An Harmonization Framework A unified framework with: O&M model, GF Model and IGCD Model Harmonization –O&M Coverage implementation for the Coverage domain –Scientific feature implementations as Coverage GI:coverage is a GI:feature type <<General Feature Model>> ISO 19107,… O&M Coverages > CF- NetCDF Grid Model CF-NetCDF Coverage > CF- NetCDF Observation Model …. This allows to implement and harmonize other well- accepted data models ES GI > OGC model > ISO 19123

From Event to Datasets (O&M) Instruments and sensors observe and measure properties of Feature-of- interests Observations and measurements generate datasets Datasets can be modeled/stored as either boundary or coverage data –It depends on the observed property variability over the Feature_of_interest domain O&M Model IGCD Model

CoverageObservation implementation proximateFeatureOf Interest 1 0..* > Un-UniformFeatureProperty un-uniformObservedProperty > FeatureDomainExtent Set CoverageObservation. featureOfInterest is the Feature Domain Extent CoverageDomain is a proximate FeatureOfInterest CoverageObservation. observedProperty is un-uniform over the Feature Domain Extent Observed Phenomenon is the Range-set of generated coverages. O&M Model IGCD Model

SamplingFeature implementation CV_featureOfInterest 1 CoverageSamplingFeature > FeatureDomainExtent ObservationProcedure tileShape: GM_Object [0..*] tileDistributionType: TypedDistr [0..*] tileResolution: TypedRes [0..*] CoverageSurveyProcedure 0..1 CV_surveyDetails observableShape [0..1] observableAxes: CS_Axis [1..*] observableFrequency: TM_int [1..*] Set [ * CV_relatedObservation In a coverage observation framework, to define a CoverageSampling Feature as a SamplingFeature subtype. CoverageSamplingF eature is a proximate Feature of interest FeatureDomainExtent class is the simplest way to implement the CoverageSampling Feature concept O&M Model IGCD Model

The CoverageSamplingFeature is characterized by the following fields: –the possible shapes of the observation coverage domain –the possible observation domain axes –the possible observation frequencies The CoverageSurveyProcedure characterizes the sampling observation procedure –described by the following fields: shape(s), domain distribution(s) and resolution(s) of coverage tiles worked out by observations. CV_featureOfInterest 1 CoverageSamplingFeature > FeatureDomainExtent ObservationProcedure tileShape: GM_Object [0..*] tileDistributionType: TypedDistr [0..*] tileResolution: TypedRes [0..*] CoverageSurveyProcedure 0..1 CV_surveyDetails observableShape [0..1] observableAxes: CS_Axis [1..*] observableFrequency: TM_int [1..*] Set [ * CV_relatedObservation SamplingFeature implementation O&M Model IGCD Model

ExtensiveSamplingFeatures implementations Most of extensive sampling features may be implemented as CoverageSamplingFeature sub-types. –Specific values of CoverageSurveyProcedure and CoverageSamplingFeature fields characterize the site types for observation sampling. Fields / ExtensiveSamplingF eature FeatureDomainExtent. Observable (domain) Axes CoverageSurveyProce dure. Tile Shape CoverageSurveyProc edure. Tile Distribution SceneSpace (2-3 D)quadrilateralRegular/Irregular SwathSpace (2D)ellipseRegular ProfileSpace (1D) and Vertical (i.e. pressure or density) quadrilateralIrregular Scanning RadarSpace (2D)circle sectorRegular Time Series (Profile Series) Space (1D) or Vertical (i.e. pressure or density) and Time (Actual or Forecast time) quadrilateralRegular/Irregular O&M Model IGCD Model

An Harmonization Framework A unified framework with: O&M model, GF Model and IGCD Model Harmonization –O&M Coverage implementation –Scientific feature implementations for the Coverage domain GI:coverage is a GI:feature type O&M Coverages > CF- NetCDF Grid Model CF-NetCDF Coverage > CF- NetCDF Observation Model …. This allows to implement and harmonize other well- accepted data models ES GI > OGC model <<General Feature Model>> ISO 19107,… > ISO 19123

Coverage is a Feature type Discrete coverage modeled as a logical complex feature –implicit geometry (derived from the coverage function concept) –a collections of tiles which cover/sample the coverage spatial&temporal domain Tiles may be modeled as simple features, characterized by explicit geometry. Location Properties Property 1Property 2 … Property m (x 1, y 1 )Value 1 1 Value 1 2 … Value 1 m (x 2, y 2 )Value 2 1 Value 2 2 … Value 2 m (x 3, y 3 )Value 3 1 Value 3 2 … Value 3 m (x n, y n )Value n 1 Value n 2 …Value n m GF Model IGCD Model

Scientific Feature types are ExtensiveSamplingFeatures (Coverage sub-types implem.) Scientific Feature types (Extensive Sampling Features) –Certain feature types are only associated with sampling, and have no significant function outside of their role in the observation process …. Feature types having this behaviour are similar across all application domains [O&M] Fields / Scientific Feature types FeatureDomainExtent. Observable (domain) Axes CoverageSurveyP rocedure. Tile Shape CoverageSurveyPro cedure. Tile Distribution GridSpace (2-3 D)quadrilateralRegular/Irregular SwathSpace (2D)ellipseRegular Ragged sectionSpace (1D) and Vertical (i.e. pressure or density) quadrilateralIrregular Scanning RadarSpace (2D)circle sectorRegular Time Series (Profile Series) Space (1D) or Vertical (i.e. pressure or density) and Time (Actual or Forecast time) quadrilateralRegular/Irregular GF Model IGCD Model

Scientific Feature types are ExtensiveSamplingFeatures (Coverage sub-types implem) ISO recognizes some coverage sub-types which can be easily mapped to Scientific feature types Coverage sub-types GF Model IGCD Model

Content Vs Container Features ScientificFeatures ContainerFeatures (ExtentSamplingFeatures) (Coverage sub-type implementations) In order to facilitate interoperability, coverage can be modeled as a general container feature (i.e. proximate Feature-of-interest) Observation values Observation Sampling/functions Encoding ModelAbstract Model Application Elements Behavior Application specific Application agnostic Arrays Proximate FOI (Coverage Subtypes) Application FOI Inference Rules Content Model Binary File XML elements Semantics level Datasets Metadata XML elements Container Content Impliciteness GF Model IGCD Model

Future Work To extent the harmonization model between netCDF and GI Coverage realms –O&M concepts –Scientific Features (Sampling Features) To encode it in XML –Encode and access netCDF datasets ncML-Gml CSML, O&M ML, other GML dialects?

Main Conclusions ES and GI data model interoperability is more and more important for Society’s applications The GI coverage concept seems to be a good solution to bridge GI and ES data models We need a unified framework to harmonize concepts coming from O&M, GF and IGCD models in the coverage domain To facilitate the mediation between ES and GI communities –A valuable case in point: the FES realm (CF-netCDF)

Back-up slides