Presentation is loading. Please wait.

Presentation is loading. Please wait.

The IMPEx Data Model and Protocol

Published byRonald Barrett Modified over 6 years ago

Similar presentations

Presentation on theme: "The IMPEx Data Model and Protocol"— Presentation transcript:

1 The IMPEx Data Model and Protocol
A common standard for the analysis of simulated and observational space plasma physics data T. Al-Ubaidi1, M. Khodachenko1, E. Kallio2, V. Génot3, I. Alexeev4, M. Scherf1, F. Topf1, S. Hess3 , R. Modolo3, M. Gangloff3, W. Schmidt2, N. André3, E. Belenkaya4 Space Research Institute, Austrian Academy of Sciences, Austria (IWF-OeAW), Finnish Meteorological Institute, Finland (FMI) Centre National de la Recherche Scientifique, France (CESR-CDPP; LATMOS)‏ Institute of Nuclear Physics, Moscow State University, Russia (SINP/MSU)‏ REA Grant agreement no 1

2 Goals, Objectives Scientific focus of IMPEx: plasma and magnetic environments of… Mercury (BepiColombo)‏ Venus (VenusExpress)‏ Earth (Cluster, Themis) Mars (MarsExpress)‏ Jupiter and Ganymede (Galileo, JGO)‏ Saturn and Titan (Cassini)‏ Comet 67P (Rosetta)‏ IMPEx enables Selection, downloading, visualization and analysis of data from observations and modelling runs Support in finding matching modelling runs and request of specific runs Superimposing modelling data with spacecraft measurements (visual)‏ Scientific tools and functionalities for the support of preparation and operation of space missions (virtual spacecraft in modelled environment, 3DView)‏ REA Grant agreement no

3 IMPEx Resources Relation between the current set of models, tools and databases in IMPEx Simulation Models and Databases 3D hybrid & MHD platforms (FMI, Finland)‏ 3D hybrid code (CNRS/LATMOS)‏ Paraboloid Magnetospheric Model (SINP, Russia)‏ IMPEx enabled tools AMDA sci. data access (CNRS/IRAP-CDPP)‏ 3DView visualization CLWeb data analysis IMPEx Portal (under construction) REA Grant agreement no

4 ! Basic principles Mandatory features of the IMPEx infrastructure
Extendibility (easily adding of new models, databases and analysis tools)‏ In-line with worldwide trends and standards (e.g. IVOA) Generality of approaches and interoperability with existing tools Web-based applications in a service-oriented environment ! Can be seen as a prototype for a general infrastructure capable of operating a broad spectrum of data and models. REA Grant agreement no

5 The IMPEx Data Model A common metadata standard was developed for the description of simulation models and archived datasets. IMPEx Data Model is based on SPASE (xml) that originated from the Heliospheric community. SPASE was developed for the description of observational data Ideal to add ability to describe respective simulation data on top and use it as a unified Data Model in the context of IMPEx. REA Grant agreement no

6 The IMPEx Data Model First consistent data model to unify observational data and simulation outputs Based on SPASE Version 1.3 available since July 2015 Main components of the are: <SimulationRun/> Information about the parameters used for the run, refers to detailed model description <SimulationModel/> Information about the simulation model, tracks the version used <NumericalOutput/> Detailed information about a set of data files, refers to simulation run that generated the dataset. <Granule/> Basic information about a data file (defines order within a data set) - refers to the actual file and data set containing the numerical output Further extensions regarding e.g. large data sets are being discussed for the next update (v1.4, 2017?). IMPEx DM scheduled to be integrated into next major update of SPASE REA Grant agreement no

7 The IMPEx Data Model <spase/> www <SimulationModel/>
<Repository/> AccessURL/URL Model/ModelID www <SimulationRun/> RepositoryID Source/URL InputResourceID <Granule/> <NumericalOutput/> <Granule/> <NumericalOutput/> <Granule/> <NumericalOutput/> ResourceID <Granule/> <Granule/> REA Grant agreement no

8 IMPEx Protocol Central configuration file for access to methods and data The IMPEx configuration file (impexconf.xml) contains basic access information for every tool and every SMDB The configuration file is existing only once for the whole system, the caching policy is part of the definition (see Technical Documentation on website) Also includes the technical protocol to use for access and basic information about the service All data is accessible via the (static) tree.xml files. All data in the trees is stored compliant with IMPEx DM All available functionalities and methods are accessible as a web service as defined in the methods.wsdl file IMPEx configuration <xml> … </xml> Tool/SMDB tree.xml <xml> Tool/SMDB methods.wsdl REA Grant agreement no

9 IMPEx Protocol Common set of web services for SMDB and Tools access
A very challenging task – diverse software systems (SMDBs) had to be unified under a common umbrella, i.e. communication protocol There are 28 methods defined in total, 3 are shared between all SMDBs and constitute the core part getDataPointValue: Generic method to return data from 0D (given point) up to 3D (volume) getDataPointValueSpacecraft: Interpolates simulation values along a given (spacecraft) trajectory. getSurface: Generates a meshgrid and compute interpolation for one or several parameters. AMDA provides further methods for data exchange (i.e. virtual workspace). IMPEx Protocol FMI 3 4 3 SINP 12 LATMOS 1 AMDA 2 4 ClWeb REA Grant agreement no

10 IMPEx Protocol Usage of standard exchange formats and communication protocols The main exchange formats used by all methods defined in method.xml file are: VOTable - XML standard for the interchange of data represented as a set of tables NetCDF - software libraries and self-describing, machine-independent data formats On the client side IMPEx uses SAMP (Simple Application Messaging Protocol) for inter-tool communication AMDA and 3DView use e.g. SAMP to exchange commands and data REA Grant agreement no

11 IMPEx Topology & Architecture
IMPEx resources Database structures exposed via xml (IMPEx DM) Web Service interface (SOAP/REST)‏ IMPEx enabled tools Central administration via IMPEx configuration Central Access to SMDB’s via IMPEx Portal SAMP Hubs for client side communication‏ SMDB I SMDB II ObsArch xml xml IMPEx Config. SOAP SOAP SOAP www SOAP/HTTP REST IMPEx Portal SOAP/HTTP SOAP/HTTP SAMP User Access (Browser) IMPEx enabled tools (3DView, AMDA) REA Grant agreement no

12 IMPEx Webpage IMPEx Webpage http://impex-fp7.oeaw.ac.at
Developed by IWF-Team in Graz based on Typo3 CMS. Information about Technical Documentation Project activities and news Project meetings Publications and talks Project science news Collaborations with other projects Podcast REA Grant agreement no

13 The IMPEx Tool-Set 3DView - http://3dview.cesr.fr/
AMDA - HWA - CLWeb - SINP - Portal - REA Grant agreement no

14 Thank you! Questions, comments…? REA Grant agreement no

Download ppt "The IMPEx Data Model and Protocol"

Similar presentations

Heinrich Stamerjohanns Institute for Science Networking Distributed Open Archives Dr. Heinrich Stamerjohanns Institute for Science Networking at the University.

Heinrich Stamerjohanns Institute for Science Networking Distributed Open Archives Dr. Heinrich Stamerjohanns Institute for Science Networking at the University.
Remote Visualisation System (RVS) By: Anil Chandra.

Remote Visualisation System (RVS) By: Anil Chandra.
Snejina Lazarova Senior QA Engineer, Team Lead CRMTeam Dimo Mitev Senior QA Engineer, Team Lead SystemIntegrationTeam Telerik QA Academy SOAP-based Web.

Snejina Lazarova Senior QA Engineer, Team Lead CRMTeam Dimo Mitev Senior QA Engineer, Team Lead SystemIntegrationTeam Telerik QA Academy SOAP-based Web.
1 CEOS/WGISS20 – Kyiv – September 13, 2005 Paul Kopp SIPAD New Generation: Dominique Heulet CNES 18, Avenue E.Belin 31401 Toulouse Cedex 9 France

1 CEOS/WGISS20 – Kyiv – September 13, 2005 Paul Kopp SIPAD New Generation: Dominique Heulet CNES 18, Avenue E.Belin Toulouse Cedex 9 France
14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.

14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.
Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.

Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.
1 Introduction to XML. XML eXtensible implies that users define tag content Markup implies it is a coded document Language implies it is a metalanguage.

1 Introduction to XML. XML eXtensible implies that users define tag content Markup implies it is a coded document Language implies it is a metalanguage.
Data Portal for the “Climate changes Spatial Planning” program Henk Klein Baltink (KNMI) Fred Bosveld (KNMI) Hans de Wolf (Dutch Space)

Data Portal for the “Climate changes Spatial Planning” program Henk Klein Baltink (KNMI) Fred Bosveld (KNMI) Hans de Wolf (Dutch Space)
G O B E Y O N D C O N V E N T I O N WORF: Developing DB2 UDB based Web Services on a Websphere Application Server Kris Van Thillo, ABIS Training & Consulting.

G O B E Y O N D C O N V E N T I O N WORF: Developing DB2 UDB based Web Services on a Websphere Application Server Kris Van Thillo, ABIS Training & Consulting.
Talend 5.4 Architecture Adam Pemble Talend Professional Services.

Talend 5.4 Architecture Adam Pemble Talend Professional Services.
Discussion and conclusion The OGC SOS describes a global standard for storing and recalling sensor data and the associated metadata. The standard covers.

Discussion and conclusion The OGC SOS describes a global standard for storing and recalling sensor data and the associated metadata. The standard covers.
HDMC VxO SPASE 10-12 June 2008 VO-oriented activities of the CDPP M. Gangloff, E. Budnik, M. Bouchemit C. Jacquey, V. Génot, N. André, B. Cecconi, B. Lavraud.

HDMC VxO SPASE June 2008 VO-oriented activities of the CDPP M. Gangloff, E. Budnik, M. Bouchemit C. Jacquey, V. Génot, N. André, B. Cecconi, B. Lavraud.
Österreichische Akademie der Wissenschaften (ÖAW) / Institut für Weltraumforschung (IWF), Graz, Austria, T +43/316/4120-400, iwf.oeaw.ac.atDownload:2013.

Österreichische Akademie der Wissenschaften (ÖAW) / Institut für Weltraumforschung (IWF), Graz, Austria, T +43/316/ , iwf.oeaw.ac.atDownload:2013.
EARTH SCIENCE MARKUP LANGUAGE “Define Once Use Anywhere” INFORMATION TECHNOLOGY AND SYSTEMS CENTER UNIVERSITY OF ALABAMA IN HUNTSVILLE.

EARTH SCIENCE MARKUP LANGUAGE “Define Once Use Anywhere” INFORMATION TECHNOLOGY AND SYSTEMS CENTER UNIVERSITY OF ALABAMA IN HUNTSVILLE.
AMDA and CLweb Two complementary web-based services for spacecraft data exploitation with help from LESIA and support from CNES and CNRS Benoit Lavraud,

AMDA and CLweb Two complementary web-based services for spacecraft data exploitation with help from LESIA and support from CNES and CNRS Benoit Lavraud,
CIS 375—Web App Dev II Microsoft’s.NET. 2 Introduction to.NET Steve Ballmer (January 2000): Steve Ballmer Delivering an Internet-based platform of Next.

CIS 375—Web App Dev II Microsoft’s.NET. 2 Introduction to.NET Steve Ballmer (January 2000): Steve Ballmer "Delivering an Internet-based platform of Next.
SITools Enhanced Use of Laboratory Services and Data Romain Conseil

SITools Enhanced Use of Laboratory Services and Data Romain Conseil
WP 9 (former Task 1b of WP 1): Data infrastructure Robert Huber UNI-HB Esonet 2nd all regions workshop, Paris 5.10.-7.10.2009.

WP 9 (former Task 1b of WP 1): Data infrastructure Robert Huber UNI-HB Esonet 2nd all regions workshop, Paris
Flexibility and user-friendliness of grid portals: the PROGRESS approach Michal Kosiedowski

Flexibility and user-friendliness of grid portals: the PROGRESS approach Michal Kosiedowski
Introduction to Apache OODT Yang Li Mar 9, 2012. What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.

Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.

Similar presentations

Ads by Google