Www.efda-taskforce-itm.org The european ITM Task Force data structure F. Imbeaux.

Slides:



Advertisements
Similar presentations
Configuration management
Advertisements

EUFORIA FP7-INFRASTRUCTURES , Grant JRA4 Overview and plans M. Haefele, E. Sonnendrücker Euforia kick-off meeting 22 January 2008 Gothenburg.
Alternative Approach to Systems Analysis Structured analysis
SYSTEM PROGRAMMING & SYSTEM ADMINISTRATION
Programming Languages Marjan Sirjani 2 2. Language Design Issues Design to Run efficiently : early languages Easy to write correctly : new languages.
Snejina Lazarova Senior QA Engineer, Team Lead CRMTeam Dimo Mitev Senior QA Engineer, Team Lead SystemIntegrationTeam Telerik QA Academy SOAP-based Web.
2P13 Week 11. A+ Guide to Managing and Maintaining your PC, 6e2 RAID Controllers Redundant Array of Independent (or Inexpensive) Disks Level 0 -- Striped.
Dr Gordon Russell, Napier University Unit Data Dictionary 1 Data Dictionary Unit 5.3.
Objectives In this session, you will learn to:
COSC 120 Computer Programming
Chapter 2 Machine Language. Machine language The only language a computer can understand directly. Each type of computer has its own unique machine language.
DCS Architecture Bob Krzaczek. Key Design Requirement Distilled from the DCS Mission statement and the results of the Conceptual Design Review (June 1999):
Software Frameworks for Acquisition and Control European PhD – 2009 Horácio Fernandes.
Input Validation For Free Text Fields ADD Project Members: Hagar Offer & Ran Mor Academic Advisor: Dr Gera Weiss Technical Advisors: Raffi Lipkin & Nadav.
Application architectures
Chapter 2: Impact of Machine Architectures What is the Relationship Between Programs, Programming Languages, and Computers.
1 A Student Guide to Object- Orientated Development Chapter 9 Design.
Operating Systems Concepts 1. A Computer Model An operating system has to deal with the fact that a computer is made up of a CPU, random access memory.
Talend 5.4 Architecture Adam Pemble Talend Professional Services.
Chapter 2 Database System Concepts and Architecture
CASE Tools And Their Effect On Software Quality Peter Geddis – pxg07u.
MDC Open Information Model West Virginia University CS486 Presentation Feb 18, 2000 Lijian Liu (OIM:
CS102 Introduction to Computer Programming
A Free sample background from © 2001 By Default!Slide 1.NET Overview BY: Pinkesh Desai.
QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.
COMPUTER SOFTWARE Section 2 “System Software: Computer System Management ” CHAPTER 4 Lecture-6/ T. Nouf Almujally 1.
Zhonghua Qu and Ovidiu Daescu December 24, 2009 University of Texas at Dallas.
Copyright © 2012 Accenture All Rights Reserved.Copyright © 2012 Accenture All Rights Reserved. Accenture, its logo, and High Performance Delivered are.
4 - 1 Copyright © 2006, The McGraw-Hill Companies, Inc. All rights reserved.
ITEC224 Database Programming
Software Engineering 2003 Jyrki Nummenmaa 1 CASE Tools CASE = Computer-Aided Software Engineering A set of tools to (optimally) assist in each.
DCS Overview MCS/DCS Technical Interchange Meeting August, 2000.
Imperial College Tracker Slow Control & Monitoring.
Chapter 7: Database Systems Succeeding with Technology: Second Edition.
CS 390- Unix Programming Environment CS 390 Unix Programming Environment Topics to be covered: Distributed Computing Fundamentals.
Database structure for the European Integrated Tokamak Modelling Task Force F. Imbeaux On behalf of the Data Coordination Project.
QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.
HERA/LHC Workshop, MC Tools working group, HzTool, JetWeb and CEDAR Tools for validating and tuning MC models Ben Waugh, UCL Workshop on.
EARTH SCIENCE MARKUP LANGUAGE Why do you need it? How can it help you? INFORMATION TECHNOLOGY AND SYSTEMS CENTER UNIVERSITY OF ALABAMA IN HUNTSVILLE.
ITPA/IMAGE 7-10 May 2007 Software and Hardware Infrastructure for the ITM B.Guillerminet, on behalf of the ITM & ISIP teams (P Strand, F Imbeaux, G Huysmans,
Lecture2: Database Environment Prepared by L. Nouf Almujally & Aisha AlArfaj 1 Ref. Chapter2 College of Computer and Information Sciences - Information.
Eurostat Expression language (EL) in Eurostat SDMX - TWG Luxembourg, 5 Jun 2013 Adam Wroński.
 Three-Schema Architecture Three-Schema Architecture  Internal Level Internal Level  Conceptual Level Conceptual Level  External Level External Level.
Lecture2: Database Environment Prepared by L. Nouf Almujally 1 Ref. Chapter2 Lecture2.
CE Operating Systems Lecture 3 Overview of OS functions and structure.
4 - 1 Copyright © 2006, The McGraw-Hill Companies, Inc. All rights reserved. Computer Software Chapter 4.
4/5/2007Data handling and transfer in the LHCb experiment1 Data handling and transfer in the LHCb experiment RT NPSS Real Time 2007 FNAL - 4 th May 2007.
_______________________________________________________________CMAQ Libraries and Utilities ___________________________________________________Community.
TRANSP users meeting Agenda 1.TRANSP development in 2010/2011, user’s suggestions to facilitate/accelerate TRANSP submissions. 2.TRANSP work:
17 th October 2005CCP4 Database Meeting (York) CCP4(i)/BIOXHIT Database Project: Scope, Aims, Plans, Status and all that jazz Peter Briggs, Wanjuan Yang.
LHCb Software Week November 2003 Gennady Kuznetsov Production Manager Tools (New Architecture)
Chapter 2 Database System Concepts and Architecture Dr. Bernard Chen Ph.D. University of Central Arkansas.
May 2003National Coastal Data Development Center Brief Introduction Two components Data Exchange Infrastructure (DEI) Spatial Data Model (SDM) Together,
The EDGeS project receives Community research funding 1 Porting Applications to the EDGeS Infrastructure A comparison of the available methods, APIs, and.
SOAP-based Web Services Telerik Software Academy Software Quality Assurance.
Architecture View Models A model is a complete, simplified description of a system from a particular perspective or viewpoint. There is no single view.
J.P. Wellisch, CERN/EP/SFT SCRAM Information on SCRAM J.P. Wellisch, C. Williams, S. Ashby.
Plasma State Component. Software Status Current Version: 1.001; no recent updates. Implemented as 3 libraries layered over the NTCC library set: xplasma,
CHAPTER 1: Introduction to Computers and Programming CSEB113 PRINCIPLES of PROGRAMMING CSEB134 PROGRAMMING I by Badariah Solemon 1BS (May 2012)
3/6: Data Management, pt. 2 Refresh your memory Relational Data Model
With TANGO S. Poirier – Data management group.
Copyright (c) 2014 Pearson Education, Inc. Introduction to DBMS.
Level 1-2 Trigger Data Base development Current status and overview Myron Campbell, Alexei Varganov, Stephen Miller University of Michigan August 17, 2000.
The Database Project a starting work by Arnauld Albert, Cristiano Bozza.
(on behalf of the POOL team)
Operating System Interface between a user and the computer hardware
System Design.
Unit# 8: Introduction to Computer Programming
TRANSLATORS AND IDEs Key Revision Points.
Introduction to Computer Programming
Presentation transcript:

The european ITM Task Force data structure F. Imbeaux

Data structure : logic, tools Universal Access Layer (UAL) Data base organisation, storage, and catalogue system Outline

Key elements for the communication between different codes : –Definition of a common format –This common format should be generic enough : to be shared by all codes to apply to all machines / circumstances for a given physical problem Practical point of view : –Code i/o interface uses directly the data structure –Many modern languages (Fortran 90, C++, …) can use structured objects  the data structure design is language independent –The content of the structured object can be changed (e.g. add a new variable) without having to modify the i/o interface of the codes Conceptual point of view : –Object-oriented approach to Consistent Physical Objects (CPO) Why a data structure ?

Full description of a tokamak : physics quantities + subsystems characteristics + diagnostics measurements  Object oriented data structure : High degree of organisation : several subtrees corresponding to « Consistent Physical Objects » (avoid flat structures with long list of parameter names). Substructures correspond to Consistent Physical Object : –Subsystem : (e.g. a heating system, or a diagnostic) : will contain structured information on the hardware setup and the measured data by / related to this object. –Code results (e.g. a given plasma equilibrium, or the various source terms and fast particle distribution function from an RF code) : will contain structured information on the code parameters and the physics results. Programming Language flexibility : use of recent software technologies : database structure is defined using XML schemas Object oriented data structure

XML is a generic and standardised object-oriented language, quite convenient to describe structures XML files can also contain the actual data, but we do not use this possibility (ASCII format not convenient for large size numerical data) XML schemas are used to define the data structure (arborescence, type of the objects, …). User-friendly tools (XML editors) allow fast and easy design of the data structure. Small translations scripts allow an automated translation of the structure for various purposes : –Generate type definitions in various languages –Generate access routines to CPO in various languages –Generate documentation –Extract specific parts of the data structure (e.g. machine description) Use of XML schemas

Just below the top : subtrees representing Consistent Physical Objects : subsystems of the tokamak, or topical code results (equilibrium, MHD, RF, …) One general bookkeeping node (contains in particular the reference GPN) Set of reduced data summarising the main simulation parameters ("0D") for the data base catalogue Detailed data structure : TOP

Typical diagnostic structure Each subtree (CPO) has its own time array. Each subtree (CPO) has one bookkeeping structure. Detailed data structure : CPO diagnostic

Typical code structure Each subtree (CPO) has its own time array. Each subtree (CPO) has one bookkeeping structure. Stores code results and code parameters Detailed data structure : CPO code

A database entry is an instance of the data structure During simulations, the « plasma state component » is an instance of the data structure All data exchanged between codes are part of the data structure (CPOs) Access to the data structure is managed by the Universal Access Layer The data structure is a key central object

Library allowing to access to the data structure –From the database –From the plasma state component during the run Low level routines : Get/Put a single variable. Developed in C. User level routines : Get/Put a whole CPO, with time interpolation / resampling options. Developed for F90 and C++. Transport layer : Access to the data (knows about the storage method) Since the « user level » routines deal with CPO, they must adhere to the data structure  they are generated by XSL (XML) scripts from the schemas The Universal Access Layer

The data is presently stored on an MDS+ server –Widely used data access system in the fusion community –Interfaces already exist with many languages –Convenient for storing multi-dimensional arrays, no problem with large data size –Not really object oriented (arrays of objects not possible), slow for large number of data calls The XML schemas defining the data structure are used to build the MDS+ model tree (automated script) The data storage system may evolve in the future The data structure is in principle storage-independent : CPOs can be stored with different methods. They contain nodes describing the storage method, to be used by the Transport Layer to access data. Database storage

A database entry is defined by : –The tokamak name (MDS tree name) –The shot number –The version of the data / reference number of the simulation Use of the MDS+ shot number as a Generalised Pulse Number (GPN) A full tokamak simulator should be able to compute all possible experimental quantities  Unique data structure for experimental data and all kinds of simulations Guarantee the consistency of a given dataset with minimum bookkeeping effort  Each entry of the database corresponds to a unique consistent physics dataset Each new simulation or version of the experimental data creates a new entry Definition of database entries

Guarantee data consistency within one entry  each new simulation or version of the experimental data creates a new entry. Copying all data present in the structure would cost a lot of storage space. Only data that are modified are explicitly written in the « output » GPN The unmodified data can be tracked down using a signal referencing the « input » GPN. –This signal would be located at the top of the tree –Valid for all subtrees (subtrees of different origin not allowed, since it may violate data consistency)  simple and efficient bookkeeping A catalogue system, based on a relational database allows to keep track of all existing entries in the ITM DB, and their relation (input/output) Referencing system

Referencing system Exp. Data Ref : none Ref : Simulation # Ref : Simulation # Ref : Simulation # Ref : Simulation # Exp. Data Ref : none Guarantees data consistency Referencing system  recursive search, hidden from the user if he does not want to know about it

Data structure completed for IMP #1 (equilibrium). Design ongoing for IMP3 (integration – core and edge transport) and IMP5 (H&CD sources) MDS+ server set up, model tree adheres to the DS UAL : first prototype has been produced, being tested. Hope to deliver first user version in June (C++ and F90) Extend DS to other physics areas Data base catalogue and referencing system designed conceptually, tools must be developed. First test of the whole set DS + UAL : benchmarking equilibrium codes (IMP1 task) Present status and perspectives