Archive Service Nikolay Malitsky October 2, 2013.

Slides:



Advertisements
Similar presentations
Support for Time in ITK Patrick Reynolds Patrick Cheng John Galeotti Arnaud Gelas.
Advertisements

C++ crash course Class 10 practice problems. Pointers The following function is trying to swap the contents of two variables. Why isnt it working? void.
V4 Status and Workshop Report CSS, DISCS, an V4 team.
Control System Studio (CSS) Data Access Layer (DAL) Kay Kasemir, Xihui Chen July 2009.
EPICS V4/areaDetector Integration
PvData,pvAccess,pvIOC,pvService Status EPICS Meeting June 2011 NSRRC,Hsinchu, Taiwan Marty Kraimer, Guobao Shen, and Matej Sekoranja.
Content-Based Image Retrieval
Programming Environment S. Awad, Ph.D. M. Corless, M.S.E.E. E.C.E. Department University of Michigan-Dearborn Introduction to Matlab: Cells and Structures.
PvData,pvAccess,javaIOC,pvService Status EPICS Meeting Aix-en-Provence, France Marty Kraimer, Guobao Shen, and Matej Sekoranja.
ORNL is managed by UT-Battelle for the US Department of Energy EPICS V4 for SNS Neutron Data Kay Kasemir Oct
The Future of NetCDF Russ Rew UCAR Unidata Program Center Acknowledgments: John Caron, Ed Hartnett, NASA’s Earth Science Technology Office, National Science.
DDS Integration Nikolay Malitsky.
Chapter 6 Structured Data Types Arrays Records. Copyright © 2007 Addison-Wesley. All rights reserved. 1–2 Definitions data type –collection of data objects.
ISBN Chapter 6 Data Types: Structured types.
CMPUT 101 Lab #6 October 29, :00 – 17:00. Array in C/C++ Array is a structure type variable. One dimension of array int: int num[3]; There are.
Department of Computer Science and Engineering, CUHK 1 Final Year Project 2003/2004 LYU0302 PVCAIS – Personal Video Conference Archives Indexing System.
Arrays Declare the Array of 100 elements 1.Integers: int[] integers = new int[100]; 2.Strings: String[] strings = new String[100]; 3.Doubles: double[]
Raghav Ayyamani. Copyright Ellis Horowitz, Why Another Data Warehousing System? Problem : Data, data and more data Several TBs of data everyday.
DØ Channel Archiver Tutorial V.Sirotenko, 4/4/2001.
Presented by, MySQL AB® & O’Reilly Media, Inc. Developing INFORMATION_SCHEMA Plugins Mark Leith Support Manager, Americas
From the ChannelArchiver to the Best Ever Archive Utility, Yet July 2009.
Jan Hatje, DESY CSS ITER March 2009: Alarm System, Authorization, Remote Management XFEL The European X-Ray Laser Project X-Ray Free-Electron.
History Server & API Christopher Larrieu Jefferson Laboratory.
Archive Engine for Large Data Sets Nikolay Malitsky EPICS Collaboration Meeting San Francisco, USA October 5, 2013.
1 High level view of HDF5 Data structures and library HDF Summit Boeing Seattle September 19, 2006.
Cloud Computing Other High-level parallel processing languages Keke Chen.
MapReduce – An overview Medha Atre (May 7, 2008) Dept of Computer Science Rensselaer Polytechnic Institute.
NERC DataGrid Vocabulary Server Access Vocabulary Workshop, RAL, February 25, 2009.
Department of Computer Science and Engineering, CUHK 1 Final Year Project 2003/2004 LYU0302 PVCAIS – Personal Video Conference Archives Indexing System.
Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.
C++ Basics Structure of a Program. C++ Source Code Plain text file Typical file extension .CPP Must compile the C++ source code without errors before.
Computational Methods of Scientific Programming Lecturers Thomas A Herring, Room A, Chris Hill, Room ,
The netCDF-4 data model and format Russ Rew, UCAR Unidata NetCDF Workshop 25 October 2012.
GUIDED BY- A.S.MODI MADE BY- 1. SHWETA ALWANI 2. PRIYANKA.
New Features in EPICS V4 Release 4.4 EPICS Meeting 2014, CEA, October 2014 Marty Kraimer, Matej Sekoranja.
Integrating netCDF and OPeNDAP (The DrNO Project) Dr. Dennis Heimbigner Unidata Go-ESSP Workshop Seattle, WA, Sept
MASAR Server & Application Guobao Shen Photon Sciences Department Brookhaven National Laboratory Collaboration Working Group Oct 02, 2013.
Lattice QCD Data Grid Middleware: status report M. Sato, CCS, University of Tsukuba ILDG6, May, 12, 2005.
Wir schaffen Wissen – heute für morgen Paul Scherrer Institut EPICS V4 Archiver Service and Matlab client Timo Korhonen.
Jan Hatje, DESY CSS GSI Feb. 2009: Alarm System, Authorization, Remote Management XFEL The European X-Ray Laser Project X-Ray Free-Electron.
Calibration API in JANA David Lawrence, Jlab July 17, 2007.
D.G.Muir May 2010 MAST’s Integrated Data Access Management system: IDAM An Overview: D.G.Muir May 2010 designed to address specific MAST data management.
Scope When we create variables and functions, they are limited in where they are visible and where they can be referenced For the most part, the identifiers.
Writing a Channel Access Client in EPICS Bob Dalesio, April 5, 2000.
1 BROOKHAVEN SCIENCE ASSOCIATES Control System Overview Bob Dalesio, Control Group HLA Review for NSLS-II Project April
ICALEPCS (Geneva, Switzerland, October, 2005) Visual DCT’s Latest Hits Jaka Bobnar Matej Sekoranja, Igor Verstovsek.
EPICS EPICS Limitations Bob Dalesio Marty Kraimer.
Data Types written by Gabriele Carcassi performed by Bob Dalesio based on ideas/requirements from Bob Dalesio, Kay Kasemir, Ralph Lange, James Roland,
EPICS Development for the ASKAP Design Enhancements Program ASTRONOMY AND SPACE SCIENCE Craig Haskins 18 th October 2015 EPICS User Meeting – Melbourne.
PvaPy: Python API for EPICS PV Access Siniša Veseli Scientific Software Engineering & Data Management Advanced Photon Source EPICS Meeting October 2015.
Client Data Types Gabriele Carcassi Oct Scope and requirements  Define client side data structures to represent data for control systems and.
Computer Programming for Engineers
Abstract metadata for multidimensional image data considered as functions B Gibaud 25/10/2007.
Struct s (7.4) Used as data aggregates for an entity can be different types of data e.g. for student id, name, GPA, address,... Similar to classes, but.
Extra Recitations Wednesday 19:40-22:30 FENS L055 (tomorrow!) Friday 13:40-16:30 FENS L063 Friday 17: :30 FENS L045 Friday 19:40-22:30 FENS G032.
P OINTERS A pointer is an address All data is stored in memory in some location that is indexed with an address Can refer to variables by name or by memory.
Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Function Templates 16.2.
The HDF Group Introduction to HDF5 Session 7 Datatypes 1 Copyright © 2010 The HDF Group. All Rights Reserved.
Optical Flow walk through Aidean Sharghi Spring 14.
1 BROOKHAVEN SCIENCE ASSOCIATES EPICS Version 4 – Normative V4 Team – presented by Bob Dalesio EPICS Meeting October 7, 2011.
ORNL is managed by UT-Battelle for the US Department of Energy ADnED Handling V4 Neutron Event Data Matt Pearson SNS ORNL 18 th -22 nd May 2015.
YANG Roque Gagliano.
Prototyping the Next EPICS Archiver
EPICS Version 4 Abstract:
Cleaning up the mess.
Proposed Changes of the DDS Dynamic Data interface based on the consideration of EPICS PVData and Google’s Protocol Buffers Nikolay Malitsky 1.
Theorems about LINEAR MAPPINGS.
Presented By: Kwangsung Oh
EPICS 7 Matej Sekoranyja, Marty Karimer, Michael Davidsaver, Ralph Lange, Andrew Johnson, Timo Korhonen, Heinz Junkes, Patrick Marschalik, Murali Shankar,
Presentation transcript:

Archive Service Nikolay Malitsky October 2, 2013

Content  XML-RPC interface  C++ ea4::pvrpc server  CSS plug-in  TS Frames model

XML-RPC Interface  archiver info: common information used in other commands, such as enumeration of alarm statuses and severities  archiver names: a list of archiver names and paths to the index files used by this data server  channel names: a list of channels matched to the input pattern  channel values: data of selected channels The XML-RPC interface consists of four commands and associated messages:

Archiver Info struct SevrInfo { int num; string sevr; int has_value; int txt_stat; }; struct GetInfoRequest { string command; }; struct GetInfoResult { int ver; string desc; vector how; vector stat; vector sevr; }; In Plain C++

Archiver Names struct ArchiveInfo { int key; string name; string path; }; struct GetArchivesRequest { string command; }; struct GetArchivesResult { vector archives; }; In Plain C++

Channel Names struct ChanInfo { string name; int start_sec; int start_nano; int end_sec; int end_nano; }; struct GetChannelsRequest { string command; int key; string pattern; }; struct GetChannelsResult { vector channels; }; In Plain C++

Channel Values Problem: heterogeneous array Solution: PVData-based dynamic structure of self-described members

C++ ea4::pvrpc server

CSS Plug-in

TS Frames Model - 1 of 2 TS Frames Timestamp Bin Metadata Frame array of Coord Metadata Position array of pixel_meta Coord Metadata frame meta 1 …* 1 array of position meta 1 …* struct TimeStamp { int64_t secPastEpoch; int32_t nsec; int32_t userTag; } ; struct CoordMetadata { double low; // display range double high; // string name; // string unit; // “counts” uint32_t type; // e.g. UInt16, Double }; struct BinMetadata { uint32_t size; // size of dimension double min; // interval range double max; // string label; // “x”, “y” string unit; // uint32_t type; // Double double* ticks; // null in the interval case }; timestamps frames positions The proposed generic structure of the sparse multi-dimensional array is defined after the “natural” experiment-oriented representation built from the combination of two datasets: time series of detector-specific frames and time series of the frame positions in the scan-specific multi-dimensional space (angle, energy, pressure, etc)

TS Frames Model – 2 of 2  consistent with all (22) NeXus Application Definitions  mapped into C++, Python, and HDF5  supported by EPICS 4 PVData and RPC  suggests the transaction-oriented interface: select and fetch