STAR C OMPUTING Maker and I/O Model in STAR Victor Perevoztchikov.

Slides:



Advertisements
Similar presentations
Chapter 4: Trees Part II - AVL Tree
Advertisements

Database Systems A 1. 2  Project goal: to tackle and resolve real-life DB related development issues  So what do we need to do:  Design.
File Management Chapter 12. File Management A file is a named entity used to save results from a program or provide data to a program. Access control.
CS 211 Inheritance AAA.
Reconstruction and Analysis on Demand: A Success Story Christopher D. Jones Cornell University, USA.
Other time considerations Source: Simon Garrett Modifications by Evan Korth.
Aalborg Media Lab 23-Jun-15 Inheritance Lecture 10 Chapter 8.
B + -Trees (Part 1) Lecture 20 COMP171 Fall 2006.
B + -Trees (Part 1). Motivation AVL tree with N nodes is an excellent data structure for searching, indexing, etc. –The Big-Oh analysis shows most operations.
B + -Trees (Part 1) COMP171. Slide 2 Main and secondary memories  Secondary storage device is much, much slower than the main RAM  Pages and blocks.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 4: Implementing and Managing Group and Computer Accounts.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 4: Implementing and Managing Group and Computer Accounts.
Guide To UNIX Using Linux Third Edition
Binary Trees Chapter 6.
Composite Design Pattern. Motivation – Dynamic Structure.
Using ROOT geometry for FVTX reconstruction Zhengyun You Peking University Los Alamos National Lab Jan 22, 2007.
CLEO’s User Centric Data Access System Christopher D. Jones Cornell University.
Wolfgang Friebel, April AFS Administration Framework.
Click to edit Master title style 21 st Nov, 2003 Valeri Fine (BNL) ROOT and ROOT4STAR framework V. Fine or Software for “C++ speaking” people.
T Network Application Frameworks and XML Web Services and WSDL Sasu Tarkoma Based on slides by Pekka Nikander.
ALICE Offline week, CERN 21 February 2005 I. Hrivnacova 1 New geometry framework in MUON I.Hrivnacova IPN, Orsay ALICE Offline week, CERN 21 February 2005.
70-294: MCSE Guide to Microsoft Windows Server 2003 Active Directory, Enhanced Chapter 4: Active Directory Architecture.
Interacting with a UNIX computer: Navigating through the directory tree.
B-Trees. CSM B-Trees 2 Motivation for B-Trees So far we have assumed that we can store an entire data structure in main memory What if we have so.
Chapter 6 Binary Trees. 6.1 Trees, Binary Trees, and Binary Search Trees Linked lists usually are more flexible than arrays, but it is difficult to use.
Learning to Use Root for STAR Gene Van Buren (UCLA) for the STAR Collaboration Accessing STAR Data Programming for STAR (Makers) Star => Computing => Tutorials.
Introduction CS 3358 Data Structures. What is Computer Science? Computer Science is the study of algorithms, including their  Formal and mathematical.
ATLAS Detector Description Database Vakho Tsulaia University of Pittsburgh 3D workshop, CERN 14-Dec-2004.
© Janice Regan, CMPT 300, May CMPT 300 Introduction to Operating Systems Memory: Relocation.
STAR Sti, main features V. Perevoztchikov Brookhaven National Laboratory,USA.
File Storage Organization The majority of space on a device is reserved for the storage of files. When files are created and modified physical blocks are.
B + -Trees. Motivation An AVL tree with N nodes is an excellent data structure for searching, indexing, etc. The Big-Oh analysis shows that most operations.
B-Trees. Motivation for B-Trees So far we have assumed that we can store an entire data structure in main memory What if we have so much data that it.
COMP3190: Principle of Programming Languages
Refactoring & Testability. Testing in OOP programming No life in flexible methodologies and for refactoring- infected developers without SOME kind of.
Design Patterns -- Omkar. Introduction  When do we use design patterns  Uses of design patterns  Classification of design patterns  Creational design.
STAR STAR VMC tracker V. Perevoztchikov Brookhaven National Laboratory,USA.
STAR Kalman Track Fit V. Perevoztchikov Brookhaven National Laboratory,USA.
STAR Event data storage and management in STAR V. Perevoztchikov Brookhaven National Laboratory,USA.
Λειτουργικά Συστήματα - Lab1 Γιάννης Πετράκης. The Operating System  Unix is a layered operating system  The innermost layer is the hardware that provides.
STAR Collaboration, July 2004 Grid Collector Wei-Ming Zhang Kent State University John Wu, Alex Sim, Junmin Gu and Arie Shoshani Lawrence Berkeley National.
STAR Schema Evolution Implementation in ROOT I/O V. Perevoztchikov Brookhaven National Laboratory,USA.
12 CVS Mauro Jaskelioff (originally by Gail Hopkins)
General Purpose ROOT Utilities Victor Perevoztchikov, BNL.
V.Fine ALICE-STAR Joint meeting April, 9, 2000 The STAR offline framework* V. Fine *) See also:
B-TREE. Motivation for B-Trees So far we have assumed that we can store an entire data structure in main memory What if we have so much data that it won’t.
V.Fine for STAR collaboration The STAR offline framework and 3D visualization V. Fine, Y. Fisyak, V. Perevoztchikov, T.Wenaus.
Online Monitoring System at KLOE Alessandra Doria INFN - Napoli for the KLOE collaboration CHEP 2000 Padova, 7-11 February 2000 NAPOLI.
General requirements for BES III offline & EF selection software Weidong Li.
Classes - Part II (revisited) n Constant objects and member functions n Definition Form of Member Functions n friend functions and friend classes n Constructors.
Silberschatz, Galvin and Gagne ©2011 Operating System Concepts Essentials – 8 th Edition Chapter 2: The Linux System Part 2.
V.Fine for STAR collaboration The STAR offline framework V. Fine, Y. Fisyak, V. Perevoztchikov, T.Wenaus.
STAR Persistent Pointers in the STAR Micro-DST V. Perevoztchikov Brookhaven National Laboratory,USA.
StEvent I/O Model And Writing a Maker Or How to Add a New Detector Akio Ogawa BNL 2003 Nov Dubna.
STAR Simulation. Status and plans V. Perevoztchikov Brookhaven National Laboratory,USA.
20 October 2005 LCG Generator Services monthly meeting, CERN Validation of GENSER & News on GENSER Alexander Toropin LCG Generator Services monthly meeting.
AliRoot survey: Calibration P.Hristov 11/06/2013.
Modularization of Geant4 Dynamic loading of modules Configurable build using CMake Pere Mato Witek Pokorski
B-Tree Michael Tsai 2017/06/06.
Game Architecture Rabin is a good overview of everything to do with Games A lot of these slides come from the 1st edition CS 4455.
B+-Trees.
Magento Technical Guidelines Eugene Shakhsuvarov, Software Magento
Designing and Debugging Batch and Interactive COBOL Programs
Chapter 2: The Linux System Part 2
8 EGR preliminary frame and validation tasks
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 4: Implementing and Managing Group and Computer Accounts.
Other time considerations
OO-Design in PHENIX PHENIX, a BIG Collaboration A Liberal Data Model
Lesson 5: More on Creational Patterns
B+-trees In practice, B-trees are not used much as defined earlier.
Presentation transcript:

STAR C OMPUTING Maker and I/O Model in STAR Victor Perevoztchikov

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Why? STAR framework(root4star or StarGate), based on maker schema was developed several years ago. It is working well a long time. Why I reporting about it now? 1. Documentation unfortunately is still bad or absent; 2.We have a lot of new physicists in STAR who are not really familiar with STAR framework; 3.Because of reasons above, the new code more and more often contradicts to STAR framework “ideology” and to support it becomes more and more hard.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Main Concepts  ROOT as a basic tool for framework;  High level of modularity;  Hierarchy, tree of modules;  Data and methods separation;  Data and modules in one structure;  Structured data cash;  Steering – ROOT macro;  ROOT I/O with schema evolution;

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Glue Class TDataSet The main class supporting STAR framework is a TDataSet. It was developed in STAR with name StDataSet. Later accepted in ROOT as TDataSet. The class is simple, and very similar to UNIX directory/file system.  Everything important for framework is inherited from TDataSet. Tables,StEvent,i/o branches and modules==makers are inherited from TDataSet.  STAR framework structure is a TDataSet tree, where all framework objects are placed, in some specific order. This structure contains the data objects and the methods(makers) together;  From the ROOT prompt, user can browse them and can invoke the methods of each object manually. It is very important for debugging and understanding the structure.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 STAR Framework Tree Organization The organization of this tree is very simple:  The top level object is always maker;  Under it the system directories (nodes) with names: “.make”, “.data”, “.const”, “.garb”, etc…  “.make” node contains subordinated Makers, which in their turn contain the same substructure as the top Maker, and so on;  “.data” node contains list of Data objects, belonged to this Maker and deleted automatically before each event;  “.const” same as “.data” but objects are deleted only by user decision;  “.garb” same as “.data” but deleted automatically after return from StMaker:Make() method; The deep of this structure is unlimited. All the nodes in this Tree are equal, but the Makers are “more equal”.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 User Makers User Maker must be inherited from StMaker (which in his turn inherited from TDataSet). Methods:  The main method is Make() which usually must be overloaded by user. When user in his Make() calls StMaker::Make(), this triggers the call Make() of all subordinated Makers, and so on.  Methods Init(),Clear(),Finish() could be overloaded too. All these methods also trigger calls to the same method of subordinated Makers, and so on.  Method GetDataSet(“DataObjName”) returns pointer to Data object produced by another Maker.  Method AddData, AddConst and AddGarb add user objects to maker subdirectories “.data”,”.const”,”.garb” respectively. After this user objects are accessible by other makers by GetDataSet(…);  Method GetDataBase(“name”) returns pointer to Data Base object, already updated for given time stamp;

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Makers Rules Any application is constructed from makers. As a result we have a Tree of makers and Data objects. Creation of this Tree is based on loading of shared libraries related to concrete makers. To do it in a flexible way, some limitations are imposed to Makers:  Independency of Makers: l Maker is independent from other maker. It does not know which makers are now in the Tree. l An exception in case of subordinated makers. Parent maker could be dependent of child makers. Not vice versa. l Adding or removing of Maker to/from Tree, must not lead to code changes in any maker. l Changing of code in one maker should not lead to recompilation or especially modification of another maker.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Maker Rules (Continued)  Dependency of Makers: l Maker depends from Data objects. But again, Maker must not know, which maker produced the input Data objects, and which one will use Data objects made by him. l Maker depends from the third party classes. Like common for experiment or detector or group classes.  Violation of rules: method GetMaker() allows to find a maker by name. Soon some makers started to do direct call of methods of other makers. Flexibility, modularity decreased. Debugging becomes more and more hard.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Data Cash Data objects in STAR frame work are kept in Data Cash. Methods:  StMaker::AddData(TDataSet *ptr) adds produced object into Data Cash;  TDataSet *ptr = StMaker::GetDataSet(name); requests object from Cash by name; Data Cash in STAR is structured and based on Maker Tree. Search by name is not arbitrary. The strategy is: 1.Search data from sub-makers. If not: 2.Search data from the same level makers, already processed. 3.Search from upper processed makers; 4.Etc… This allows to resolve name clashes. Maker gets data from the nearest maker sub-tree. It allows to organize the different streams of data with the same name. Makers do not need to account it in code.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Data Object Name Each Data Object has a name, which is used in search. It is desirable that this name is unique. But it is not always possible. There are problems with the name search:  The search strategy gives the wrong object with correct name;  The name was changed, but requested name in maker still the old one. To modify the code is a bad idea. This maker could be used in other Maker Tree with this name;  User creates a new maker, but he does not know which name of object will be used in a future application. All these problems are solved by Logical Name approach. The method GetDataSet(name) actually use not real name but a logical one. By default, these names are the same. But this could be changed by. StMaker::SetAlias(logName,actName); This method is called in steering part, where Makers are instantiated. So no need to change the code and all problems are solved.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 System Makers STAR framework is a simple framework. All the work provided by makers only. So there are user Makers and system makers. System makers:  StBFChain is a top maker for reconstruction. It creates a maker tree using set of flags.  Set of I/O makers: StIOMaker,StTreeMaker, StDAQMaker. StIOMaker is a top I/O maker. Depending of the file type it calls appropriate I/O maker.  StGeantMaker (St_geant_Maker) reads GEANT data. It can also to run real GEANT;  StDBMaker(St_db_Maker) request data from MySql DB or from set of UNIX directories;  StEventMaker: collects data objects and creates the basic STAR Data Object - StEvent. With except of StBFChain and StGeantMaker, all these makers could have several instantiations in the maker Tree.

STAR C OMPUTING Victor Perevoztchikov, BNL STAR Collaboration Meeting 8/2003 Conclusions  STAR framework is in good shape and working well;  Documentation is still bad;  There is a danger of violation it’s basic principles. As a result could be decreasing flexibility, debugging problems and instability. This report is an attempt to improve the situation.  The feedback from users is very appreciated.