21 April, 1999 Vincenzo Innocente LHC++ Meeting1 Time-Ordered Persistent Collections Vincenzo Innocente CMS Collaboration see also contribution to RD45.

Slides:



Advertisements
Similar presentations
Protein – Protein Interactions Lisa Chargualaf Simon Kanaan Keefe Roedersheimer Others: Dr. Izaguirre, Dr. Chen, Dr. Wuchty, ChengBang Huang.
Advertisements

Concurrency Control Part 2 R&G - Chapter 17 The sequel was far better than the original! -- Nobody.
Overview of Data Structures and Algorithms
Dr. Kalpakis CMSC 661, Principles of Database Systems Representing Data Elements [12]
March 24-28, 2003Computing for High-Energy Physics Configuration Database for BaBar On-line Rainer Bartoldus, Gregory Dubois-Felsmann, Yury Kolomensky,
Sponsored by the U.S. Department of Defense © 2005 by Carnegie Mellon University 1 Pittsburgh, PA Dennis Smith, David Carney and Ed Morris DEAS.
Computer Monitoring System for EE Faculty By Yaroslav Ross And Denis Zakrevsky Supervisor: Viktor Kulikov.
Client/Server Databases and the Oracle 10g Relational Database
1 Introduction to Database Management Systems Lila Rao Graham.
Transaction Management and Concurrency Control
Chapter 4: Transaction Management
Chapter 4: Database Management. Databases Before the Use of Computers Data kept in books, ledgers, card files, folders, and file cabinets Long response.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Transaction Management and Concurrency Control.
Automated Tests in NICOS Nightly Control System Alexander Undrus Brookhaven National Laboratory, Upton, NY Software testing is a difficult, time-consuming.
CMS Alignment and Calibration Yuriy Pakhotin on behalf of CMS Collaboration.
The Unified Modeling Language (UML) Class Diagrams.
Database Application Security Models
Information storage: Introduction of database 10/7/2004 Xiangming Mu.
Introduction –All information systems create, read, update and delete data. This data is stored in files and databases. Files are collections of similar.
Chapter 1 Overview of Database Concepts Oracle 10g: SQL
Database System Concepts and Architecture
Chapter 9 Designing Databases Modern Systems Analysis and Design Sixth Edition Jeffrey A. Hoffer Joey F. George Joseph S. Valacich.
Chapter 1 In-lab Quiz Next week
Computers Data Representation Chapter 3, SA. Data Representation and Processing Data and information processors must be able to: Recognize external data.
Lecture 2 An Overview of Relational Database IST 318 – DB Admin.
Conditions DB in LHCb LCG Conditions DB Workshop 8-9 December 2003 P. Mato / CERN.
Scheduling policies for real- time embedded systems.
ALICE Upgrade for Run3: Computing HL-LHC Trigger, Online and Offline Computing Working Group Topical Workshop Sep 5 th 2014.
Event Data History David Adams BNL Atlas Software Week December 2001.
Chapter 1Introduction to Oracle9i: SQL1 Chapter 1 Overview of Database Concepts.
Ihr Logo Operating Systems Internals & Design Principles Fifth Edition William Stallings Chapter 2 (Part II) Operating System Overview.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 20 Slide 1 Critical systems development 3.
A Technical Validation Module for the offline Auger-Lecce, 17 September 2009  Design  The SValidStore Module  Example  Scripting  Status.
Time Management.  Time management is concerned with OS facilities and services which measure real time, and is essential to the operation of timesharing.
Storage Structures. Memory Hierarchies Primary Storage –Registers –Cache memory –RAM Secondary Storage –Magnetic disks –Magnetic tape –CDROM (read-only.
AUDIT SOFTWARE Chapter 16. Generalized Audit Software Off-the-shelf software that provides a means to gain access to and manipulate data maintained on.
1 Technical & Business Writing (ENG-715) Muhammad Bilal Bashir UIIT, Rawalpindi.
Session 1 Module 1: Introduction to Data Integrity
Chapter 1 Revealed Distributed Objects Design Concepts CSLA.
Time Management.  Time management is concerned with OS facilities and services which measure real time.  These services include:  Keeping track of.
Vincenzo Innocente, CERN/EPUser Collections1 Grid Scenarios in CMS Vincenzo Innocente CERN/EP Simulation, Reconstruction and Analysis scenarios.
18 September 2008CIS 340 # 1 Last Covered (almost)(almost) Variety of middleware mechanisms Gain? Enable n-tier architectures while not necessarily using.
M.Frank, CERN/LHCb Persistency Workshop, Dec, 2004 Distributed Databases in LHCb  Main databases in LHCb Online / Offline and their clients  The cross.
VI/ CERN Dec 4 CMS Software Architecture vs Hybrid Store Vincenzo Innocente CMS Week CERN, Dec
Vincenzo Innocente, CERN/EP Persistency: October 1999, CERN 1st Internal Review of CMS Software and Computing Why a Commercial ODMBS can suit CMS.
Introduction: Databases and Database Systems Lecture # 1 June 19,2012 National University of Computer and Emerging Sciences.
Chapter 13 Managing Transactions and Concurrency Database Principles: Fundamentals of Design, Implementation, and Management Tenth Edition.
SPECIAL PURPOSE DATABASES 13/09/ Temporal Database Concepts  Time is considered ordered sequence of points in some granularity Use the term choronon.
ISC321 Database Systems I Chapter 2: Overview of Database Languages and Architectures Fall 2015 Dr. Abdullah Almutairi.
Data Integrity & Indexes / Session 1/ 1 of 37 Session 1 Module 1: Introduction to Data Integrity Module 2: Introduction to Indexes.
Chapter 2 Database Environment.
Introduction to Database Management Systems
Course Outcomes of Object Oriented Modeling Design (17630,C604)
Architecture Concept Documents
SOFTWARE DESIGN AND ARCHITECTURE
Transaction Management and Concurrency Control
MongoDB Distributed Write and Read
Chapter 15 Basics of Functional Dependencies and Normalization for Relational Databases.
Chapter 2 Database Environment.
Chapter 2 Database Environment Pearson Education © 2009.
Vincenzo Innocente CERN/EP/CMC
Chapter 10 Transaction Management and Concurrency Control
Introduction of Week 13 Return assignment 11-1 and 3-1-5
CMS Persistent Event Structure
Unit 2: Fundamentals of Computer Systems
Ensuring the Quality and Best Use of Information
Chapter 22 Object-Oriented Systems Analysis and Design and UML
CMS Software Architecture
Chapter 2 Database Environment Pearson Education © 2009.
Offline framework for conditions data
Presentation transcript:

21 April, 1999 Vincenzo Innocente LHC++ Meeting1 Time-Ordered Persistent Collections Vincenzo Innocente CMS Collaboration see also contribution to RD45 workshop of 4/98

21 April, 1999 Vincenzo Innocente LHC++ Meeting2 Problem l Given an object, whose state depends upon time, I want to know its state for any given time in the past (and predict the future?) n Its state is determined from direct or deduced measurements (or equivalent procedure) of unspecified nature, precision and frequency l Well known solution: the object store a history of its measured states in a time-series and compute the best value for a given time using a suitable interpolation or extrapolation algorithm.

21 April, 1999 Vincenzo Innocente LHC++ Meeting3 Assumptions (Requirements??) l Writing (insertion in the collection): n once (never delete, never modify in place) n in object natural time order n by a single producer l Object Time Validity n until a new object is inserted n at each time one, and only one object, is valid (per version)

21 April, 1999 Vincenzo Innocente LHC++ Meeting4 Assumptions (Requirements??) l Version Validity n Data themselves are WORM, the collection may not s objects are obsoleted only by “versioning” n a version “tags” all objects in a certain time period s for a given “tag” and a given time a valid object should exists n an object can belong to more than one version n current version makes sense n all versions should be available n reproducing the state of the collection at a given past time should be possible

21 April, 1999 Vincenzo Innocente LHC++ Meeting5 Assumptions (Requirements??) l Obsolescence: n rare process: few new versions will be created (0-10) n some objects can NOT be obsoleted (sort of raw-data) n In most of the cases all objects belonging to a certain time period will be obsoleted at the same time n Often the validity time of the new version will be the same of the previous version n In some cases the new version will have a different time granularity (usually finer)

21 April, 1999 Vincenzo Innocente LHC++ Meeting6 Assumptions (Requirements??) l Read: n Very very often, by multiple clients n besides online applications, current real time is not special at all n often in natural sequence, not necessarily contiguous n random access is NOT a secondary scenario n interpolation (and extrapolation?) should be supported, not necessarily implemented

21 April, 1999 Vincenzo Innocente LHC++ Meeting7 Assumptions (Misuses) l Versioning should be used only for making object obsolete n NOT to keep two different concurrent “version” of the same “logical entity” (for instance: online and offline calibrations): use distinct collections n The same applies for “user” and “test” versions (even transient ones) n NOT to differentiate among different parts of the detector: use different collections or different components in the same “super-object”

21 April, 1999 Vincenzo Innocente LHC++ Meeting8 Assumption summary (Primary Scenarios) l Data (measurements) are WORM and not related to time collections by themselves l Main insertion mode is “push_back” l Random update is not frequent and it is the result of a computing intensive activity l A new “release tag” affects long “time periods” l Random reading will be frequent l Interpolation/extrapolation should be left to the application software

21 April, 1999 Vincenzo Innocente LHC++ Meeting9 Persistent Architecture Version (Insertion time) Time

21 April, 1999 Vincenzo Innocente LHC++ Meeting10 Persistent Architecture Version (Insertion time) Time

21 April, 1999 Vincenzo Innocente LHC++ Meeting11 Time Series (column-wise) Insertion time Persistent List of stacks (time- slices) This mechanism guarantees that the selected object is valid for the whole time slice

21 April, 1999 Vincenzo Innocente LHC++ Meeting12

21 April, 1999 Vincenzo Innocente LHC++ Meeting13 Transient Components