CHEP ‘06 David Lawrence JLab C++ Introspection and Object Persistency through JIL David Lawrence Ph. D. Jefferson Lab, Newport News VA.

Slides:



Advertisements
Similar presentations
Networked Digital Whiteboard with Handwritten-Symbol Interpreter and Dynamic-Display-Object Creator Atsuhide Kobashi Henry M. Gunn High School Palo Alto,
Advertisements

DSLs: The Good, the Bad, and the Ugly Kathleen Fisher AT&T Labs Research.
CSIE30300 Computer Architecture Unit 10: Virtual Memory Hsin-Chou Chi [Adapted from material by and
JAXB Java Architecture for XML Binding Andy Fanton Khalid AlBayat.
A Package For Tracking Validation Chris Meyer UC Santa Cruz July 6, 2007.
David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL March 25, 2003 CHEP 2003 Data Analysis Environment and Visualization.
Combining Static and Dynamic Data in Code Visualization David Eng Sable Research Group, McGill University PASTE 2002 Charleston, South Carolina November.
Microsoft Office Open XML Formats Brian Jones Lead Program Manager Microsoft Corporation.
1 The World Wide Web. 2  Web Fundamentals  Pages are defined by the Hypertext Markup Language (HTML) and contain text, graphics, audio, video and software.
IBM Software Group © 2005 IBM Corporation University of Nantes Eclipse dayMarch 2005 The Eclipse Modeling Framework and the IBM Model Transformation Framework.
CVSQL 2 The Design. System Overview System Components CVSQL Server –Three network interfaces –Modular data source provider framework –Decoupled SQL parsing.
Guide to Using Message Maker Robert Snelick National Institute of Standards & Technology (NIST) December 2005
Component-Based Software Engineering Introduction to Java Beans Paul Krause and Sotiris Moschoyiannis.
Multi-threaded Event Processing with JANA David Lawrence – Jefferson Lab Nov. 3, /3/08 Multi-threaded Event Processing with JANA - D. Lawrence JLab.
XML I/O in ROOT S. Linev, R. Brun, H.G. Essel CHEP 2004.
ROOT: A Data Mining Tool from CERN Arun Tripathi and Ravi Kumar 2008 CAS Ratemaking Seminar on Ratemaking 17 March 2008 Cambridge, Massachusetts.
Avro Apache Course: Distributed class Student ID: AM Name: Azzaya Galbazar
Zubanov Alexei, 2006 Aug 07 – Sep 28 QtROOT package extension with Coin3D lib.
Introduction to Hall-D Software February 27, 2009 David Lawrence - JLab.
NetTech Solutions Working with Web Elements Lesson 6.
December 15, 2011 Use of Semantic Adapter in caCIS Architecture.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
Simulations Progress at Regina ➔ Event generation with genr8 – output in ascii format ➔ Conversion to either HDFast input (stdhep) or HDGeant input (hddm)
Imperial College Tracker Slow Control & Monitoring.
11 MANAGING AND DISTRIBUTING SOFTWARE BY USING GROUP POLICY Chapter 5.
How P3P Works Lorrie Faith Cranor P3P Specification Working Group Chair AT&T Labs-Research 4 February 2002
Offline Coordinators  CMSSW_7_1_0 release: 17 June 2014  Usage:  Generation and Simulation samples for run 2 startup  Limited digitization and reconstruction.
GlueX Software Status April 28, 2006 David Lawrence, JLab.
David N. Brown Lawrence Berkeley National Lab Representing the BaBar Collaboration The BaBar Mini  BaBar  BaBar’s Data Formats  Design of the Mini 
Grid Computing Research Lab SUNY Binghamton 1 XCAT-C++: A High Performance Distributed CCA Framework Madhu Govindaraju.
March 27, 2007HPC 07 - Norfolk, VA1 C++ Reflection for High Performance Problem Solving Environments Tharaka Devadithya 1, Kenneth Chiu 2, Wei Lu 1 1.
Introduction Advantages/ disadvantages Code examples Speed Summary Running on the AOD Analysis Platforms 1/11/2007 Andrew Mehta.
VIRTUAL MEMORY By Thi Nguyen. Motivation  In early time, the main memory was not large enough to store and execute complex program as higher level languages.
Creating Graphical User Interfaces (GUI’s) with MATLAB By Jeffrey A. Webb OSU Gateway Coalition Member.
DANA uses a factory model to deliver data “just in time”
The Alternative Larry Moore. 5 Nodes and Variant Input File Sizes Hadoop Alternative.
Framework for MDO Studies Amitay Isaacs Center for Aerospace System Design and Engineering IIT Bombay.
C++ Introspection and Object persistency Through JIL David Lawrence Ph. D. Jefferson Lab, Newport News VA.
Optimizing CMS Data Formats for Analysis Peerut Boonchokchuay August 11 th,
STAR Event data storage and management in STAR V. Perevoztchikov Brookhaven National Laboratory,USA.
Renesas Technology America Inc. 1 M16C Seminars Lab 3 Creating Projects Using HEW4 14 March 2005 M16C Seminars Lab 3 Creating Projects Using HEW4 Last.
David Adams ATLAS DIAL: Distributed Interactive Analysis of Large datasets David Adams BNL August 5, 2002 BNL OMEGA talk.
STAR Schema Evolution Implementation in ROOT I/O V. Perevoztchikov Brookhaven National Laboratory,USA.
SoLID simulation with GEMC Zhiwen Zhao 2015/03/26.
Dr. David Roldán Martínez Universidad Politécnica de Valencia, Spain & Nuno Fernandes Universidade Fernando Pessoa, Portugal Site Stats, the power of event.
20081 Converting workspaces and using SALT & subversion to maintain them. V1.02.
September 6, GJXDM Users Conference NCIC Schema Challenges Patrice A. Yuh
© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/3 JSP Application Models.
MVC WITH CODEIGNITER Presented By Bhanu Priya.
The JANA Reconstruction Framework David Lawrence - JLab May 25, /25/101JANA - Lawrence - CLAS12 Software Workshop.
ISA 95 Working Group Process Centric Exchanges Gavan W Hood July 23, 2015 GWH 2.1.
Gaudi Framework Tutorial, April Creating Objects and Writing Data.
LHCbDirac and Core Software. LHCbDirac and Core SW Core Software workshop, PhC2 Running Gaudi Applications on the Grid m Application deployment o CVMFS.
GlueX Software Status + Framework Development David Lawrence JLab September 19, /19/081Software Status -- David Lawrence, JLab.
Partially Populated for ADT Messages
October 19, 2010 David Lawrence JLab Oct. 19, 20101RootSpy -- CHEP10, Taipei -- David Lawrence, JLab Parallel Session 18: Software Engineering, Data Stores,
Apache Cocoon – XML Publishing Framework 데이터베이스 연구실 박사 1 학기 이 세영.
Wednesday NI Vision Sessions
Evaluation of the C++ binding to the Oracle Database System Dirk Geppert and Krzysztof Nienartowicz, IT/DB CERN IT Fellow Seminar November 20, 2002.
“Port Monitor”: progress & open questions Torsten Wilde and James Kohl Oak Ridge National Laboratory CCA Forum Quarterly Meeting Santa Fe, NM ~ October.
POOL Based CMS Framework Bill Tanenbaum US-CMS/Fermilab 04/June/2003.
1 GlueX Software Oct. 21, 2004 D. Lawrence, JLab.
David Lawrence JLab May 11, /11/101Reconstruction Framework -- GlueX Collab. meeting -- D. Lawrence.
DANA David Lawrence Oct. 21, D. Lawrence, JLab GlueX Software Workshop Oct Outline Requirements Philosophy Overview of Features Open Questions.
*DT Project Model Leo Treggiari Intel Corp. Dec, 2005.
ROOT IO workshop What relates to ATLAS. General Both CMS and Art pushing for parallelization at all levels. Not clear why as they are anyhow CPU bound.
LOCO Extract – Transform - Load
How P3P Works Lorrie Faith Cranor P3P Specification Working Group Chair AT&T Labs-Research 4 February
The DANA Analysis Framework David Lawrence Jlab Jan. 19,
Deepak Shenoy Agni Software
Presentation transcript:

CHEP ‘06 David Lawrence JLab C++ Introspection and Object Persistency through JIL David Lawrence Ph. D. Jefferson Lab, Newport News VA

CHEP ‘06 David Lawrence JLab Introspection and Object Persistency Introspection: Ability for a program to query the structure of its classes from within the program. Object Persistency: Writing the data portion of an object to a file so that it can be re-created later by another program.

CHEP ‘06 David Lawrence JLab Motivation for JIL CPU intensive analyses need staging (e.g. track fitting first, then physics analysis later) The GlueX experiment will require this Make general use tool that allows 3rd party objects to be made persistent without modification of 3rd party source Allow object persistence to be added to legacy file formats (CODA) Incorporate object meta-data in persistency framework (DANA factory tags)

CHEP ‘06 David Lawrence JLab Serializers Files are 1-dimensional things. In order to write a multi-dimensional “object” into a file, it must be serialized. A corresponding deserializer must also be written to reconstitute the object later.

CHEP ‘06 David Lawrence JLab Multiple approaches to creating serializers in the absence of introspection C++ parser/interpreter (ROOT) Write class definitions in XML format and generate C++ headers Hand coded serializers

CHEP ‘06 David Lawrence JLab A Simple Example

CHEP ‘06 David Lawrence JLab Use Gnu Debugger (gdb) to mine class/struct structure

CHEP ‘06 David Lawrence JLab XML representation of “myClass”

CHEP ‘06 David Lawrence JLab (De)Serializer Routines: Auto-generated code Convert complex structure into “simple” type problem

CHEP ‘06 David Lawrence JLab

Writing objects to a file

CHEP ‘06 David Lawrence JLab Reading objects from a file

CHEP ‘06 David Lawrence JLab Serialization time vs. # events

CHEP ‘06 David Lawrence JLab Deserialization time vs. # events

CHEP ‘06 David Lawrence JLab Summary JIL provides a mechanism to add object persistency through files to C++ projects JIL can use either C++ code or XML as the source from which to generate serializer methods JIL can be used to make 3rd party objects persistent without modifying the 3rd party source code

CHEP ‘06 David Lawrence JLab Summary JIL (optionally) uses zlib compression which reduces file size by a factor of 3 JIL is multi-thread enabled (not just thread- safe) JIL allows users to implement customized file format Alpha development stage. For more info, contact:

CHEP ‘06 David Lawrence JLab Extra Slides

CHEP ‘06 David Lawrence JLab Surely the object persistency problem has been solved for C++ many times before. Yes. Projects like ROOT, s11n, Poet, ROM, and MSVSC++ add object persistence. However, only ROOT is really designed for high volume, efficient, compact usage like in HEP and MENP.