1 Grid in STAR: Needs and Plans A. Vaniachine STAR Grid Components in STAR: Experience, Needs and Plans A. Vaniachine Lawrence Berkeley National Laboratory.

Slides:

Advertisements

Similar presentations

Foundations for an LHC Data Grid Stu Loken Berkeley Lab.

Advertisements

The File Mover: An Efficient Data Transfer System for Grid Applications C. Anglano, M. Canonico Dipartimento di Informatica Universita' del Piemonte Orientale,

23/04/2008VLVnT08, Toulon, FR, April 2008, M. Stavrianakou, NESTOR-NOA 1 First thoughts for KM3Net on-shore data storage and distribution Facilities VLV.

Peer-to-Peer Networks as a Distribution and Publishing Model Jorn De Boever (june 14, 2007)

David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL March 25, 2003 CHEP 2003 Data Analysis Environment and Visualization.

Grid Collector: Enabling File-Transparent Object Access For Analysis Wei-Ming Zhang Kent State University John Wu, Alex Sim, Junmin Gu and Arie Shoshani.

Grids and Grid Technologies for Wide-Area Distributed Computing Mark Baker, Rajkumar Buyya and Domenico Laforenza.

Figure 1.1 Interaction between applications and the operating system.

David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL June 23, 2003 GAE workshop Caltech.

Magda – Manager for grid-based data Wensheng Deng Physics Applications Software group Brookhaven National Laboratory.

2/10/2000 CHEP2000 Padova Italy The BaBar Online Databases George Zioulas SLAC For the BaBar Computing Group.

M i SMob i S Mob i Store - Mobile i nternet File Storage Platform Chetna Kaur.

Alexandre A. P. Suaide VI DOSAR workshop, São Paulo, 2005 STAR grid activities and São Paulo experience.

David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL July 15, 2003 LCG Analysis RTAG CERN.

Miguel Branco CERN/University of Southampton Enabling provenance on large-scale e-Science applications.

A User’s Introduction to the Grand Challenge Software STAR-GC Workshop Oct 1999 D. Zimmerman.

100 Million events, what does this mean ?? STAR Grid Program overview.

NoSQL Databases Oracle - Berkeley DB. Content A brief intro to NoSQL About Berkeley Db About our application.

Grid Status - PPDG / Magda / pacman Torre Wenaus BNL U.S. ATLAS Physics and Computing Advisory Panel Review Argonne National Laboratory Oct 30, 2001.

PPDG and ATLAS Particle Physics Data Grid Ed May - ANL ATLAS Software Week LBNL May 12, 2000.

1 Data Warehouses BUAD/American University Data Warehouses.

May PEM status report. O.Bärring 1 PEM status report Large-Scale Cluster Computing Workshop FNAL, May Olof Bärring, CERN.

Introduction to dCache Zhenping (Jane) Liu ATLAS Computing Facility, Physics Department Brookhaven National Lab 09/12 – 09/13, 2005 USATLAS Tier-1 & Tier-2.

STAR C OMPUTING STAR Computing Infrastructure Torre Wenaus BNL STAR Collaboration Meeting BNL Jan 31, 1999.

Computer Science Research and Development Department Computing Sciences Directorate, L B N L 1 Storage Management and Data Mining in High Energy Physics.

NOVA Networked Object-based EnVironment for Analysis P. Nevski, A. Vaniachine, T. Wenaus NOVA is a project to develop distributed object oriented physics.

Magda Distributed Data Manager Status Torre Wenaus BNL ATLAS Data Challenge Workshop Feb 1, 2002 CERN.

Using Bitmap Index to Speed up Analyses of High-Energy Physics Data John Wu, Arie Shoshani, Alex Sim, Junmin Gu, Art Poskanzer Lawrence Berkeley National.

Chapter 6 Distributed File Systems Summary Bernard Chen 2007 CSc 8230.

Laboratoire LIP6 The Gedeon Project: Data, Metadata and Databases Yves DENNEULIN LIG laboratory, Grenoble ACI MD.

LCG Phase 2 Planning Meeting - Friday July 30th, 2004 Jean-Yves Nief CC-IN2P3, Lyon An example of a data access model in a Tier 1.

Acat OctoberRene Brun1 Future of Analysis Environments Personal views Rene Brun CERN.

The Replica Location Service The Globus Project™ And The DataGrid Project Copyright (c) 2002 University of Chicago and The University of Southern California.

ATLAS Grid Data Processing: system evolution and scalability D Golubkov, B Kersevan, A Klimentov, A Minaenko, P Nevski, A Vaniachine and R Walker for the.

1 GCA Application in STAR GCA Collaboration Grand Challenge Architecture and its Interface to STAR Sasha Vaniachine presenting for the Grand Challenge.

STAR Event data storage and management in STAR V. Perevoztchikov Brookhaven National Laboratory,USA.

February 28, 2003Eric Hjort PDSF Status and Overview Eric Hjort, LBNL STAR Collaboration Meeting February 28, 2003.

STAR C OMPUTING STAR Analysis Operations and Issues Torre Wenaus BNL STAR PWG Videoconference BNL August 13, 1999.

A B A B AR InterGrid Testbed Proposal for discussion Robin Middleton/Roger Barlow Rome: October 2001.

Argonne National Laboratory Tom LeCompte1 Testbeam Requirements and Requests ATLAS Software Week Tom LeCompte Argonne National Laboratory

Creating a Data Warehouse Data Acquisition: Extract, Transform, Load Extraction Process of identifying and retrieving a set of data from the operational.

STAR Collaboration, July 2004 Grid Collector Wei-Ming Zhang Kent State University John Wu, Alex Sim, Junmin Gu and Arie Shoshani Lawrence Berkeley National.

January 26, 2003Eric Hjort HRMs in STAR Eric Hjort, LBNL (STAR/PPDG Collaborations)

Workforce Scheduling Release 5.0 for Windows Implementation Overview OWS Development Team.

Grand Challenge in MDC2 D. Olson, LBNL 31 Jan 1999 STAR Collaboration Meeting

STAR C OMPUTING Plans for Production Use of Grand Challenge Software in STAR Torre Wenaus BNL Grand Challenge Meeting LBNL 10/23/98.

1 fileCatalog, tagDB and GCA A. Vaniachine Grand Challenge STAR fileCatalog, tagDB and Grand Challenge Architecture A. Vaniachine presenting for the Grand.

Copyright © 2006, GemStone Systems Inc. All Rights Reserved. Increasing computation throughput with Grid Data Caching Jags Ramnarayan Chief Architect GemStone.

Computing R&D and Milestones LHCb Plenary June 18th, 1998 These slides are on WWW at:

David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL November 17, 2003 SC2003 Phoenix.

1 VLDB - Data Management in Grids B. Del-Fabbro, D. Laiymani, J.M. Nicod and L. Philippe Laboratoire d’Informatique de l’Université de Franche-Comté Séoul,

13 1 Chapter 13 The Data Warehouse Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.

David Adams ATLAS ATLAS Distributed Analysis: Overview David Adams BNL December 8, 2004 Distributed Analysis working group ATLAS software workshop.

Nanbor Wang, Balamurali Ananthan Tech-X Corporation Gerald Gieraltowski, Edward May, Alexandre Vaniachine Argonne National Laboratory 2. ARCHITECTURE GSIMF:

Magda Distributed Data Manager Prototype Torre Wenaus BNL September 2001.

FP CumuloNimbo: A Highly Scalable Transactional Cloud PaaS Bettina Kemme, McGill University CumuloNimbo Canada EU Future Internet Workshop, 2011.

Grid Status - PPDG / Magda / pacman Torre Wenaus BNL DOE/NSF Review of US LHC Software and Computing Fermilab Nov 29, 2001.

DOE/NSF Quarterly review January 1999 Particle Physics Data Grid Applications David Malon Argonne National Laboratory

1 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Lawrence.

FESR Trinacria Grid Virtual Laboratory gLite Information System Muoio Annamaria INFN - Catania gLite 3.0 Tutorial Trigrid Catania,

Magda Distributed Data Manager Torre Wenaus BNL October 2001.

David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL May 19, 2003 BNL Technology Meeting.

Hall D Computing Facilities Ian Bird 16 March 2001.

Open Source distributed document DB for an enterprise

StratusLab Final Periodic Review

StratusLab Final Periodic Review

Data Warehouse.

OO-Design in PHENIX PHENIX, a BIG Collaboration A Liberal Data Model

Chapter 13 The Data Warehouse

Development of LHCb Computing Model F Harris

Presentation transcript:

1 Grid in STAR: Needs and Plans A. Vaniachine STAR Grid Components in STAR: Experience, Needs and Plans A. Vaniachine Lawrence Berkeley National Laboratory July 14, 2000 PPDG Workshop, Argonne

2 Grid in STAR: Needs and Plans A. Vaniachine STAR Outline STAR is taking data: Grid in analysis Grid for production? Plans

3 Grid in STAR: Needs and Plans A. Vaniachine STAR Decisions Taken Event split into components: –daq (raw data) or fzd (Geant3) –dst: legacy C-structures –event: same info in OO –tags: like an n-tuple –hist: for QA –... ROOT-based Architecture –Persistency: STL containers - structural and reference schema evolution –Analysis: “good-bye” PAW

4 Grid in STAR: Needs and Plans A. Vaniachine STAR Grid Components Deployed for Analysis tag database: to speed up user response –data in ROOT TTree files (n-tuple): data description stored together with the data (solves the schema evolution problem) comments preserved 2-level hierarchy: 500 tags are in seven groups split mode for structures (not objects: no need to load classes to analyze tags) indexed tags (Grand Challenge deployed) –efficient cache management –transparent user access to HPSS in principle, to the raw data as well: “the Dream” is coming true

5 Grid in STAR: Needs and Plans A. Vaniachine STAR Component-based Architectures Grand Challenge: –large size components one component per developer NOVA: –smaller-size components one components per task Needed: small components, easily adaptable to experiments e.g., “ls” for the HPSS

6 Grid in STAR: Needs and Plans A. Vaniachine STAR Grid Components for Production? $$$ constraints have to use low- cost commodity components: latency tolerant algorithms have to load all distributed nodes: sustained data transfer methods have to use HPPS data storage: efficient cache management transparent user access

7 Grid in STAR: Needs and Plans A. Vaniachine STAR Plans for PPDG Deploy file transfer methods for the BNL-LBNL link current rate: rcf  pdsf = 100 GB/day Managed file caching for high- throughput computing –pdsf as a development test-bed –rcf as a target for deployment