3D Application Tests Application test proposals

Slides:



Advertisements
Similar presentations
RLS Production Services Maria Girone PPARC-LCG, CERN LCG-POOL and IT-DB Physics Services 10 th GridPP Meeting, CERN, 3 rd June What is the RLS -
Advertisements

D. Düllmann - IT/DB LCG - POOL Project1 POOL Release Plan for 2003 Dirk Düllmann LCG Application Area Meeting, 5 th March 2003.
Database monitoring and service validation Dirk Duellmann CERN IT/PSS and 3D
LHCC Comprehensive Review – September WLCG Commissioning Schedule Still an ambitious programme ahead Still an ambitious programme ahead Timely testing.
LCG 3D StatusDirk Duellmann1 LCG 3D Throughput Tests Scheduled for May - extended until end of June –Use the production database clusters at tier 1 and.
5 November 2001F Harris GridPP Edinburgh 1 WP8 status for validating Testbed1 and middleware F Harris(LHCb/Oxford)
RLS Tier-1 Deployment James Casey, PPARC-LCG Fellow, CERN 10 th GridPP Meeting, CERN, 3 rd June 2004.
ATLAS Metrics for CCRC’08 Database Milestones WLCG CCRC'08 Post-Mortem Workshop CERN, Geneva, Switzerland June 12-13, 2008 Alexandre Vaniachine.
Conditions DB in LHCb LCG Conditions DB Workshop 8-9 December 2003 P. Mato / CERN.
SRM 2.2: status of the implementations and GSSD 6 th March 2007 Flavia Donno, Maarten Litmaath INFN and IT/GD, CERN.
Workshop Summary (my impressions at least) Dirk Duellmann, CERN IT LCG Database Deployment & Persistency Workshop.
LCG 3D Project Status and Production Plans Dirk Duellmann, CERN IT On behalf of the LCG 3D project CHEP 2006, 15th February, Mumbai.
ATLAS Database Operations Invited talk at the XXI International Symposium on Nuclear Electronics & Computing Varna, Bulgaria, September 2007 Alexandre.
Databases E. Leonardi, P. Valente. Conditions DB Conditions=Dynamic parameters non-event time-varying Conditions database (CondDB) General definition:
Database Administrator RAL Proposed Workshop Goals Dirk Duellmann, CERN.
CERN Physics Database Services and Plans Maria Girone, CERN-IT
3D Workshop Outline & Goals Dirk Düllmann, CERN IT More details at
LCG Service Challenges: Planning for Tier2 Sites Update for HEPiX meeting Jamie Shiers IT-GD, CERN.
LCG Service Challenges: Planning for Tier2 Sites Update for HEPiX meeting Jamie Shiers IT-GD, CERN.
LFC Replication Tests LCG 3D Workshop Barbara Martelli.
Feedback from the POOL Project User Feedback from the POOL Project Dirk Düllmann, LCG-POOL LCG Application Area Internal Review October 2003.
CERN - IT Department CH-1211 Genève 23 Switzerland t COOL Conditions Database for the LHC Experiments Development and Deployment Status Andrea.
CERN-IT Oracle Database Physics Services Maria Girone, IT-DB 13 December 2004.
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Pre-GDB Storage Classes summary of discussions Flavia Donno Pre-GDB.
Database Readiness Workshop Summary Dirk Duellmann, CERN IT For the LCG 3D project SC4 / pilot WLCG Service Workshop.
CERN Database Services for the LHC Computing Grid Maria Girone, CERN.
Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 24 January 2005.
3D Project Status Dirk Duellmann, CERN IT For the LCG 3D project Meeting with LHCC Referees, March 21st 06.
Plans for Service Challenge 3 Ian Bird LHCC Referees Meeting 27 th June 2005.
CERN IT Department CH-1211 Geneva 23 Switzerland t WLCG Operation Coordination Luca Canali (for IT-DB) Oracle Upgrades.
LCG 3D Project Update (given to LCG MB this Monday) Dirk Duellmann CERN IT/PSS and 3D
Maria Girone CERN - IT Tier0 plans and security and backup policy proposals Maria Girone, CERN IT-PSS.
Enabling Grids for E-sciencE INFSO-RI Enabling Grids for E-sciencE Gavin McCance GDB – 6 June 2007 FTS 2.0 deployment and testing.
Site Services and Policies Summary Dirk Düllmann, CERN IT More details at
Status of tests in the LCG 3D database testbed Eva Dafonte Pérez LCG Database Deployment and Persistency Workshop.
Database Project Milestones (+ few status slides) Dirk Duellmann, CERN IT-PSS (
D. Duellmann, IT-DB POOL Status1 POOL Persistency Framework - Status after a first year of development Dirk Düllmann, IT-DB.
INFSO-RI Enabling Grids for E-sciencE File Transfer Software and Service SC3 Gavin McCance – JRA1 Data Management Cluster Service.
A quick summary and some ideas for the 2005 work plan Dirk Düllmann, CERN IT More details at
CMS Experience with the Common Analysis Framework I. Fisk & M. Girone Experience in CMS with the Common Analysis Framework Ian Fisk & Maria Girone 1.
CERN - IT Department CH-1211 Genève 23 Switzerland t Service Level & Responsibilities Dirk Düllmann LCG 3D Database Workshop September,
DB Questions and Answers open session (comments during session) WLCG Collaboration Workshop, CERN Geneva, 24 of April 2008.
1 LCG Distributed Deployment of Databases A Project Proposal Dirk Düllmann LCG PEB 20th July 2004.
Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 6 April 2005.
LCG 3D Project Status For more details: Dirk Düllmann, CERN IT-ADC LCG Application Area Meeting 6 April, 2005.
SPI Report for the LHCC Comprehensive Review Stefan Roiser for the SPI project.
INFSO-RI Enabling Grids for E-sciencE Running reliable services: the LFC at CERN Sophie Lemaitre
INFN Tier1/Tier2 Cloud WorkshopCNAF, 22 November 2006 Conditions Database Services How to implement the local replicas at Tier1 and Tier2 sites Andrea.
L1Calo DBs: Status and Plans ● Overview of L1Calo databases ● Present status ● Plans Murrough Landon 20 November 2006.
Jean-Philippe Baud, IT-GD, CERN November 2007
Persistency Project (POOL) Status and Work Plan Proposal
Status of Distributed Databases (LCG 3D)
Dirk Duellmann CERN IT/PSS and 3D
Database Replication and Monitoring
LCG Service Challenge: Planning and Milestones
(on behalf of the POOL team)
LCG 3D Distributed Deployment of Databases
Database Services at CERN Status Update
Data Challenge with the Grid in ATLAS
Elizabeth Gallas - Oxford ADC Weekly September 13, 2011
Database Readiness Workshop Intro & Goals
Savannah to Jira Migration
LCG Distributed Deployment of Databases A Project Proposal
Dirk Düllmann CERN Openlab storage workshop 17th March 2003
Conditions Data access using FroNTier Squid cache Server
Workshop Summary Dirk Duellmann.
Bernd Panzer-Steindel CERN/IT
LHCb Conditions Database TEG Workshop 7 November 2011 Marco Clemencic
Data Management cluster summary
Database Services for CERN Deployment and Monitoring
Presentation transcript:

3D Application Tests Application test proposals Dirk Düllmann, CERN IT-ADC

Workshop Site reports Experiment reports Setup available db servers / db backup, LCG service involvements, capacity planning (and uncertainties) Experiment reports Applications status and schedules, data volumes and client numbers, distribution plans 3D meeting, 15. September 2005 D.Duellmann, CERN

3D application tests 3D Application test bed is in place First replication stability tests look promising Need to proceed with some real application tests now Need to determine the size the service for T1’s What hardware is required / is available? Need to defined concrete deployment models And determine scalability of alternative approaches Requests should be discussed here Should be delivered to the T1/T2 sites by the experiments (as this seams to be the established practice in LCG) Need to agree on a schedule based on Experiment priorities Existing production application, 3D meeting, 15. September 2005 D.Duellmann, CERN

Proposed Focus: Conditions COOL production release has happened Performance and scalability tests ongoing Reference work load implementation exists ATLAS and LHCb have requested distributed deployment Some manpower became available to the project Stefan Stonjek will be able to work 50% of his time on setting up and running the COOL tests together with the testbed sites 3D meeting, 15. September 2005 D.Duellmann, CERN

Proposed COOL test setup Define a complete end-to-end setup T0: install database and work load Based on existing validation test software T1: install replicas Need to connect master db at CERN to testbed nodes at T1’s T2: establish extraction procedure Need to define MySQL package for T2 Need to define T2 testbed sites (Chicago, Oxford,?) Develop extraction procedure for subset of data Once up and running Test scalability with T0, T0+T1, T0+T1+T2 3D meeting, 15. September 2005 D.Duellmann, CERN

COOL Tests: Site Installation T0 installation on testbed (2weeks) Pickup upcoming COOL 1.2 release (Andrea) Agree on test schema and fix it (Stefan/Sasha/Dirk) Develop replication scripts (Eva) Start modes (but permanent) workload T1 Installation of COOL replica (1week) Complete OEM configuration for sites which are missing (site reps) Apply replication scripts (site reps) Start replication process (Eva/site reps) T2 Install of MySQL and COOL snapshot (2 weeks) Define MySQL installation kit (Sasha/T2 reps) Define extraction process and frequency (Stefan/ATLAS) Start regular extraction and apply at T2 (Stefan + T1/T2) 3D meeting, 15. September 2005 D.Duellmann, CERN

COOL Tests : Client Setup Define client software package (2 weeks) Vanilla COOL or full experiment package (Stefan/Sasha) In the latter we can maybe take advantage of the established experiment s/w distribution to sites But we would need a compatible packaged release of the experiment s/w Check allocation of required client nodes Initial tests can hopefully run with limited client resources at T1/T2 (Dirk/Sasha) Run the tests… (Stefan) Outcome: How many clients can run against T0 test system alone T0 + n*T1 T0 + n*T1 + m*T2 A few new deployment issues :-) 3D meeting, 15. September 2005 D.Duellmann, CERN

AMI ATLAS would like to deploy AMI meta data application with replicas at Lyon and CERN Lyon is proposed by ATLAS as master site First Discussion between CERN database team IN2P3 Lyon database team AMI developers Agreed on draft installation and test plan 3D meeting, 15. September 2005 D.Duellmann, CERN

AMI production and test setup First (single instance) production setup at Lyon Lyon is looking at 10gR2 CERN is doing first 10gR2 test as well Production service expected to become available in October Application server setup/support still to be defined AMI replication tests Independent setup using the 3D testbed resources Goal: validate that AMI data can be distributed via Oracle streams Test Schedule Replication test suite (AMI developers - November) Setup of streams connected instances at CERN and Lyon (DB teams - December/January) Will require a 10g database box in Lyon to join the testbed Execution of tests (AMI/Lyon/CERN) 3D meeting, 15. September 2005 D.Duellmann, CERN

LFC Replication Tests Grid Deployment Group is in discussion with LHCb about requirements for LFC replication Possible setup: LFC read-only replicas at (some?) LHCb Tier 1 sites Streams based replication between T0 and T1 Looks very similar to Catalog Distribution tests we already did with the POOL catalog As LFC is now integrated into POOL catalog set we could simply recycle replication work load which has been developed for the earlier tests GD and LHCb will meet next week to come up with concrete requirements and a proposed test schedule 3D meeting, 15. September 2005 D.Duellmann, CERN