Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 24 January 2005.

Slides:



Advertisements
Similar presentations
Storage Review David Britton,21/Nov/ /03/2014 One Year Ago Time Line Apr-09 Jan-09 Oct-08 Jul-08 Apr-08 Jan-08 Oct-07 OC Data? Oversight.
Advertisements

Storage Workshop Summary Wahid Bhimji University Of Edinburgh On behalf all of the participants…
Database Architectures and the Web
SSRS 2008 Architecture Improvements Scale-out SSRS 2008 Report Engine Scalability Improvements.
Introduction to DBA.
RLS Production Services Maria Girone PPARC-LCG, CERN LCG-POOL and IT-DB Physics Services 10 th GridPP Meeting, CERN, 3 rd June What is the RLS -
CERN - IT Department CH-1211 Genève 23 Switzerland t Relational Databases for the LHC Computing Grid The LCG Distributed Database Deployment.
12. March 2003Bernd Panzer-Steindel, CERN/IT1 LCG Fabric status
CERN IT Department CH-1211 Genève 23 Switzerland t Some Hints for “Best Practice” Regarding VO Boxes Running Critical Services and Real Use-cases.
Database Services for Physics at CERN with Oracle 10g RAC HEPiX - April 4th 2006, Rome Luca Canali, CERN.
CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
LHCC Comprehensive Review – September WLCG Commissioning Schedule Still an ambitious programme ahead Still an ambitious programme ahead Timely testing.
ASGC 1 ASGC Site Status 3D CERN. ASGC 2 Outlines Current activity Hardware and software specifications Configuration issues and experience.
CERN - IT Department CH-1211 Genève 23 Switzerland t Tier0 database extensions and multi-core/64 bit studies Maria Girone, CERN IT-PSS LCG.
ATLAS Metrics for CCRC’08 Database Milestones WLCG CCRC'08 Post-Mortem Workshop CERN, Geneva, Switzerland June 12-13, 2008 Alexandre Vaniachine.
3rd June 2004 CDF Grid SAM:Metadata and Middleware Components Mòrag Burgon-Lyon University of Glasgow.
Workshop Summary (my impressions at least) Dirk Duellmann, CERN IT LCG Database Deployment & Persistency Workshop.
LCG 3D Project Status and Production Plans Dirk Duellmann, CERN IT On behalf of the LCG 3D project CHEP 2006, 15th February, Mumbai.
ATLAS Database Operations Invited talk at the XXI International Symposium on Nuclear Electronics & Computing Varna, Bulgaria, September 2007 Alexandre.
OSG Area Coordinator’s Report: Workload Management April 20 th, 2011 Maxim Potekhin BNL
Databases E. Leonardi, P. Valente. Conditions DB Conditions=Dynamic parameters non-event time-varying Conditions database (CondDB) General definition:
Database Administrator RAL Proposed Workshop Goals Dirk Duellmann, CERN.
CERN Physics Database Services and Plans Maria Girone, CERN-IT
9 Systems Analysis and Design in a Changing World, Fourth Edition.
CERN - IT Department CH-1211 Genève 23 Switzerland t Oracle Real Application Clusters (RAC) Techniques for implementing & running robust.
Grid User Interface for ATLAS & LHCb A more recent UK mini production used input data stored on RAL’s tape server, the requirements in JDL and the IC Resource.
CASTOR evolution Presentation to HEPiX 2003, Vancouver 20/10/2003 Jean-Damien Durand, CERN-IT.
CERN-IT Oracle Database Physics Services Maria Girone, IT-DB 13 December 2004.
Database Readiness Workshop Summary Dirk Duellmann, CERN IT For the LCG 3D project SC4 / pilot WLCG Service Workshop.
CERN Database Services for the LHC Computing Grid Maria Girone, CERN.
Hadoop IT Services Hadoop Users Forum CERN October 7 th,2015 CERN IT-D*
SLACFederated Storage Workshop Summary For pre-GDB (Data Access) Meeting 5/13/14 Andrew Hanushevsky SLAC National Accelerator Laboratory.
Ashish Prabhu Douglas Utzig High Availability Systems Group Server Technologies Oracle Corporation.
Rutherford Appleton Lab, UK VOBox Considerations from GridPP. GridPP DTeam Meeting. Wed Sep 13 th 2005.
CERN - IT Department CH-1211 Genève 23 Switzerland t High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN,
Database Competence Centre openlab Major Review Meeting nd February 2012 Maaike Limper Zbigniew Baranowski Luigi Gallerani Mariusz Piorkowski Anton.
3D Project Status Dirk Duellmann, CERN IT For the LCG 3D project Meeting with LHCC Referees, March 21st 06.
Tier-1 Andrew Sansum Deployment Board 12 July 2007.
Last update 29/01/ :01 LCG 1Maria Dimou- cern-it-gd Maria Dimou IT/GD CERN VOMS server deployment LCG Grid Deployment Board
Data Transfer Service Challenge Infrastructure Ian Bird GDB 12 th January 2005.
CERN IT Department CH-1211 Geneva 23 Switzerland t WLCG Operation Coordination Luca Canali (for IT-DB) Oracle Upgrades.
Andrea Valassi (CERN IT-DB)CHEP 2004 Poster Session (Thursday, 30 September 2004) 1 HARP DATA AND SOFTWARE MIGRATION FROM TO ORACLE Authors: A.Valassi,
Maria Girone CERN - IT Tier0 plans and security and backup policy proposals Maria Girone, CERN IT-PSS.
1 A Scalable Distributed Data Management System for ATLAS David Cameron CERN CHEP 2006 Mumbai, India.
Database CNAF Barbara Martelli Rome, April 4 st 2006.
Site Services and Policies Summary Dirk Düllmann, CERN IT More details at
Database Project Milestones (+ few status slides) Dirk Duellmann, CERN IT-PSS (
The Grid Storage System Deployment Working Group 6 th February 2007 Flavia Donno IT/GD, CERN.
D. Duellmann, IT-DB POOL Status1 POOL Persistency Framework - Status after a first year of development Dirk Düllmann, IT-DB.
ARDA Massimo Lamanna / CERN Massimo Lamanna 2 TOC ARDA Workshop Post-workshop activities Milestones (already shown in December)
A quick summary and some ideas for the 2005 work plan Dirk Düllmann, CERN IT More details at
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES The Common Solutions Strategy of the Experiment Support group.
Dominique Boutigny December 12, 2006 CC-IN2P3 a Tier-1 for W-LCG 1 st Chinese – French Workshop on LHC Physics and associated Grid Computing IHEP - Beijing.
CERN - IT Department CH-1211 Genève 23 Switzerland t Service Level & Responsibilities Dirk Düllmann LCG 3D Database Workshop September,
Database Requirements Updates from LHC Experiments WLCG Grid Deployment Board Meeting CERN, Geneva, Switzerland February 7, 2007 Alexandre Vaniachine (Argonne)
Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 6 April 2005.
Oracle structures on database applications development
High Availability Linux (HA Linux)
IT-DB Physics Services Planning for LHC start-up
The “Understanding Performance!” team in CERN IT
Database Services at CERN Status Update
Service Challenge 3 CERN
3D Application Tests Application test proposals
Database Readiness Workshop Intro & Goals
WLCG Service Interventions
Workshop Summary Dirk Duellmann.
Oracle Storage Performance Studies
Database Services for CERN Deployment and Monitoring
Language Processors Application Domain – ideas concerning the behavior of a software. Execution Domain – Ideas implemented in Computer System. Semantic.
Client/Server Computing and Web Technologies
Presentation transcript:

Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 24 January 2005

Services and Support Levels Maria Girone 2 Outline Database Services for Physics at CERN Current problems and limitations Scalable service for the LHC Planning the deployment of physics applications –Defining applications requirements –Optimization –Resource allocation Conclusions

24 January 2005Services and Support Levels Maria Girone 3 Database Services for Physics Mandate –Coordinate the deployment of physics database applications –Administers the physics databases in co-operation with the experiments or grid deployment teams –Consultancy for application design, development and tuning Provide database services for LHC and non-LHC experiments –Grid File Catalogs –Sun Cluster (PDB01) public 2-node for applications related to detector construction, calibration and physics production processing –~30 dedicated Linux disk servers, including COMPASS and HARP

24 January 2005Services and Support Levels Maria Girone 4 Service People and Responsibilities Physics Database Services and CERN T0 in 3D: Maria Girone Link people to experiments/projects: –Grid Deployment & Middleware: Miguel Anjo –ALICE: Jacek Wojcieszuk –ATLAS: Dirk Geppert / Ioannis Papadopoulos (POOL) –CMS & COMPASS: Giacomo Govi –LHCb & HARP: Andrea Valassi Openlab Oracle Fellow: Marta Jakubowska-Sobczak

24 January 2005Services and Support Levels Maria Girone 5 Current Problems and Limitations Service requirements from experiments will increase for LHC start-up preparation –Requirements still uncertain in some areas –Need to plan for a scalable service Current infrastructure based on Sun cluster is already not sufficient to achieve required performance and application isolation –Linux based solution with ORACLE Real Application Cluster looks promising –Need to provide guaranteed resources to key applications Need to identify them with experiments Deploy them only after a validation phase

24 January 2005Services and Support Levels Maria Girone 6 Towards a scalable service for LHC 10g RAC/Linux based service may provide: –Isolation – 10g ‘services’ and / or physical separation –Scalability - in both database processing power and storage –Reliability – automatic failover in case of problems –Manageability – significantly easier to administer than now Hardware in place, under acceptance tests for individual components.. –We set-up a common work-plan across several IT groups..but target date for pre-production is summer 2005 –Need a stop-gap solution until then

24 January 2005Services and Support Levels Maria Girone 7 RAC Work plan Main items view –RAC assembly & config tests: Q –RAC optimization & stability test: Q –Ramping up to production phase: Q –Migration of all applications: Q Web site:

24 January 2005Services and Support Levels Maria Girone 8

24 January 2005Services and Support Levels Maria Girone 9 Stop-gap Solution Several applications which either are high priority or large resource consumers have been identified –Priorities are set by the experiments Allocate dedicated resources to service these applications –One box per experiment for now - propose to move to Oracle 10g –Once ready move to a defined slice of the new service cluster Application owners have been contacted to prepare the move

24 January 2005Services and Support Levels Maria Girone 10 Planning for Application Deployment Introduce a better defined deployment process to insure –Proper planning of database capacity (volume & CPU) –Insure the optimisation of key applications before production starts Propose to separate between two basic application types –Resource consuming applications Eg >20% of CPU, >20% of server memory, >20 concurrent sessions Provide application isolation –Standard applications Smaller database applications which can run in a shared service

24 January 2005Services and Support Levels Maria Girone 11 Service Levels Following Fermilab model, we propose –Development Service Code development, no large data volumes, no backup –Integration and Optimization Service (for key apps) Sufficient resources for larger tests and optimisation Allocated together with consultancy manpower in advance: need 3 month notice –Production Service full production quality service, including backup, monitoring services, on call intervention procedures Monitoring to detect new resource consuming applications or changes in access patterns All Physics Database Service will be based on Oracle 10g

24 January 2005Services and Support Levels Maria Girone 12 A Typical Application Deployment Cycle Development and definition of application requirements –Application development will take place as usual using the database development service –Once sufficiently stable application code moves to a validation phase Schema validation and optimization phase –Performance requirements are needed for key apps How many operations? Which query mix? From how many concurrent clients? Experiments should provide test work load –Work load is run in a specialised validation service until required performance is achieved Resource Allocation & Planning –Optimised application with resource requirements is used to estimate h/w resources required for the service –Total database volume requirements are handled via COCOtime

24 January 2005Services and Support Levels Maria Girone 13 Conclusions Oracle deployment needs for physics are rapidly ramping up –Raising need for consultancy, deployment resources Developing a new service to achieve scalability, availability, isolation Current performance limitations will require stop- gap solution to ensure guaranteed resources to production applications Resource consuming applications need to be identified early in order to schedule validation and optimisation and to allocate guaranteed resources