FroNTier at BNL Implementation and testing of FroNTier database caching and data distribution John DeStefano, Carlos Fernando Gamboa, Dantong Yu Grid Middleware.

Slides:

Advertisements

Similar presentations

Peter Berrisford RAL – Data Management Group SRB Services.

Advertisements

Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.1 PowerSchool 7.0 PowerSchool Application Architecture –PowerSchool 7.0.

GUMS status Gabriele Carcassi PPDG Common Project 12/9/2004.

Synera The Software That Thinks Like You Do Synera Technical Presentation.

DB server limits (process/sessions) Carlos Fernando Gamboa, BNL Andrew Wong, TRIUMF WLCG Collaboration Workshop, CERN Geneva, April 2008.

Local Area Networks Outline –Basic Components of a LAN –Network Architectures –Topologies and LAN Technologies –Selecting a LAN –Improving LAN Performance.

GGF Toronto Spitfire A Relational DB Service for the Grid Peter Z. Kunszt European DataGrid Data Management CERN Database Group.

The Architecture of Transaction Processing Systems

E-business Architecture.NET vs J2EE Judith Molka-Danielsen Feb.27, 2004.

F Fermilab Database Experience in Run II Fermilab Run II Database Requirements Online databases are maintained at each experiment and are critical for.

1 Bridging Clouds with CernVM: ATLAS/PanDA example Wenjing Wu

Microsoft Load Balancing and Clustering. Outline Introduction Load balancing Clustering.

BNL Oracle database services status and future plans Carlos Fernando Gamboa RACF Facility Brookhaven National Laboratory, US Distributed Database Operations.

Dynamics AX Technical Overview Application Architecture Dynamics AX Technical Overview.

Web Application Architecture: multi-tier (2-tier, 3-tier) & mvc

Module 13: Network Load Balancing Fundamentals. Server Availability and Scalability Overview Windows Network Load Balancing Configuring Windows Network.

Chapter 4: Core Web Technologies

HTTP client wide area network (Internet) HTTP proxy HTTP server HTTP gateway firewall HTTP tunnel Copyright Springer Verlag Berlin Heidelberg 2004.

Networked Application Architecture Design. Application Building Blocks Application Software Data Infrastructure Software Local Area Network Server Desktop.

Introduction to distributed systems Dr. S. Indran 23 January 2004.

CS 493/693: Distributed Systems Programming V. “Juggy” Jagannathan CSEE, West Virginia University March 21, 2005.

FroNtier: High Performance Database Access Using Standard Web Components in a Scalable Multi-tier Architecture Marc Paterno Fermilab CHEP 2004 Sept. 27-Oct.

+ discussion in Software WG: Monte Carlo production on the Grid + discussion in TDAQ WG: Dedicated server for online services + experts meeting (Thusday.

Slide 1 Physical Architecture Layer Design Chapter 13.

ATLAS Metrics for CCRC’08 Database Milestones WLCG CCRC'08 Post-Mortem Workshop CERN, Geneva, Switzerland June 12-13, 2008 Alexandre Vaniachine.

Introduction: Distributed POOL File Access Elizabeth Gallas - Oxford – September 16, 2009 Offline Database Meeting.

1 DIRAC – LHCb MC production system A.Tsaregorodtsev, CPPM, Marseille For the LHCb Data Management team CHEP, La Jolla 25 March 2003.

OSG Area Coordinator’s Report: Workload Management April 20 th, 2011 Maxim Potekhin BNL

Databases E. Leonardi, P. Valente. Conditions DB Conditions=Dynamic parameters non-event time-varying Conditions database (CondDB) General definition:

Instrumentation of the SAM-Grid Gabriele Garzoglio CSC 426 Research Proposal.

RAC parameter tuning for remote access Carlos Fernando Gamboa, Brookhaven National Lab, US Frederick Luehring, Indiana University, US Distributed Database.

D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.

Introduction to dCache Zhenping (Jane) Liu ATLAS Computing Facility, Physics Department Brookhaven National Lab 09/12 – 09/13, 2005 USATLAS Tier-1 & Tier-2.

Fermi National Accelerator Laboratory SC2006 Fermilab Data Movement & Storage Multi-Petabyte tertiary automated tape store for world- wide HEP and other.

NOVA Networked Object-based EnVironment for Analysis P. Nevski, A. Vaniachine, T. Wenaus NOVA is a project to develop distributed object oriented physics.

Operated by the Southeastern Universities Research Association for the U.S. Depart. Of Energy Thomas Jefferson National Accelerator Facility Andy Kowalski.

CMS Conditions Data Access using FroNTier Lee Lueking CMS Offline Software and Computing 5 September 2007 CHEP 2007 – Distributed Data Analysis and Information.

09/02 ID099-1 September 9, 2002Grid Technology Panel Patrick Dreher Technical Panel Discussion: Progress in Developing a Web Services Data Analysis Grid.

What is SAM-Grid? Job Handling Data Handling Monitoring and Information.

US ATLAS Tier 1 Facility Rich Baker Brookhaven National Laboratory Review of U.S. LHC Software and Computing Projects Fermi National Laboratory November.

Changes to CernVM-FS repository are staged on an “installation box" using a read/write file system interface. There is a dedicated installation box for.

Ashish Prabhu Douglas Utzig High Availability Systems Group Server Technologies Oracle Corporation.

December 26, 2015 RHIC/USATLAS Grid Computing Facility Overview Dantong Yu Brookhaven National Lab.

BNL Service Challenge 3 Status Report Xin Zhao, Zhenping Liu, Wensheng Deng, Razvan Popescu, Dantong Yu and Bruce Gibbard USATLAS Computing Facility Brookhaven.

CERN - IT Department CH-1211 Genève 23 Switzerland t High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN,

3D Testing and Monitoring Lee Lueking LCG 3D Meeting Sept. 15, 2005.

Testing and integrating the WLCG/EGEE middleware in the LHC computing Simone Campana, Alessandro Di Girolamo, Elisa Lanciotti, Nicolò Magini, Patricia.

BNL Oracle database services status and future plans Carlos Fernando Gamboa, John DeStefano, Dantong Yu Grid Group, RACF Facility Brookhaven National Lab,

ATLAS FroNTier cache consistency stress testing David Front Weizmann Institute 1September 2009 ATLASFroNTier chache consistency stress testing.

BNL dCache Status and Plan CHEP07: September 2-7, 2007 Zhenping (Jane) Liu for the BNL RACF Storage Group.

Replicazione e QoS nella gestione di database grid-oriented Barbara Martelli INFN - CNAF.

Deployment and operations of a high availability infrastructure for relational databases in a heterogeneous Tier 1 workload environment Carlos Fernando.

FIFE Architecture Figures for V1.2 of document. Servers Desktops and Laptops Desktops and Laptops Off-Site Computing Off-Site Computing Interactive ComputingSoftware.

2 Copyright © Oracle Corporation, All rights reserved. Basic Oracle Net Architecture.

A Presentation Presentation On JSP On JSP & Online Shopping Cart Online Shopping Cart.

Dario Barberis: ATLAS DB S&C Week – 3 December Oracle/Frontier and CondDB Consolidation Dario Barberis Genoa University/INFN.

Open Science Grid Consortium Storage on Open Science Grid Placing, Using and Retrieving Data on OSG Resources Abhishek Singh Rana OSG Users Meeting July.

A Scalable and Resilient PanDA Service for Open Science Grid Dantong Yu Grid Group RHIC and US ATLAS Computing Facility.

Towards Dynamic Database Deployment LCG 3D Meeting November 24, 2005 CERN, Geneva, Switzerland Alexandre Vaniachine (ANL)

Fault – Tolerant Distributed Multimedia Streaming Web Application By Nirvan Sagar – Srishti Ganjoo – Syed Shahbaaz Safir

USATLAS Data Transfer Monitoring Dantong Yu Jay Packard John DeStefano Jason Smith.

The Frontier System For ATLAS E.Vamvakopoulos and G.Rahal CAF meeting– CCIN2P3 9 Sep 2011.

Database Replication and Monitoring

Lee Lueking WLCG Workshop DB BoF 22 Jan. 2007

Conditions Data access using FroNTier Squid cache Server

TYPES OF SERVER. TYPES OF SERVER What is a server.

Patrick Dreher Research Scientist & Associate Director

IS 4506 Server Configuration (HTTP Server)

Computer Based Adaptive Testing

Production Manager Tools (New Architecture)

Presentation transcript:

FroNTier at BNL Implementation and testing of FroNTier database caching and data distribution John DeStefano, Carlos Fernando Gamboa, Dantong Yu Grid Middleware Group RHIC/ATLAS Computing Facility Brookhaven National Laboratory April 15, 2009

What is FroNTier? A distribution system for centralized databases Web service provides read access to database Supports N-Tier data distribution layers Developed by Fermilab for CDF data Adapted by CMS at CERN Used for distributing conditions data

FroNTier architecture CMS: FroNTier server with tiered Squids to cache database data Image source: CERN Courier,

FroNTier prerequisites and components FroNTier: Java Tomcat Ant Xerxes XML parser Oracle JDBC driver libraries FroNTier servlet Cache: Squid Custom shared object libraries HW prereqs/recommendations: 100+ GB, 2+ GB, 64-bit

Frontier servlet Squid cache Oracle Conditions database (one node) Remote Squid/Frontier client requests BNL LHC-OPN Initial test bed: single node WAN 2 active connections permitted

Frontier servlet Squid cache (accelerator mode) Oracle Conditions database (one node) Remote Squid/Frontier client requests BNL LHC-OPN Squid cache Initial test bed: two nodes Site server node "Launchpad" node WAN Short expiration time increased from 10 minute to 1 hour

AGLT2 Results: Dual-node configuration Feb. 27 – March 2, muon reconstruction, 3 cosmic data (89507, and 91639) used: o cosmic data jobs, 600 maximum job submission rate from UofM  83 crashes (all from data) o cosmic data jobs, 738 maximum job submission rate from UofM  35 crashes o cosmic data jobs, 738 maximum job submission rate from UofM  82 crashes; test terminated by Squid log size crash o Network glitches may have contributed to results

Frontier servlet Squid cache (accelerator mode) Oracle Conditions database (one node) Remote Squid/Frontier client requests BNL LHC-OPN Current test bed configuration "Launchpad" node WAN Frontier servlet Squid cache (accelerator mode) "Launchpad" node F5 Big-IP Load Balancer

Frontier Clients Squid Cache Frontier Launch Pad Frontier Launch Pad … F5 Server Load Balancing switch rewrites IP header for src. and dest. Addr., IP relay. Frontier Clients Squid Cache Virtual Services Physical Servers VIP Reliable/High Performance Frontier Services 11 Two RAID CONTROLERS Storage Oracle SERVERS

Future plans Additional scalability tests for Reconstruction and Analysis jobs Additional cache nodes Implement Oracle/Frontier triggers to guarantee cache freshness and coherency, based on Oracle data modification time (Dave Dykstra at Fermi, Tier 0 DBAs at CERN)

References and resources CERN Frontier site CERN CMS TWiki: Squid for CMS US ATLAS Admins TWiki: Tier 2 Frontier configuration RACF Frontier at BNL: BNL Frontier Mailing List:

Acknowledgments RACF GCE Robert Petkus, James Pryor RACF Grid Carlos Gamboa, Xin Zhao, Dantong Yu US ATLAS Physics Application Software Shuwei Ye, Torre Wenaus SLAC Wei Yang, Douglas Smith Fermilab Dave Dykstra TRIUMF Rod Walker