LCG/gLite BDII performance measurements Lev Shamardin Scobeltsyn Institute of Nuclear Physics, Moscow State University.

Slides:

Advertisements

Similar presentations

Network II.5 simulator ..

Advertisements

Fast Data at Massive Scale Lessons Learned at Facebook Bobby Johnson.

Applications Area Issues RWL Jones GridPP13 – 5 th June 2005.

Databasteknik Databaser och bioinformatik Data structures and Indexing (II) Fang Wei-Kleiner.

Operating System.

Indications in green = Live content Indications in white = Edit in master Indications in blue = Locked elements Indications in black = Optional elements.

Channel Access Protocol Andrew Johnson Computer Scientist, AES Controls Group.

Cache Coherent Distributed Shared Memory. Motivations Small processor count –SMP machines –Single shared memory with multiple processors interconnected.

Computer Memory GCSE Computing.

1 Content Delivery Networks iBAND2 May 24, 1999 Dave Farber CTO Sandpiper Networks, Inc.

1 Measurement and Analysis of LDAP Performance Xin Wang( ), Henning Schulzrinne, Dilip Kandlur, Dinesh Verma.

Extensible Scalable Monitoring for Clusters of Computers Eric Anderson U.C. Berkeley Summer 1997 NOW Retreat.

1 CAPS: A Peer Data Sharing System for Load Mitigation in Cellular Data Networks Young-Bae Ko, Kang-Won Lee, Thyaga Nandagopal Presentation by Tony Sung,

70-293: MCSE Guide to Planning a Microsoft Windows Server 2003 Network, Enhanced Chapter 7: Planning a DNS Strategy.

Cross Cluster Migration Remote access support Adianto Wibisono supervised by : Dr. Dick van Albada Kamil Iskra, M. Sc.

INFN experience with Globus GIS A. Cavalli - F. Semeria INFN Grid Information Services workshop CERN, March 2001.

Application Layer  We will learn about protocols by examining popular application-level protocols  HTTP  FTP  SMTP / POP3 / IMAP  Focus on client-server.

Database System Architectures  Client-server Database System  Parallel Database System  Distributed Database System Wei Jiang.

Distributed Computer Architecture Benjamin Jordan, Kevin Cone, Jason Bradley.

Common Services in a network Server : provide services Type of Services (= type of servers) –file servers –print servers –application servers –domain servers.

1 Lecture 7: Data structures for databases I Jose M. Peña

Research on cloud computing application in the peer-to-peer based video-on-demand systems Speaker : 吳靖緯 MA0G rd International Workshop.

1 3 Web Proxies Web Protocols and Practice. 2 Topics Web Protocols and Practice WEB PROXIES  Web Proxy Definition  Three of the Most Common Intermediaries.

Slow Web Site Problem Analysis Last Update Copyright 2013 Kenneth M. Chipps Ph.D. 1.

Operating System. Architecture of Computer System Hardware Operating System (OS) Programming Language (e.g. PASCAL) Application Programs (e.g. WORD, EXCEL)

Application-Layer Anycasting By Samarat Bhattacharjee et al. Presented by Matt Miller September 30, 2002.

ATLAS DQ2 Deletion Service D.A. Oleynik, A.S. Petrosyan, V. Garonne, S. Campana (on behalf of the ATLAS Collaboration)

Test Of Distributed Data Quality Monitoring Of CMS Tracker Dataset H->ZZ->2e2mu with PileUp - 10,000 events ( ~ 50,000 hits for events) The monitoring.

GLite D. Zaborov, SA1-SA3 meeting, Dubna1 Dmitry Zaborov (SINP-MSU-Moscow) Dubna, Jul 26, 2007 Testing 3.0 under SLC 4.

2: Application Layer1 Chapter 2 outline r 2.1 Principles of app layer protocols r 2.2 Web and HTTP r 2.3 FTP r 2.4 Electronic Mail r 2.5 DNS r 2.6 Socket.

Dynamic Content On Edge Cache Server (using Microsoft.NET) Name: Aparna Yeddula CS – 522 Semester Project Project URL: cs.uccs.edu/~ayeddula/project.html.

DBI313. MetricOLTPDWLog Read/Write mixMostly reads, smaller # of rows at a time Scan intensive, large portions of data at a time, bulk loading Mostly.

Grid infrastructure analysis with a simple flow model Andrey Demichev, Alexander Kryukov, Lev Shamardin, Grigory Shpiz Scobeltsyn Institute of Nuclear.

Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.

June 24-25, 2008 Regional Grid Training, University of Belgrade, Serbia Introduction to gLite gLite Basic Services Antun Balaž SCL, Institute of Physics.

The Replica Location Service The Globus Project™ And The DataGrid Project Copyright (c) 2002 University of Chicago and The University of Southern California.

Fast Crash Recovery in RAMCloud. Motivation The role of DRAM has been increasing – Facebook used 150TB of DRAM For 200TB of disk storage However, there.

ROOT and Federated Data Stores What Features We Would Like Fons Rademakers CERN CC-IN2P3, Nov, 2011, Lyon, France.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Performance Improvements to BDII - Grid Information.

Reliable File Transfer: Lessons Learned Bill Allcock, ANL Ravi Madduri, ANL.

ICP and the Squid Web Cache Duane Wessels and K. Claffy 산업공학과 조희권.

1 Xin Wang Internet Real -Time Laboratory Internet Real -Time Laboratory Columbia University ( Joint work with Henning Schulzrinne, Dilip Kandlur, and.

ITGS Network Architecture. ITGS Network architecture –The way computers are logically organized on a network, and the role each takes. Client/server network.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Site Architecture Resource Center Deployment Considerations MIMOS EGEE Tutorial.

LCG LCG-1 Deployment and usage experience Lev Shamardin SINP MSU, Moscow

JAliEn Java AliEn middleware A. Grigoras, C. Grigoras, M. Pedreira P Saiz, S. Schreiner ALICE Offline Week – June 2013.

Distributed Logging Facility Castor External Operation Workshop, CERN, November 14th 2006 Dennis Waldron CERN / IT.

Scalable Data Scale #2 site on the Internet (time on site) >200 billion monthly page views Over 1 million developers in 180 countries.

GIIS Implementation and Requirements F. Semeria INFN European Datagrid Conference Amsterdam, 7 March 2001.

Hiearchial Caching in Traffic Server. Hiearchial Caching  A set of techniques and mechanisms to increase the size and performance of network caches.

09/13/04 CDA 6506 Network Architecture and Client/Server Computing Peer-to-Peer Computing and Content Distribution Networks by Zornitza Genova Prodanoff.

Parallelizing Functional Tests for Computer Systems Using Distributed Graph Exploration Alexey Demakov, Alexander Kamkin, and Alexander Sortov

The GridPP DIRAC project DIRAC for non-LHC communities.

WMS baseline issues in Atlas Miguel Branco Alessandro De Salvo Outline  The Atlas Production System  WMS baseline issues in Atlas.

NCBI Grid Presentation. NCBI Grid Structure NetCache NetSchedule Load Balancer (LBSM) Load Balancer (LBSM) Worker Nodes CGI Gateway.

VO Box discussion ATLAS NIKHEF January, 2006 Miguel Branco -

Design and Implementation of a High-Performance distributed web crawler Vladislav Shkapenyuk and Torsten Suel Proc. 18 th Data Engineering Conf., pp ,

تجارت الکترونیک سیار جلسه پنجم مدرس : دکتررامین کریمی.

Web and Proxy Server.

Remote execution of long-running CGIs

Affinity Depending on the application and client requirements of your Network Load Balancing cluster, you can be required to select an Affinity setting.

Cellular IP: A New Approach to Internet Host Mobility

BDII Performance Tests

Data Management cluster summary

Distributed Content in the Network: A Backbone View

Parallel and Multiprocessor Architectures – Shared Memory

A Network Operating System Edited By Maysoon AlDuwais

PEER-TO-PEER SYSTEMS.

Information System (BDII)

Caching 50.5* + Apache Kafka

Presentation transcript:

LCG/gLite BDII performance measurements Lev Shamardin Scobeltsyn Institute of Nuclear Physics, Moscow State University

Lev Shamardin 2 What is BDII really? OpenLDAP with LDBM backend Three running servers with role rotation:  One is updated every minute with a set of processed ldapsearch results from any grid site.  One is used to serve the requests via a port forwarding proxy  One is setteling down

Lev Shamardin 3 BDII users Primary users:  lfc-* utilities looking for storage elements  Resource Brokers updating grid information cache Some other grid utilities, but these are much less popular then the first two.

Lev Shamardin 4 lfc utilities Typcal query:  (GlueSEUniqueID= )  (&(GlueServiceURI=* *) (GlueServiceType=srm_v1)) Results:  Several rows for each query

Lev Shamardin 5 WMS cache update Typcal query:  (|(objectClass=gluevoview) (|(objectClass=gluecesebind) (|(objectClass=gluece) (|(objectClass=gluecluster) (objectClass=gluesubcluster))))) Results:  Huge dump of grid resources, ~15-20 Mbytes.

Lev Shamardin 6 Testing method A number of parallel quering processes, with synchronized start. Each process runs a number of query sets in a random sequence. Results from each process are collected. First implementation in perl using Net::LDAP appeared to be amazingly slow and memory- hungry, so written in C with openldap-client libs.

Lev Shamardin 7 SE search queries There is almost no dependence of the response time from the number of sequential queries

Lev Shamardin 8 SE search queries (2)

Lev Shamardin 9 SE search queries (3)

Lev Shamardin 10 WMS queries

Lev Shamardin 11 WMS queries (2)

Lev Shamardin 12 Conclusion Seems to scale linearly. Just throw in more CPUs? High loads cause connection timeouts, but our test show that „high load“ for a production BDII means >1000 simulteneous queries! Protocol and implementation are quite inefficient. Network delay for the transfer of WMS dump data is ~2 sec, so could be the response time for the sequential same queries.

Lev Shamardin 13 Conclusion (2) Numbers for the modeling (next talk). Net::LDAP is very slow.

Lev Shamardin 14 Acknowledgements The research was partially supported by  INTAS-CERN Grant  RFBR Grant