HLRmon accounting portal DGAS (Distributed Grid Accounting System) sensors collect accounting information at site level. Site data are sent to site or.

Slides:



Advertisements
Similar presentations
Futures – Alpha Cloud Deployment and Application Management.
Advertisements

Consorzio COMETA - PI2S2 Project UNIONE EUROPEA SAGE – Storage Accounting for Grid Environments in gLite Fabio Scibilia Consorzio.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Makrand Siddhabhatti Tata Institute of Fundamental Research Mumbai 17 Aug
Accounting Update Dave Kant Grid Deployment Board Nov 2007.
Sergey Belov, Tatiana Goloskokova, Vladimir Korenkov, Nikolay Kutovskiy, Danila Oleynik, Artem Petrosyan, Roman Semenov, Alexander Uzhinskiy LIT JINR The.
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
Storage Accounting John Gordon, STFC GDB June 2012.
Summary of Accounting Discussion at the GDB in Bologna Dave Kant CCLRC, e-Science Centre.
A.Guarise – F.Rosso 1 Enabling Grids for E-sciencE INFSO-RI Comprehensive Accounting Views on large computing farms. Andrea Guarise & Felice Rosso.
Monitoring in EGEE EGEE/SEEGRID Summer School 2006, Budapest Judit Novak, CERN Piotr Nyczyk, CERN Valentin Vidic, CERN/RBI.
Database weekly reports Zbigniew Baranowski Carlos Fernando Gamboa.
Optimizer Deployment Centralized Database module on Optimizer hub server Each monitored server has an instance of optimizer installed.
Cracow Grid Workshop October 2009 Dipl.-Ing. (M.Sc.) Marcus Hilbrich Center for Information Services and High Performance.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks WMSMonitor: a tool to monitor gLite WMS/LB.
Accounting in LCG Dave Kant CCLRC, e-Science Centre.
Job and Data Accounting on the Open Science Grid Ruth Pordes, Fermilab with thanks to Brian Bockelman, Philippe Canal, Chris Green, Rob Quick.
GDB March User-Level, VOMS Groups and Roles Dave Kant CCLRC, e-Science Centre.
Overview of Privilege Project at Fermilab (compilation of multiple talks and documents written by various authors) Tanya Levshina.
US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.
INFSO-RI Enabling Grids for E-sciencE EGEE is a project funded by the European Union under contract INFSO-RI Grid Accounting.
O PERATING S YSTEM. What is an Operating System? An operating system is an event driven program which acts as an interface between a user of a computer,
Glite. Architecture Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware Higher-Level Grid Services are supposed.
INFSO-RI Enabling Grids for E-sciencE GridICE: Grid and Fabric Monitoring Integrated for gLite-based Sites Sergio Fantinel INFN.
EMI INFSO-RI Accounting John Gordon (STFC) APEL PT Leader.
Recent improvements in HLRmon, an accounting portal suitable for national Grids Enrico Fattibene (speaker), Andrea Cristofori, Luciano Gaido, Paolo Veronesi.
Automatic Resource & Usage Monitoring Steve Traylen/Flavia Donno CERN/IT.
Accounting Update John Gordon and Stuart Pullinger January 2014 GDB.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks APEL CPU Accounting in the EGEE/WLCG infrastructure.
LCG WLCG Accounting: Update, Issues, and Plans John Gordon RAL Management Board, 19 December 2006.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Storage Accounting for Grid Environments Fabio Scibilia INFN - Catania.
Global ADC Job Monitoring Laura Sargsyan (YerPhI).
LCG Accounting Update John Gordon, CCLRC-RAL WLCG Workshop, CERN 24/1/2007 LCG.
INFSO-RI Enabling Grids for E-sciencE Policy management and fair share in gLite Andrea Guarise HPDC 2006 Paris June 19th, 2006.
Accounting in LCG/EGEE Can We Gauge Grid Usage via RBs? Dave Kant CCLRC, e-Science Centre.
ROC managers meeting, Barcelona, Luciano Gaido (thanks to Paolo Veronesi for the slides) ROC-IT status.
HLRmon accounting portal The accounting layout A. Cristofori 1, E. Fattibene 1, L. Gaido 2, P. Veronesi 1 INFN-CNAF Bologna (Italy) 1, INFN-Torino Torino.
Gennaro Tortone, Sergio Fantinel – Bologna, LCG-EDT Monitoring Service DataTAG WP4 Monitoring Group DataTAG WP4 meeting Bologna –
INFN GRID Production Infrastructure Status and operation organization Cristina Vistoli Cnaf GDB Bologna, 11/10/2005.
INFSO-RI Enabling Grids for E-sciencE DGAS, current status & plans Andrea Guarise EGEE JRA1 All Hands Meeting Plzen July 11th, 2006.
OSG Status and Rob Gardner University of Chicago US ATLAS Tier2 Meeting Harvard University, August 17-18, 2006.
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
INFN/IGI contributions Federated Clouds Task Force F2F meeting November 24, 2011, Amsterdam.
TIFR, Mumbai, India, Feb 13-17, GridView - A Grid Monitoring and Visualization Tool Rajesh Kalmady, Digamber Sonvane, Kislay Bhatt, Phool Chand,
G. Russo, D. Del Prete, S. Pardi Kick Off Meeting - Isola d'Elba, 2011 May 29th–June 01th A proposal for distributed computing monitoring for SuperB G.
HLRmon Enrico Fattibene INFN-CNAF 1EGI-TF Lyon, France19-23 September 2011.
DGAS Distributed Grid Accounting System INFN Workshop /05/1009, Palau Giuseppe Patania Andrea Guarise 6/18/20161.
WLCG Accounting Task Force Update Julia Andreeva CERN GDB, 8 th of June,
Using HLRmon for advanced visualization of resource usage Enrico Fattibene INFN - CNAF ISCG 2010 – Taipei March 11 th, 2010.
Enabling Grids for E-sciencE INFN Workshop – May 7-11 Rimini 1 Grid Accounting Status at INFN Riccardo Brunetti INFN-TORINO.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Storage Accounting John Gordon STFC GDB, Lyon 6 th April2011 GDB January 2012.
Enabling Grids for E-sciencE Claudio Cherubino INFN DGAS (Distributed Grid Accounting System)
DGAS Accounting – toward national grid infrastructures HPDC workshop on Monitoring, Logging and Accounting, (MLA) in production Grids 10/06/2009, Munich.
Accounting Update Dave Kant, John Gordon RAL Javier Lopez, Pablo Rey Mayo CESGA.
EGEE is a project funded by the European Union under contract INFSO-RI DGAS Grid accounting L.Gaido on behalf of A.Guarise LCG Workshop November.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI solution for high throughput data analysis Peter Solagna EGI.eu Operations.
How to integrate portals with EGI accounting system R.Graciani EGI TF 2012.
AuthN and AuthZ in StoRM A short guide
Job monitoring and accounting data visualization
DGAS A.Guarise April 19th, Athens
POW MND section.
Accounting at the T1/T2 Sites of the Italian Grid
Advancements in Availability and Reliability computation Introduction and current status of the Comp Reports mini project C. Kanellopoulos GRNET.
Giuseppe Patania Nov, Martina Franca (Ta)‏
Ákos Frohner EGEE'08 September 2008
WLCG dedicated accounting portal (proposal)
HLRmon accounting portal
User Accounting Integration Spreading the Net.
Presentation transcript:

HLRmon accounting portal DGAS (Distributed Grid Accounting System) sensors collect accounting information at site level. Site data are sent to site or multi-site HLRs (Home Location Registers). A top level HLR receives accounting data from all the Grid HLRs. HLRmon retrieves data from the top level HLR and presents them through a web interface. Aggregated data are also sent to the central EGI Accounting Portal. HLRmon is successfully deployed as accounting portal for IGI and other Grid projects/NGIs (National Grid Initiatives) using DGAS as accounting system. General purpose CPU reports Different metrics to analyze: CPU and Wall Clock Time, expressed in days and normalized on the mean power capacity of the site Number of executed jobs Job efficiency in terms of (CPU Time / Wall Time) ratio Different keys of data aggregation: Site Virtual Organization VOMS group and role Certification Authority and Registration Authority Job type (Grid or local) Grid user DN The charts allow the analysis of CPU and storage resources usage at the WLCG Tier1 and Tier2 by the LHC experiments, over the last month and the last year. The resource usage and the amount of pledged resources are compared. Accounting data are grouped by site and by VO. Storage data are taken by a script running on the sites and sent to HLRmon via . Architecture Storage reports Non-standard schema is used to generate records (no standard schema exists). The information on the used space is collected from the BDII once a day. Storage URs (Usage Records) are stored on the HLR server as the computing URs. HLRmon is able to retrieve storage information directly from the desired HLR. The portal shows the daily disk consumption in terms of used space. Data are aggregated per site, SE and Storage Area (SA). Used space will be compared to the total available space (at SA, SE and site level). Collectors retrieve:  aggregated CPU accounting information from a desired HLR  storage accounting data by probes running on the SRM instances at WLCG sites  values of some GLUE attributes about the computing power capacity of the WLCG sites Used space per site Used space per site per day Used space per SA Used per SA per day A. Cristofori 1, E. Fattibene 1, L. Gaido 2, P. Veronesi 1 - INFN-CNAF Bologna (Italy) 1, INFN-Torino Torino (Italy) An aggregation layer is responsible for the data manipulation to produce charts and tabular reports visualized by the graphical interface. Data aggregated per Grid user are directly retrieved from the HLR database at run-time and are just accessible by authorized users. The accounting layout An advanced selection menu permits to choose a desired subgroup of values for each aggregation key. A specific view with data grouped per discipline is provided: the mapping information between VOs and disciplines is currently retrieved from the EGI Operations Portal. WLCG reports D istributed G rid A ccounting S ystem