Www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 GLUE 2: Deployment and Validation Stephen Burke egi.eu EGI OMB March 26 th.

Slides:



Advertisements
Similar presentations
The Middleware Readiness Working Group LHCb Computing Workshop LHCb Computing Workshop Maria Dimou IT/SDC 2014/05/22.
Advertisements

1 © 2006 Cisco Systems, Inc. All rights reserved. Session Number Presentation_ID Using the Cisco Technical Support & Documentation Website for Security.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks gLite Release Process Maria Alandes Pradillo.
LHCC Comprehensive Review – September WLCG Commissioning Schedule Still an ambitious programme ahead Still an ambitious programme ahead Timely testing.
Using Gstat 2 to Check your Published Information Stephen Burke RAL.
Africa & Arabia ROC tutorial The GSTAT2 Grid Monitoring tool Mario Reale GARR - Italy ASREN-JUNET Grid School - 24 November 2011 Africa & Arabia ROC Tutorial.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EG recent developments T. Ferrari/EGI.eu ADC Weekly Meeting 15/05/
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Migration to the GLUE 2.0 information schema in the LCG/EGEE/EGI.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks GStat 2.0 Joanna Huang (ASGC) Laurence Field.
0 eCPIC Admin Training: OMB Submission Packages and Annual Submissions These training materials are owned by the Federal Government. They can be used or.
Towards a Global Service Registry for the World-Wide LHC Computing Grid Maria ALANDES, Laurence FIELD, Alessandro DI GIROLAMO CERN IT Department CHEP 2013.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks State of Interoperability Laurence Field.
GLite – An Outsider’s View Stephen Burke RAL. January 31 st 2005gLite overview Introduction A personal view of the current situation –Asked to be provocative!
MW Readiness WG Update Andrea Manzi Maria Dimou Lionel Cons 10/12/2014.
Information System Status and Evolution Maria Alandes Pradillo, CERN CERN IT Department, Grid Technology Group GDB 13 th June 2012.
EMI is partially funded by the European Commission under Grant Agreement RI Discovering Infrastructures with EMI Registry (EMIR) Emidio Giorgio.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Using GStat 2.0 for Information Validation.
SAM Sensors & Tests Judit Novak CERN IT/GD SAM Review I. 21. May 2007, CERN.
EMI INFSO-RI EMIR integration in BDII Maria Alandes Pradillo (CERN) Information System Product Team.
Accounting Update John Gordon and Stuart Pullinger January 2014 GDB.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI How to integrate portals with the EGI monitoring system Dusan Vudragovic.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Ops Portal New Requirements.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Middleware Update Maria Alandes Pradillo.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI-InSPIRE APEL for Accounting John Gordon, Stuart Pullinger STFC.
Enabling Grids for E-sciencE INFSO-RI Enabling Grids for E-sciencE Gavin McCance GDB – 6 June 2007 FTS 2.0 deployment and testing.
RI EGI-InSPIRE RI UMD 2 Decommissioning Status Cristina Aiftimiei EGI.eu.
EMI Inter-component and Large Scale Testing Infrastructure Danilo Dongiovanni INFN-CNAF.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Requirements Status EGI.eu UCB
WLCG Operations Coordination report Maria Alandes, Andrea Sciabà IT-SDC On behalf of the WLCG Operations Coordination team GDB 9 th April 2014.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Grid Configuration Data or “What should be.
SAM Status Update Piotr Nyczyk LCG Management Board CERN, 5 June 2007.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Security Monitoring Daniel Kouřil EGI-TF 2011.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Regionalisation summary Prague 1.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The Dashboard for Operations Cyril L’Orphelin.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Storage Accounting John Gordon, STFC OMB August 2013.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Status of ARGUS support Peter Solagna – EGI.eu.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI GGUS Report Generator Günter Grein, KIT Helmut Dres, KIT Torsten Antoni,
WLCG Information System Status Maria Alandes Pradillo, CERN CERN IT Department, Support for Distributed Computing Group GDB 9 th September 2015.
John Gordon EMI TF and EGI CF March 2012 Accounting Workshop.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Update on Service Availability Monitoring (SAM) Marian Babik, David Collados,
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Regional tools use cases overview Peter Solagna – EGI.eu On behalf of the.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI UMD Platform Documentation Álvaro Simón CESGA 02/12/2011 SA2 TF2011, Amsterdam1.
Maria Alandes Pradillo, CERN Training on GLUE 2 information validation EGI Technical Forum September 2013.
RI EGI-InSPIRE RI Puppet community Next steps Peter Solagna – EGI.eu.
Implementation of GLUE 2.0 support in the EMI Data Area Elisabetta Ronchieri on behalf of JRA1’s GLUE 2.0 Working Group INFN-CNAF 13 April 2011, EGI User.
WLCG Operations Coordination Andrea Sciabà IT/SDC GDB 11 th September 2013.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI APEL Regional Accounting Alison Packer (STFC) Iván Díaz Álvarez (CESGA) APEL.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI IPv6 Report for HEPiX CERN October 5, 2012 CERN 1
The Grid Information System Maria Alandes Pradillo IT-SDC White Area Lecture, 4th June 2014.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI MPI VT report OMB Meeting 28 th February 2012.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Information system workshop Stephen Burke egi.eu EGI TF Madrid September.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Current status of ARC integration within monitoring, accounting, and EGI.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Middleware updates Peter Solagna – EGI.eu OMB – 16/07/ /3/
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Accounting task force T. Ferrari/EGI.eu Accounting task force/TCB meeting.
EMI is partially funded by the European Commission under Grant Agreement RI EMI Status And Plans Laurence Field, CERN Towards an Integrated Information.
The evolution of the information system in EGI/WLCG
EGI Operations Management Board
Short term improvements to the Information System: a status report
CREAM Status and Plans Massimo Sgaravatto – INFN Padova
Towards an Integrated Information system: the EMI view
SRM2 Migration Strategy
Outline Introduction Objectives Motivation Expected Output
GLUE 2 Support in gLite Data Management
ARC information in the (top) BDII
EMI: dal Produttore al Consumatore
Operations Officer, EGI
New Types of Accounting Beyond CPU
Stephen Burke egi.eu EGI TF Prague September 20th 2012
Presentation transcript:

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI GLUE 2: Deployment and Validation Stephen Burke egi.eu EGI OMB March 26 th 2013

EGI-InSPIRE RI Overview Current state of GLUE 2 deployment GLUE 2 support by services –Current and expected GLUE 2 profile document Validation –glue-validator –Nagios Next steps GLUE 2 Deployment - EGI OMB2

EGI-InSPIRE RI Deployment status All EGI sites now publish GLUE 2 –Helped by the gLite 3 retirement campaign –Remaining configuration problems all fixed –Should pay attention to new sites – part of certification? –OSG currently has no plans for GLUE 2 Currently have 325 sites, 4306 endpoints, 51 endpoint types GLUE 2 Deployment - EGI OMB3

EGI-InSPIRE RI Service publication All gLite services (+dCache) publish from EMI 2 –Deployment will advance as EMI 1 is retired –Argus only publishes in GLUE 2 ARC integration is part of the EMI 3 top BDII –Also has performance improvements, e.g. cache size and indexing Unicore publishing should come with EMI 3 –Not documented? IGE (GridFTP, gsissh, gram) publishes from the 3.1 release GLUE 2 Deployment - EGI OMB4

EGI-InSPIRE RI EMIR New service from EMI – service registry – /emir-manual.htmlhttp://eu-emi.github.com/emiregistry/documentation/registry /emir-manual.html In EMI 3 all services can be configured to register with it Has been under test for a while – feedback is positive – initiative-registry-solution-federated-online-serviceshttp:// initiative-registry-solution-federated-online-services Need to consider how to deploy and use it in EGI GLUE 2 Deployment - EGI OMB5

EGI-InSPIRE RI Client support WMS now has support for matchmaking with GLUE 2 attributes –Data matching not yet supported, but less important – eleaseNoteshttps://wiki.italiangrid.it/twiki/bin/view/WMS/WMS34ExtendedR eleaseNotes GFAL 2 has support for data management tools –No documentation? ginfo tool in development as a replacement for lcg-info(sites) – –Will release to EPEL after feedback from WLCG GLUE 2 Deployment - EGI OMB6

EGI-InSPIRE RI EGI Profile The draft profile document was presented at the August OMB Defines detailed semantics, extends the schema definitions Classifies attributes by use case and importance –Service Discovery, Service Selection, Monitoring, Oversight, Diagnostic –Mandatory, Recommended, Desirable, Optional, Undesirable –Various implications: need for accuracy, update rate, latency, caching... Detailed validation criteria –FATAL, ERROR, WARNING, INFO GLUE 2 Deployment - EGI OMB7

EGI-InSPIRE RI Profile status Detailed comments received from four people, plus more minor comments from several others –Comments mainly asking for better/more detailed explanations Detailed comments and replies uploaded to DocDB page – Updated document with changes tracked: version 0.9 –Fairly small changes in substance Will do one final review myself, update and create version 1.0 –Hopefully within a couple of weeks GLUE 2 Deployment - EGI OMB8

EGI-InSPIRE RI glue-validator Existing python tool which just checked the basic schema structure Enhanced by Maria Alandes Pradillo –BDII developer and WLCG Information Officer Now implements essentially all validation checks in the profile document –Currently 129 separate tests –Easy to add more Standalone tool, but can output in Nagios format –Flexible configuration GLUE 2 Deployment - EGI OMB9

EGI-InSPIRE RI Middleware validation Started by looking for middleware bugs –Initially using the EMI 3 certification testbed Bugs already submitted for several gLite services –Many services were OK! Continue with ARC and Unicore Also need to look at IGE Record progress on a wiki page – 0_Validationhttps://twiki.cern.ch/twiki/bin/view/EGEE/ISproviders#GLUE_2_ 0_Validation GLUE 2 Deployment - EGI OMB10

EGI-InSPIRE RI Production BDII status Querying the lcg-bdii Top level BDII –273 ERROR messages All related to wrong totals published for the GLUE2StorageShareCapacity and GLUE2StorageServiceCapacity objects –30541 WARNING messages Mainly due to wrong types and mandatory attributes not present –31425 INFO messages Mainly recommended attributes not present and default values (i.e. MaxCPUTime, MaxWallClockTime) Note that information provider validation is ongoing: –Some errors won’t be considered if they come from bugs in the information providers GDB, 13th March 2013

EGI-InSPIRE RI Nagios A previous version of glue-validator is already running on midmon.egi.eu as a test –Some initial problems fixed –Central deployment makes it easier to update Known middleware bugs will be masked in Nagios –Sites should only get bugs for things they can fix Need to move the latest version to midmon and check that the output looks sane Can then consider raising alarms – in a few weeks? GLUE 2 Deployment - EGI OMB12

EGI-InSPIRE RI Other uses glue-validator should also be used in middleware testing and certification Could be run inside the BDII to catch problems before publication –Needs further experience A new version of gstat is in development to support GLUE 2 – useful to display the validation status for the whole Grid –Some tests intrinsically global, e.g. IDs and references GLUE 2 Deployment - EGI OMB13

EGI-InSPIRE RI Next steps - deployment Continue to deploy EMI 2 services Consider pushing deployment of EMI 3 top BDII –Should be an easy upgrade –ARC integration, several performance improvements Test EMIR and consider deployment strategy Test WMS, GFAL and ginfo Test gstat 3 when available NB: GOC DB is also moving to use the GLUE 2 info model –XML rendering but similar to LDAP GLUE 2 Deployment - EGI OMB14

EGI-InSPIRE RI Next steps - validation Review profile document and release “final” version –May well need updates after experience –Document is versioned –Compliance with a given version should be published Release new version of glue-validator and integrate in Nagios Submit bugs to developers and sites as necessary Document known bugs/features GLUE 2 Deployment - EGI OMB15

EGI-InSPIRE RI Summary All sites now publish GLUE 2 information All services can publish First versions of clients ready or on the way Profile document defines detailed validation rules, glue-validator tests them Submit bugs to developers and sites –Test and improve –Should soon be in a better state than we ever had with GLUE 1 This is the year of GLUE 2! GLUE 2 Deployment - EGI OMB16