15th January 2014 Notes not yet edited!

Slides:



Advertisements
Similar presentations
Pilots 2.0: DIRAC pilots for all the skies Federico Stagni, A.McNab, C.Luzzi, A.Tsaregorodtsev On behalf of the DIRAC consortium and the LHCb collaboration.
Advertisements

LCG Milestones for Deployment, Fabric, & Grid Technology Ian Bird LCG Deployment Area Manager PEB 3-Dec-2002.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EG recent developments T. Ferrari/EGI.eu ADC Weekly Meeting 15/05/
European Grid Initiative Federated Cloud update Peter solagna Pre-GDB Workshop 10/11/
EMI INFSO-RI EMI Structure, Plans, Deliverables Alberto Di Meglio (CERN) Project Director ATLAS Software & Computing Week Geneva, 21 July 2011.
Virtualised Worker Nodes Where are we? What next? Tony Cass GDB /12/12.
GDB July 2015 Jeremy’s quick summary notes Also refer to the meeting minutes
CASTOR evolution Presentation to HEPiX 2003, Vancouver 20/10/2003 Jean-Damien Durand, CERN-IT.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI (Present and) Future of the EGI Services for WLCG Peter Solagna – EGI.eu.
Trusted Virtual Machine Images a step towards Cloud Computing for HEP? Tony Cass on behalf of the HEPiX Virtualisation Working Group October 19 th 2010.
WLCG Overview Board, September 3 rd 2010 P. Mato, P.Buncic Use of multi-core and virtualization technologies.
Storage Accounting John Gordon, STFC GDB March 2013.
LCG Introduction John Gordon, STFC GDB June 8 th 2011.
Accounting Update John Gordon and Stuart Pullinger January 2014 GDB.
WLCG Technical Evolution Group: Operations and Tools Maria Girone & Jeff Templon Kick-off meeting, 24 th October 2011.
Workload management, virtualisation, clouds & multicore Andrew Lahiff.
December GDB Brief summary – J Coles. Meetings January meeting moved to 15 th 2014 events created. Check March meeting outside CERN. Copenhagen workshop.
Enabling Grids for E-sciencE INFSO-RI Enabling Grids for E-sciencE Gavin McCance GDB – 6 June 2007 FTS 2.0 deployment and testing.
Why a Commercial Provider should Join the Academic Cloud Federation David Blundell Managing Director 100 Percent IT Ltd Simple, Flexible, Reliable.
WLCG Operations Coordination report Maria Alandes, Andrea Sciabà IT-SDC On behalf of the WLCG Operations Coordination team GDB 9 th April 2014.
MW Readiness WG Update Andrea Manzi Maria Dimou Lionel Cons Maarten Litmaath On behalf of the WG participants GDB 09/09/2015.
Accounting Review Summary from the pre-GDB related to CPU (wallclock) accounting Julia Andreeva CERN-IT GDB 13th April
Accounting John Gordon WLC Workshop 2016, Lisbon.
Outcome should be a documented strategy Not everything needs to go back to square one! – Some things work! – Some work has already been (is being) done.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Regionalisation summary Prague 1.
WLCG Accounting Task Force Update Julia Andreeva CERN GDB, 8 th of June,
Trusted Virtual Machine Images the HEPiX Point of View Tony Cass October 21 st 2011.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Storage Accounting John Gordon, STFC OMB August 2013.
John Gordon EMI TF and EGI CF March 2012 Accounting Workshop.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.
Accounting Update John Gordon. Outline Multicore CPU Accounting Developments Cloud Accounting Storage Accounting Miscellaneous.
Maria Alandes Pradillo, CERN Training on GLUE 2 information validation EGI Technical Forum September 2013.
HEPiX IPv6 Working Group David Kelsey (STFC-RAL) GridPP33 Ambleside 22 Aug 2014.
WLCG Operations Coordination Andrea Sciabà IT/SDC GDB 11 th September 2013.
EGI-InSPIRE RI EGI-InSPIRE RI EGI-InSPIRE Software provisioning and HTC Solution Peter Solagna Senior Operations Manager.
EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number Federated Cloud Update.
LCG Introduction John Gordon, STFC GDB December 7 th 2010.
The Grid Information System Maria Alandes Pradillo IT-SDC White Area Lecture, 4th June 2014.
HEPiX Virtualisation working group Andrea Chierici INFN-CNAF Workshop CCR 2010.
Accounting Review Summary and action list from the (pre)GDB Julia Andreeva CERN-IT WLCG MB 19th April
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI solution for high throughput data analysis Peter Solagna EGI.eu Operations.
Bob Jones EGEE Technical Director
Dynamic Extension of the INFN Tier-1 on external resources
WLCG IPv6 deployment strategy
Review of the WLCG experiments compute plans
Laurence Field IT/SDC Cloud Activity Coordination
WLCG Workshop 2017 [Manchester] Operations Session Summary
OpenSSL and Java 7 vs. 512-bit proxy keys
John Gordon STFC OMB 26 July 2011
Sviluppi in ambito WLCG Highlights
Virtualization and Clouds ATLAS position
Progress on NA61/NA49 software virtualisation Dag Toppe Larsen Wrocław
gLite->EMI2/UMD2 transition
ATLAS Cloud Operations
Ian Bird GDB Meeting CERN 9 September 2003
CREAM Status and Plans Massimo Sgaravatto – INFN Padova
WLCG experiments FedCloud through VAC/VCycle in the EGI
How to enable computing
Savannah to Jira Migration
John Gordon, STFC-RAL GDB March 11, 2009
Short update on the latest gLite status
Outline Introduction Objectives Motivation Expected Output
EMI: dal Produttore al Consumatore
Discussions on group meeting
WLCG Collaboration Workshop;
Solutions for federated services management EGI
Ian Bird LCG Project - CERN HEPiX - FNAL 25-Oct-2002
New Types of Accounting Beyond CPU
Exploring Multi-Core on
Operations Management Board March 26
Presentation transcript:

15th January 2014 Notes not yet edited! January GDB 15th January 2014 Notes not yet edited!

2013-14 Meetings Second Wednesday each month March 12th hosted by CNAF, Bolgna Next WLCG workshop – July? Volunteers. Future: jobs with high-memory profiles; WLCG consolidation update; EMI-3 deployment plans; MW automatic validation. Next pre-GDBs: Ops coord; batch system support; HT condor; remote data access/federation Actions in progress. Site NAGIOS testing New SAM tests. WLCG now using VO-based SAM for availability/reliability. Discussion about SAM test scheduling. Timeouts must be ‘handled’ and consistently by experiments. SAM tests are not usin pilot jobs. Proposal – run tests as lcgadmin. Apply scheduling policy. Additional tests run as “normal” but not ‘critical’. There are known issues. Who cares about the test results? HEPiX IPv6 WG next Thursday-Friday WLCG ops coordination – 11th Feb LHCOPN evolution 10th-11th Feb ISGC March 22nd-28th HEPiX, Annecy 19th-23rd May EGI community forum, Helsinki 19th-23rd May WLCG workshop July?

Accounting update CPU accounting EMI-3 clients: supports HEPSPEC06. Richer data (e.g. MPI and submit host). Portal support for new data coming… Client was rewritten in Python. (Condor+ARC (uses JURA)). EGI-Inspire funding ends 2014. Will continue cloud and storage accounting. Multicore only seems ready on LSF and - Storage DPM and dCache released StAR publishers in EMI3. EGI federated cloud accounting. Production service by April. Issues: normalisation (use machine features?); merging grid and cloud data; inclusion of commercial data; give a global view; alternatives to infrastructure accounting. NEXT: look at utilisation for VMs. Encourage ‘machine features’ to invlude benchmark. Postscript: Glue 2 -> key value pairs in GOCDB 5.2 can be used to define custom key properties to define attributes. Allows definition of class of sites for example.

Volunteer computing Why? Outreach & resource Resources have different constraints Middleware: BOINC, XtremWeb. Condor. Berkeley Open Infrastructure Network Computing – boinc.berkeley.edu High CPU + small data = good match. Avoid porting issues by running under CernVM and VirtualBox with the BOINC client. Federico Stagni – LHCb first steps Issues: CVMFS (local cache has to be populated); job length; accounting; clients

EGI plans for future activities EGI-Inspire extension approved. Cloud and business development (e.g. pay for use and commercial resources): Federated Cloud services in production from May 2014. Cooperation between European infrastructures AAI: uniform authentication and authorization. Plan to run a pilot to access cloud services with federated IdP credentials. ICT – deadline April 2020. Cloud based proposal.

Operational updates from EGI SHA-2. Mostly done. dCache SRM clients released but not yet in UMD. Proposal to increase A/R threshold. A-> 80% and R->85%. Security contacts verification.

Toward a new HEP CPU Benchmark HS06 – for benchmark. Pledges. Procurement. Accounting. Hardware changes -> benchmark should evolve Open Systems Group working on next version HS14 requirements. Ease of use and “good enough” Benchmark – CPUv6/CPU2014. Final benchmark end 2014. Non-WLCG HEP Vos could/should join benchmark validation team? Agreement on h/w platform, compiler, compiler flags

DAVIX library Requirements for HPC Security: Reliability: I/O framework: Easy to deploy: generic Davix is best fit from existing clients Goal – tool for high performance I/O Features: Optimisations; Security; Reliability; DM extensions; integration with ROOT (in EPEL6 under root-net-davix)

Cloud pre-GDB WG explore possible use of private/community clouds. Focus on shared clouds. Build on existing work in experiments. Last meeting July. Review progress/work in cloud usage. Review implementation of graceful VM termination. Continue discussion possible models for non-static sharing of resources. Accounting: prototype based on APEL. How to normalize a VM CPU power. WC time vs CPU time – record both but CPU time provides homogeneity. Security/Traceability. Current model based on 2 roles (endorser/vm operator) that must be distinct. Traceability – need central logs and may need to add items like VM instantiated. Absence of root access for end users remains critical. User compartmentalization required on “multi-user VMs” Vulnerability handling requires ability to terminate VMs quickly No discussion of policy … Target shares: economic model complicated… without that keep CE/batch system. Main difficulties – no queuing of requests and allocation of free slots does not address quotas. Proposal to start long lived VMs for a VO up to target share.

Actions in progress Details lost…. IPv6 WMS decommissioning – glide-in submission encouraged. VOs using WMS will be contracted (e.g. LHCb) Tracking tools – Savannah to JIRA slow progress Machine job features – available at CERN. Need to make available on OPenStack Cloud environment. Provide client for experiments to test. Middleware readiness. Focus was on software repositoties and the criteria for juding a product deployment process successful Multicore – twiki and mailing list. https://twiki.cern.ch/twiki/bin/view/LCG/DeployMultiCore.

Openssl & JAVA 7 GGUS 98964 (Nov 18th). WMS delegates proxies with 512-bit keys that are refused by Java 7 used by dCache. Change key size >1024 in java.security. Dec 9th: ATLAS reported problem with openssl-1.0e-16.el6_5. LCG-ROLLOUT thread. If both endpoints running newest openssl they negotiate TLS1.2 which does not support 512-bit keys. If either end is older then use TLS1.1 and that is ok with 512 buts. Java 7 by default refuses such keys. Delegations coming from GridSite libraries: WMS, CREAM, UI, FTS-3, PanDa. New GridSite rpms available as of 16th Dec. Which nodes need fix: WMS (next Tuesday at CERN), FTS-3. PanDA. CREAM