David Cameron ATLAS Site Jamboree, 20 Jan 2017

Slides:



Advertisements
Similar presentations
1 Bridging Clouds with CernVM: ATLAS/PanDA example Wenjing Wu
Advertisements

Space Science and Engineering Center University of Wisconsin-Madison Virtual Machines: A method for distributing DB processing software Liam Gumley.
Pilots 2.0: DIRAC pilots for all the skies Federico Stagni, A.McNab, C.Luzzi, A.Tsaregorodtsev On behalf of the DIRAC consortium and the LHCb collaboration.
Installation and Integration of Virtual Clusters onto Pragma Grid NAIST Nara, Japan Kevin Lam 06/28/13.
1 port BOSS on Wenjing Wu (IHEP-CC)
UNIT - 1Topic - 2 C OMPUTING E NVIRONMENTS. What is Computing Environment? Computing Environment explains how a collection of computers will process and.
ORGANIZING AND ADMINISTERING OF VOLUNTEER DISTRIBUTED COMPUTING PROJECT Oleg Zaikin, Nikolay Khrapov Institute for System Dynamics and Control.
3rd June 2004 CDF Grid SAM:Metadata and Middleware Components Mòrag Burgon-Lyon University of Glasgow.
Wenjing Wu Andrej Filipčič David Cameron Eric Lancon Claire Adam Bourdarios & others.
Wenjing Wu Computer Center, Institute of High Energy Physics Chinese Academy of Sciences, Beijing BOINC workshop 2013.
David Cameron Claire Adam Bourdarios Andrej Filipcic Eric Lancon Wenjing Wu ATLAS Computing Jamboree, 3 December 2014 Volunteer Computing.
David Cameron Riccardo Bianchi Claire Adam Bourdarios Andrej Filipcic Eric Lançon Efrat Tal Hod Wenjing Wu on behalf of the ATLAS Collaboration CHEP 15,
Ideas for a virtual analysis facility Stefano Bagnasco, INFN Torino CAF & PROOF Workshop CERN Nov 29-30, 2007.
Cloud Status Laurence Field IT/SDC 09/09/2014. Cloud Date Title 2 SaaS PaaS IaaS VMs on demand.
1 The Adoption of Cloud Technology within the LHC Experiments Laurence Field IT/SDC 17/10/2014.
Installing, running, and maintaining large Linux Clusters at CERN Thorsten Kleinwort CERN-IT/FIO CHEP
BOINC: Progress and Plans David P. Anderson Space Sciences Lab University of California, Berkeley BOINC:FAST August 2013.
Easier Platform Administration using SAS 9.4 Grid Option Sets SAS New South Wales User Group - Nov 2015 Andrew Howell ANJ Solutions Pty Ltd.
Using the Weizmann Cluster Nov Overview Weizmann Cluster Connection Basics Getting a Desktop View Working on cluster machines GPU For many more.
DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003 – n o 1 GLUE Schema - Status Report DataTAG WP4 meeting Bologna,
Status of WLCG FCPPL project Status of Beijing site Activities over last year Ongoing work and prospects for next year LANÇON Eric & CHEN Gang.
SPI NIGHTLIES Alex Hodgkins. SPI nightlies  Build and test various software projects each night  Provide a nightlies summary page that displays all.
RAL PPD Tier 2 (and stuff) Site Report Rob Harper HEP SysMan 30 th June
1 Cloud Services Requirements and Challenges of Large International User Groups Laurence Field IT/SDC 2/12/2014.
Shifters Jamboree Kaushik De ADC Jamboree, CERN December 4, 2014.
– Past, Present, Future Volunteer Computing at CERN Helge Meinhard, Nils Høimyr / CERN for the CERN BOINC service team H. Meinhard et al. - Volunteer.
INRNE's participation in LCG Elena Puncheva Preslav Konstantinov IT Department.
OpenNebula: Experience at SZTAKI Peter Kacsuk, Sandor Acs, Mark Gergely, Jozsef Kovacs MTA SZTAKI EGI CF Helsinki.
Capacity Planning in a Virtual Environment Chris Chesley, Sr. Systems Engineer
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
© ExplorNet’s Centers for Quality Teaching and Learning 1 Explain the purpose of Microsoft virtualization. Objective Course Weight 2%
Acronyms GAS - Grid Acronym Soup, LCG - LHC Computing Project EGEE - Enabling Grids for E-sciencE.
Volunteer Clouds and Citizen Cyberscience for LHC Physics Artem Harutyunyan / CERN Carlos Aguado Sanchez / CERN, Jakob Blomer / CERN, Predrag Buncic /
Pledged and delivered resources to ALICE Grid computing in Germany Kilian Schwarz GSI Darmstadt ALICE Offline Week.
Network and Server Basics. Learning Objectives After viewing this presentation, you will be able to: Understand the benefits of a client/server network.
CernVM and Volunteer Computing Ivan D Reid Brunel University London Laurence Field CERN.
Volunteer Computing: Involving the World in Science David P. Anderson U.C. Berkeley Space Sciences Lab February 16, 2007.
All the computers in the world (~1 billion) BOINC: high-level goal Computational science biology, medicine Earth sciences, physics, astronomy, math, A.I.,...
Storage discovery in AliEn
Using Virtualization in the Classroom
Status of WLCG FCPPL project
WLCG Workshop 2017 [Manchester] Operations Session Summary
Update on CERN IT Unified Monitoring Architecture (UMA)
Xiaomei Zhang CMS IHEP Group Meeting December
Volunteer Computing for Science Gateways
Virtualization and Clouds ATLAS position
Dag Toppe Larsen UiB/CERN CERN,
Cluster Optimisation using Cgroups
Dag Toppe Larsen UiB/CERN CERN,
ATLAS Cloud Operations
Outline Benchmarking in ATLAS Performance scaling
External Focus Dr Ivan D Reid Brunel University London 02/09/2016 Ivan D Reid.
BDII Performance Tests
How to enable computing
LCGAA nightlies infrastructure
MCSA VCE
FCT Follow-up Meeting 31 March, 2017 Fernando Meireles
New developments for deploying
CernVM Status Report Predrag Buncic (CERN/PH-SFT).
Discussions on group meeting
WLCG Collaboration Workshop;
This work is supported by projects Research infrastructure CERN (CERN-CZ, LM ) and OP RDE CERN Computing (CZ /0.0/0.0/1 6013/ ) from.
Grid Canada Testbed using HEP applications
This work is partially supported by projects InterExcellence (LTT17018), Research infrastructure CERN (CERN-CZ, LM ) and OP RDE CERN Computing (CZ /0.0/0.0/1.
Ivan Reid (Brunel University London/CMS)
Accounting Repository
The Performance and Scalability of the back-end DAQ sub-system
Backfilling the Grid with Containerized BOINC in the ATLAS computing
Exploit the massive Volunteer Computing resource for HEP computation
Exploring Multi-Core on
Presentation transcript:

David Cameron ATLAS Site Jamboree, 20 Jan 2017 ATLAS@Home Work David Cameron ATLAS Site Jamboree, 20 Jan 2017

ATLAS@Home as a lightweight site solution What is it? Volunteer computing for ATLAS: providing spare CPU on home/office PCs to run ATLAS simulation BOINC software is used to pull jobs from a central server and launch VM 70% of volunteers use Windows Fully integrated with PanDA Tasks are assigned to BOINC_MCORE queue Jobs and data are hosted on BOINC server, volunteers do not talk to Panda/Grid services aCT/ARC CE provides the bridge to PanDA Currently provides 1-2% of simulation resources Grid-wide 20 January 2017 ATLAS@Home, ATLAS site jamboree

ATLAS@Home as a lightweight site solution How? Create account on http://atlasathome.cern.ch Install BOINC client and VirtualBox Connect to ATLAS project (Multicore) jobs start, using as many cores as available Why? Doing good for science Competition for credit (For you) providing much-needed resources for ATLAS 20 January 2017 ATLAS@Home, ATLAS site jamboree

Current volunteers 3 credits ~= 1 event ~= 5-10 mins CPU http://atlasathome.cern.ch/top_users.php Munich office PCs David Smith (CERN IT) Yeti (very helpful sysadmin working somewhere in Germany) CERN cluster used for benchmarking or ATLAS@Home in spare time Unused cluster/UI nodes in Prague 20 January 2017 ATLAS@Home, ATLAS site jamboree

Accounting and recognition The credit system and badges create incentives for competition among the public volunteers The main issue for ATLAS institutes is accounting in terms of Grid resources was largely missing until now Only total numbers for ATLAS@Home were available, not individual volunteers New monitoring infrastructure makes it possible Each panda job has BOINC user id and hostname stored – can filter information by volunteer name or host 20 January 2017 ATLAS@Home, ATLAS site jamboree

Kibana dashboard prototype (link) Big thanks to Ivan Glushkov who set this up 20 January 2017 ATLAS@Home, ATLAS site jamboree

Expression with part of username or hostname 20 January 2017 ATLAS@Home, ATLAS site jamboree

Jobs split by core count 20 January 2017 ATLAS@Home, ATLAS site jamboree

Jobs per host 20 January 2017 ATLAS@Home, ATLAS site jamboree

No of events 20 January 2017 ATLAS@Home, ATLAS site jamboree

To add Efficiency Pie-charts/cumulative plots 20 January 2017 ATLAS@Home, ATLAS site jamboree

Summary ATLAS@Home is a very easy way for sites with a small amount of spare resources to contribute to ATLAS Office PCs, old machines, under-used machines, new machines awaiting commissioning, … Just install 2 pieces of software on each node No CE, SE, etc services required Easily configurable when to run – e.g. only 5pm – 8am, only when CPU usage is < 25% Jobs can be suspended and resumed later ATLAS job accounting can be done as with any normal site One caveat: cannot run a VM inside another VM Native linux version using containers is on the todo list 20 January 2017 ATLAS@Home, ATLAS site jamboree