Summer Student presentation Changing Dashboard build system to Bamboo Robert Varga IT/SDC 29-07-13.

Slides:

Advertisements

Similar presentations

1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.

Advertisements

I.1 ii.2 iii.3 iv.4 1+1=. i.1 ii.2 iii.3 iv.4 1+1=

I.1 ii.2 iii.3 iv.4 1+1=. i.1 ii.2 iii.3 iv.4 1+1=

BizTalk Deployment using Visual Studio Release Management

Automated Tests in NICOS Nightly Control System Alexander Undrus Brookhaven National Laboratory, Upton, NY Software testing is a difficult, time-consuming.

Analysis demos from the experiments. Analysis demo session Introduction –General information and overview CMS demo (CRAB) –Georgia Karapostoli (Athens.

LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.

Framework for Automated Builds Natalia Ratnikova CHEP’03.

Nightly Releases and Testing Alexander Undrus Atlas SW week, May

CERN IT Department CH-1211 Geneva 23 Switzerland t The Experiment Dashboard ISGC th April 2008 Pablo Saiz, Julia Andreeva, Benjamin.

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services GS group meeting Monitoring and Dashboards section Activity.

K. Harrison CERN, 20th April 2004 AJDL interface and LCG submission - Overview of AJDL - Using AJDL from Python - LCG submission.

EMI INFSO-RI SA2 - Quality Assurance Alberto Aimar (CERN) SA2 Leader EMI First EC Review 22 June 2011, Brussels.

LC Software Workshop, May 2009, CERN P. Mato /CERN.

Geant4 Acceptance Suite for Key Observables CHEP06, T.I.F.R. Mumbai, February 2006 J. Apostolakis, I. MacLaren, J. Apostolakis, I. MacLaren, P. Mendez.

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Monitoring for the LHC experiments Irina Sidorova (CERN, JINR) on.

Grid Status - PPDG / Magda / pacman Torre Wenaus BNL U.S. ATLAS Physics and Computing Advisory Panel Review Argonne National Laboratory Oct 30, 2001.

Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.

INFSO-RI Module 01 ETICS Overview Etics Online Tutorial Marian ŻUREK Baltic Grid II Summer School Vilnius, 2-3 July 2009.

Best Practices for Standardized Desktop Images with Hardware Independent Imaging.

Build Processes and Continuous Integration Automating Build Processes Software University Technical Trainers SoftUni Team.

And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR

Enabling Grids for E-sciencE System Analysis Working Group and Experiment Dashboard Julia Andreeva CERN Grid Operations Workshop – June, Stockholm.

MW Readiness Verification Status Andrea Manzi IT/SDC 21/01/ /01/15 2.

LHCb Software Week November 2003 Gennady Kuznetsov Production Manager Tools (New Architecture)

David Adams ATLAS DIAL/ADA JDL and catalogs David Adams BNL December 4, 2003 ATLAS software workshop Production session CERN.

ATLAS is a general-purpose particle physics experiment which will study topics including the origin of mass, the processes that allowed an excess of matter.

GDB Meeting - 10 June 2003 ATLAS Offline Software David R. Quarrie Lawrence Berkeley National Laboratory

Julia Andreeva, CERN IT-ES GDB Every experiment does evaluation of the site status and experiment activities at the site As a rule the state.

LHCb-ATLAS GANGA Workshop, 21 April 2004, CERN 1 DIRAC Software distribution A.Tsaregorodtsev, CPPM, Marseille LHCb-ATLAS GANGA Workshop, 21 April 2004.

WLCG and the India-CERN Collaboration David Collados CERN - Information technology 27 February 2014.

EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.

EMI INFSO-RI Guidelines and SQA Process Maria Alandes Pradillo (CERN) SA2.2 Task Leader.

EMI is partially funded by the European Commission under Grant Agreement RI SA2 – Development Tools Andres Abad Rodriguez SA2.4 Tools Activity Leader.

INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.

© SERG Reverse Engineering (REportal) REportal: Reverse Engineering Portal (reportal.cs.drexel.edu)

Testing and integrating the WLCG/EGEE middleware in the LHC computing Simone Campana, Alessandro Di Girolamo, Elisa Lanciotti, Nicolò Magini, Patricia.

K. Harrison CERN, 22nd September 2004 GANGA: ADA USER INTERFACE - Ganga release status - Job-Options Editor - Python support for AJDL - Job Builder - Python.

Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Andrea Sciabà Hammercloud and Nagios Dan Van Der Ster Nicolò Magini.

Validation of SAM3 monitoring data (availability & reliability of services) Ivan Dzhunov, Pablo Saiz (CERN), Elena Tikhonenko (JINR, Dubna) April 11, 2014.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Monitoring of the LHC Computing Activities Key Results from the Services.

DDM FAX Dashboard status and future Luca Magnoni IT/SDC 2 nd June 2014.

MND review. Main directions of work  Development and support of the Experiment Dashboard Applications - Data management monitoring - Job processing monitoring.

INFSO-RI Enabling Grids for E-sciencE Using of GANGA interface for Athena applications A. Zalite / PNPI.

SMI 7 May 2008B. Franek SMI++ Framework Knowledge Exchange seminar 1 SMI++ Object-Oriented Framework for Designing and Implementing Distributed Control.

Predrag Buncic (CERN/PH-SFT) Software Packaging: Can Virtualization help?

Grid Status - PPDG / Magda / pacman Torre Wenaus BNL DOE/NSF Review of US LHC Software and Computing Fermilab Nov 29, 2001.

Feedback from CMS Andrew Lahiff STFC Rutherford Appleton Laboratory Contributions from Christoph Wissing, Bockjoo Kim, Alessandro Degano CernVM Users Workshop.

WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers. Alexandre Beche.

EMI is partially funded by the European Commission under Grant Agreement RI EMI SA2 Report Andres ABAD RODRIGUEZ, CERN SA2.4, Task Leader EMI AHM,

Status of gLite-3.0 deployment and uptake Ian Bird CERN IT LCG-LHCC Referees Meeting 29 th January 2007.

JRA1 Meeting – 09/02/ Software Configuration Management and Integration EGEE is proposed as a project funded by the European Union under contract.

Breaking the frontiers of the Grid R. Graciani EGI TF 2012.

CERN IT Department CH-1211 Genève 23 Switzerland t Bamboo users meeting IT-CS-CT.

1 DIRAC Project Status A.Tsaregorodtsev, CPPM-IN2P3-CNRS, Marseille 10 March, DIRAC Developer meeting.

Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Our experience with NoSQL and MapReduce technologies Fabio Souto.

DECTRIS Ltd Baden-Daettwil Switzerland Continuous Integration and Automatic Testing for the FLUKA release using Jenkins (and Docker)

WLCG Transfers monitoring EGI Technical Forum Madrid, 17 September 2013 Pablo Saiz on behalf of the Dashboard Team CERN IT/SDC.

Jenkins 2.x Chris Vogel. Agenda O Jenkins background O Installation O Jenkins 2.x highlights O Build pipelines O Pipeline DSL O Multibranch plugin.

Experience of PROOF cluster Installation and operation

Testing Dashboard web applications

Savannah to Jira Migration

4th Forum How to easily offer your application as a self-service template by using OpenShift and GitLab-CI 4th Forum Alberto.

Maintaining software solutions

TestResults.io – An Introduction –

Automated Testing and Integration with CI Tool

Alice Software Demonstration

LHCb Data Quality Check web. cern

Node.js Test Automation using Oracle Developer Cloud- Simplified

Office 365 Development July 2014.

Presentation transcript:

Summer Student presentation Changing Dashboard build system to Bamboo Robert Varga IT/SDC

Introduction (previous work)  Digital chest tomosynthesis  An imaging technique between radiograph and CT  Reconstruction algorithms  Coronal slices  CUDA C  FBP  MLEM 29 July 2013 Experiment with Bamboo CI System R. VARGA 2

29 July 2013 Experiment with Bamboo CI System R. VARGA 3

Introduction 29 July 2013 Experiment with Bamboo CI System R. VARGA 4

Related work in CERN (Introduction)  DASHBOARD  A monitoring tool for Distributed Computer System for LHC experiments (ATLAS, ALICE, CMS and LHCb)  Collects information from multiple sources  Measure and visualize:  Site status  Job processing  Data management  Site efficiency  etc… 29 July 2013 Experiment with Bamboo CI System R. VARGA 5

Related work in CERN (Inital state)  DASHBOARD  Custom made build system for modules  Release versions  Create rpm packages  Install software modules  Based on:  Python distutils  Koji build system 29 July 2013 Experiment with Bamboo CI System R. VARGA 6

Related work in CERN (Bamboo I.)  Bamboo  Continous Integration System  Automated building and testing of software source code status Why?  A common solution used by the group  Common build system which is use source code repositories (GIT)  To execute tests for the modules  eg. run Selenium tests 29 July 2013 Experiment with Bamboo CI System R. VARGA 7

Related work in CERN (Bamboo II.)  Bamboo anatomy  Stages group jobs to individual steps  Job processes a series of one or more tasks 29 July 2013 Experiment with Bamboo CI System R. VARGA 8

Related work in CERN (Bamboo III.)  Created plan 29 July 2013 Experiment with Bamboo CI System R. VARGA 9

Related work in CERN (Bamboo IV.) 29 July 2013 Experiment with Bamboo CI System R. VARGA 10

Related work in CERN (Next step)  Create separate repositories  Stable / Unstable versions  SLC 5 / SLC 6  Create jobs for installation & module testing  Test modules separately  Integrate Selenium tests with Bamboo 29 July 2013 Experiment with Bamboo CI System R. VARGA 11

Thank you!  Acknowledgement  Alexandre Beche  Andres Abad Rodriguez  Ivan Antoniev Dzhunov  Pablo Saiz 29 July 2013 Experiment with Bamboo CI System R. VARGA 12