D4Science and ETICS Building and Testing gCube and gCore Pedro Andrade CERN EGEE’08 Conference 25 September 2008 Istanbul (Turkey) www.d4science.eu.

Slides:



Advertisements
Similar presentations
Introduction to Maven 2.0 An open source build tool for Enterprise Java projects Mahen Goonewardene.
Advertisements

D4Science: a Data Infrastructure Ecosystem for Science DL.org Autumn School – Athens, 3-8 October 2010 Leonardo Candela 6 th October 2010.
Automated Software Testing: Test Execution and Review Amritha Muralidharan (axm16u)
INFSO-RI An On-Demand Dynamic Virtualization Manager Øyvind Valen-Sendstad CERN – IT/GD, ETICS Virtual Node bootstrapper.
SC7 WG6 Rome Engineering Ingegneria Informatica S.p.A. INFSO-RI Isabel Matranga ETICS Automated Building,Testing and Quality Assurance.
Software Frameworks for Acquisition and Control European PhD – 2009 Horácio Fernandes.
The Open Grid Service Architecture (OGSA) Standard for Grid Computing Prepared by: Haoliang Robin Yu.
Final Review Meeting 16 th March 2010 Brussels (Belgium) D4Science Production Infrastructure Contract n°: RI Pedro Andrade (CERN)
EC Review – 01/03/2002 – G. Zaquine – Quality Assurance – WP12 – CS-SI – n° 1 DataGrid Quality Assurance Gabriel Zaquine Quality Engineer - WP12 – CS-SI.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
Cluj Napoca, 28 August IEEE International Conference on Intelligent Computer Communication and Processing Digital Libraries Workshop Towards.
University of Palestine software engineering department Testing of Software Systems Fundamentals of testing instructor: Tasneem Darwish.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks gLite Release Process Maria Alandes Pradillo.
Some Thoughts on HPC in Natural Language Engineering Steven Bird University of Melbourne & University of Pennsylvania.
EMI INFSO-RI EMI SA2 Report Quality Assurance Alberto Aimar (CERN) SA2 WP Leader.
EMI INFSO-RI EMI Quality Assurance Processes (PS ) Alberto Aimar (CERN) CERN IT-GT-SL Section Leader EMI SA2 QA Activity Leader.
Flexibility and user-friendliness of grid portals: the PROGRESS approach Michal Kosiedowski
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
Using the Open Metadata Registry (openMDR) to create Data Sharing Interfaces October 14 th, 2010 David Ervin & Rakesh Dhaval, Center for IT Innovations.
EMI INFSO-RI SA2 - Quality Assurance Alberto Aimar (CERN) SA2 Leader EMI First EC Review 22 June 2011, Brussels.
Virtual Research Environments: e-Infrastructures beyond Digital Libraries Pasquale Pagano CNR-ISTI RCDL08 Conference Information.
INFSOM-RI Juelich, 10 June 2008 ETICS - Maven From competition, to collaboration.
INFSO-RI Module 01 ETICS Overview Alberto Di Meglio.
A DΙgital Library Infrastructure on Grid EΝabled Technology ETICS Usage in DILIGENT Pedro Andrade
Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.
INFSO-RI Module 01 ETICS Overview Etics Online Tutorial Marian ŻUREK Baltic Grid II Summer School Vilnius, 2-3 July 2009.
A DΙgital Library Infrastructure on Grid EΝabled Technology Release Beta Deployment Testing Plans Pedro Andrade
A DΙgital Library Infrastructure on Grid EΝabled Technology Release Beta Build and Deployment Testing.
The Roadmap to Software Factories Tools, Patterns and Frameworks.
European Middleware Initiative (EMI) – Release Process Doina Cristina Aiftimiei (INFN) EGI Technical Forum, Amsterdam 17. Sept.2010.
EGEE is a project funded by the European Union under contract IST JRA1-SA1 requirement gathering Maite Barroso JRA1 Integration and Testing.
Oct 2008 RCDL 2008, Dubna, Russian Federation D4Science Tutorial Preface George Kakaletris (NKUA)
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
SEAL Core Libraries and Services CLHEP Workshop 28 January 2003 P. Mato / CERN Shared Environment for Applications at LHC.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.
Testing Grid Software on the Grid Steven Newhouse Deputy Director.
Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing
G.Govi CERN/IT-DB 1 September 26, 2003 POOL Integration, Testing and Release Procedure Integration  Packages structure  External dependencies  Configuration.
Building Scientific Workflows for the Fisheries and Aquaculture Management Community based on Virtual Research Environments Pedro Andrade (CERN)
Windows SharePoint Services. Overview Windows SharePoint Services (WSS) Information Worker Infrastructure component delivered in Windows Server 2003 Enables.
On the D4Science Approach Toward AquaMaps Richness Maps Generation Pasquale Pagano - CNR-ISTI Pedro Andrade.
EMI INFSO-RI EMI Quality Assurance Tools Lorenzo Dini (CERN) SA2.4 Task Leader.
Managing deployment and activation of Web Applications in a distributed e-Infrastructure EGI Technical Forum September 2011 Lyon
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Patch Preparation SA3 All Hands Meeting.
INFSO-RI JRA2 Test Management Tools Eva Takacs (4D SOFT) ETICS 2 Final Review Brussels - 11 May 2010.
INFSOM-RI ETICS: E-infrastructure for Testing, Integration and Configuration of Software Alberto Di Meglio Project Manager.
EGEE-II INFSO-RI Enabling Grids for E-sciencE A Glance Towards the Future Mike Mineter Training Outreach and Education University.
ETICS An Environment for Distributed Software Development in Aerospace Applications SpaceTransfer09 Hannover Messe, April 2009.
JRA1 Meeting – 09/02/ Software Configuration Management and Integration EGEE is proposed as a project funded by the European Union under contract.
Parasoft : Improving Productivity in IT Organizations David McCaw.
Copyright © 2004 R2AD, LLC Submitted to GGF ACS Working Group for GGF-16 R2AD, LLC Distributing Software Life Cycles Join the ACS Team GGF-16, Athens R2AD,
Pedro Andrade > IT-GD > D4Science Pedro Andrade CERN European Organization for Nuclear Research GD Group Meeting 27 October 2007 CERN (Switzerland)
M.-E. Bégin¹, S. Da Ronco², G. Diez-Andino Sancho¹, M. Gentilini³, E. Ronchieri ², and M. Selmi² ¹CERN, Switzerland, ² INFN-Padova, Italy, ³INFN-CNAF,
Virtual Research Environments as-a-Service Donatella Castelli CNR-ISTI EGI Conference 2016, 6-8 April.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
1 Tutorial Outline 30’ From Content Management Systems to VREs 50’ Creating a VRE 80 Using a VRE 20’ Conclusions.
INFSOM-RI Elisabetta Ronchieri INFN CNAF ETICS 2 nd EU Review (CERN) 15 February 2008 WP3 - Software Configuration Tools and Methodologies.
INFSOM-RI WP3: WP3: Software configuration tools and methodologies Status Report ETICS All-Hands – 23 May 2007 E. Ronchieri.
DataGrid Quality Assurance
Pasquale Pagano CNR, Italy
E-Infrastructure for Testing, Integration and Configuration of Software Alberto Di Meglio CERN, INFN, Engineering, 4D Soft, University of Wisconsin.
Marc-Elian Bégin ETICS Project, CERN
Release BETA Integration Tool Status Geneva – 1 March 2007
The Open Grid Service Architecture (OGSA) Standard for Grid Computing
Enterprise Computing Collaboration System Example
The ETICS Build and Test Service
Introduction to D4Science
Cloud Management Mechanisms
Simplified Development Toolkit
Module 01 ETICS Overview ETICS Online Tutorials
Presentation transcript:

D4Science and ETICS Building and Testing gCube and gCore Pedro Andrade CERN EGEE’08 Conference 25 September 2008 Istanbul (Turkey)

2 D4Science D4Science is a FP7 project establishing a knowledge infrastructure of data, services and processing resources through the provision of Virtual Research Environments D4Science simplifies the discovery, access and production of existing and new data through a collaborative environment D4Science continues and consolidates the work started in the DILIGENT project opening it to new communities D4Science provides a production environment for ESA & FAO D4Science and ETICS 25 September 2008, Istanbul (Turkey)

3 gCube gCube is the software implementing the D4Science resources sharing vision via Virtual Research Environments gCube software:  Provides a multi-domain Service Oriented Infrastructure  Gathers data, services and hardware as system resources  Supports on-demand virtual collaborations  Reduces the cost of managing research infrastructures  Tailored for e-Science (content oriented, data intensive)  Is complaint with consolidated and emerging standards D4Science and ETICS 25 September 2008, Istanbul (Turkey)

4 gCube gCube resources:  Computing and storage  VRE services  content management  search and index  annotations  Collections  raw data  content  metadata  Processes to manage other resources D4Science and ETICS 25 September 2008, Istanbul (Turkey)

5 gCore gCore is a framework developed to reduce the complexity of designing and implementing gCube SOI complaint services gCore is composed by:  gCore Framework: An application framework to consolidate and develop existing/new gCube services  gCore Container: A self-contained software container distributed to facilitate the deployment of gCube services D4Science and ETICS 25 September 2008, Istanbul (Turkey)

6 D4Science in ETICS Two projects:  org.gcube: 24 subsystems  org.gcore: 2 subsystems 225 components:  160 software components  51 services, 84 libraries, and 25 portlets  Including services samples, external libs, test libs (18 components)  65 distribution components  64 gCube Software Archives  gCore Hosting Node Around 500 configurations D4Science and ETICS 25 September 2008, Istanbul (Turkey)

7 Preparing Cycle Delivering Components Delivering Subsystems Releasing to Integration Building & Packaging Deployment Testing Functional Testing Releasing to Production D4Science Release Cycle Bug Fixing Patching the Production D4Science and ETICS 25 September 2008, Istanbul (Turkey) etics component configurations etics subsystem configurations etics project configurations etics-build etics-test

8 Build The objective of the build activity is to ensure api-level compliance of components and the creation of components’ artifacts. ETICS is exploited to:  Model build scripts and manage build dependencies  Execute on-demand remote builds (via WA and CLI)  Execute nightly local builds for development configurations  Produce distribution artifacts (via dedicated components)  Feed the D4Science reporting application with xml raw reports D4Science and ETICS 25 September 2008, Istanbul (Turkey)

9 Deployment Testing The objective of the deployment testing activity is to ensure that the gCube software archives:  Can be remotely and automatically deployed  Once deployed, that it starts and is activated correctly ETICS is exploited to:  Manage the executing the deployment tests  Define the deployment test tool dependencies  Generate deployment test reports D4Science and ETICS 25 September 2008, Istanbul (Turkey)

10 Deployment Test D4Science and ETICS 25 September 2008, Istanbul (Turkey) ETICS Server Condor Node Service 2Service 1Service A Service 2Service 1Service A

11 Static Analyzers The objective of the adoption of static analyzers is to improve the quality of gCore and gCube software ETICS is planned to be exploited to:  Run jFindbugs and jDepend (under adoption) Other analyzers (not exploited via ETICS):  Check coverage of javadoc documentation (checkstyle)  Monitor the content of code repository  e.g. pre-compiled artifacts, etc..  Certification of distribution components  check structure, correctness, etc.. D4Science and ETICS 25 September 2008, Istanbul (Turkey)

12 Current Releases D4Science and ETICS 25 September 2008, Istanbul (Turkey)

13 Current Releases D4Science and ETICS 25 September 2008, Istanbul (Turkey)

14 Improvements Parametric tests on remote infrastructure  Single remote test with multiple parameterized tests Co-scheduling mechanism to be further exploited  Condor nodes firewall limitations Notification mechanism  ETICS is designed to support distributed teams  No notifications between different ETICS roles  components availability  successful build/test execution D4Science and ETICS 25 September 2008, Istanbul (Turkey)

15 Improvements ETICS configurations contains many information  Can an ETICS API be useful ? Reports summary information  Summary page with built, test, and plug-ins execution  Organized by configuration Plug-ins availability  Info about existing ones?  How to add external plug-ins? D4Science and ETICS 25 September 2008, Istanbul (Turkey)

16 Conclusions Solid exploitation:  DILIGENT/D4Science using ETICS since the early days  ETICS is the default tool in D4Science build an test tasks  ETICS completely adopted by all D4Science members Solid collaboration:  D4Science and ETICS plans kept aligned  Efficient support from ETICS D4Science and ETICS 25 September 2008, Istanbul (Turkey)