SYSTEM INTEGRATION TESTING Getting ready for testing shifts Gunter Folger CERN PH/SFT Geant4 Collaboration Workshop 2011 SLAC.

Slides:



Advertisements
Similar presentations
Electronic Timesheet User Manual
Advertisements

GP2013 (R2) New features in GP2013 (R2). New Ribbon for windows Edit List is the Print button on the right without the paper background Action pane can.
Project Perfect Pty Ltd Project Administrator Overview of Software.
New Proposal Review Form Revised 09/10/2008 Office of Sponsored Projects (512)
Customizing Word Microsoft Office Word 2007 Illustrated Complete.
Millennium Enhancements Present and Future…. Enhancements 2009A Last 2 patrons that have checked out an item. They have started rewriting the core software.
Automated Tests in NICOS Nightly Control System Alexander Undrus Brookhaven National Laboratory, Upton, NY Software testing is a difficult, time-consuming.
Agile Testing with Testing Anywhere The road to automation need not be long.
Software Testing Test Design and Implementation. Agenda Test Design Test Implementation Test Design Sources Automated Testing 2.
2 Copyright © 2010, Oracle and/or its affiliates. All rights reserved. Navigating in R12.x Oracle Applications.
Automation using Selenium Authored & Presented by : Chinmay Sathe & Amit Prabhu Cybage Software Pvt. Ltd.
Quality Assurance and Testing in LCG CHEP 2004 Interlaken, Switzerland 30 September 2004 Manuel Gallas, Jakub MOSCICKI CERN
Abstract The automated multi-platform software nightly build system is a major component in the ATLAS collaborative software organization, validation and.
M Gallas CERN EP-SFT LCG-SPI: SW-Testing1 LCG-SPI: SW-Testing LCG Applications Area GridPP 7 th Collaboration Meeting LCG/SPI LCG.
Unit tests, Integration tests Physics tests Andrea Dotti, Gunter Folger, Pere Mato CERN – PH/SFT Geant4 workshop 2012.
Introduction Purpose This course describes the process of installing the KPIT GNU toolchain on your PC. Objective Learn how easy it is to get information.
Nightly Releases and Testing Alexander Undrus Atlas SW week, May
Indispensable tools for research at its best Introducing the New Write-N-Cite.
Usability Issues Documentation J. Apostolakis for Geant4 16 January 2009.
2 Copyright © 2007, Oracle. All rights reserved. Navigating in R12 Oracle Applications.
Configuration Management Geant4 Review 19 June 2001 Gunter Folger.
Productivity Programs Common Features and Commands.
Progress with migration to SVN Part3: How to work with g4svn and geant4tags tools. Geant4.
Study for Migration from CVS to SubVersion (SVN) Gunter Folger CERN/PH/SFT.
H2OS Audit Trail And History page. What is the Audit Trail? The Audit Trail is a companion for the H2OS Customers table. It contains a historical record.
Unit 1: Java and Eclipse The Eclipse Development Environment.
LCG-SPI: SW-Testing LCG AppArea internal review (20/10/03)
Micro/Nano Fabrication Center (MFC) University of Arizona.
Documentation Gunter Folger / CERN Geant4 School, Annecy 2008.
Chapter 3 Automating Your Work. It is frustrating when you have to type the same passage of text repeatedly. For example your name and address. Word includes.
Support for Platforms in Geant4. Gunter Folger, CERN PH/SFT Geant4 Technical Forum CERN, 5 February 2004.
G.Govi CERN/IT-DB 1 September 26, 2003 POOL Integration, Testing and Release Procedure Integration  Packages structure  External dependencies  Configuration.
SPI NIGHTLIES Alex Hodgkins. SPI nightlies  Build and test various software projects each night  Provide a nightlies summary page that displays all.
Geant4 is a toolkit to simulate the passage of particles through matter, and is widely used in HEP, in medical physics and for space applications. Ongoing.
T EST T OOLS U NIT VI This unit contains the overview of the test tools. Also prerequisites for applying these tools, tools selection and implementation.
JRA1 Meeting – 09/02/ Software Configuration Management and Integration EGEE is proposed as a project funded by the European Union under contract.
Indispensable tools for research at its best RefWorks 2.0 fundamental Alan Tang
7/8/2016 OAF Jean-Jacques Gras Stephen Jackson Blazej Kolad 1.
SPI Report for the LHCC Comprehensive Review Stefan Roiser for the SPI project.
Software Release Build Process and Components in ATLAS Offline Emil Obreshkov for the ATLAS collaboration.
1 Branding 1 st Activity: Demonstrate the ability for Site Owners to adjust the Logo of the site within the SharePoint Site Navigate to Site Assets Upload.
SOFTWARE TESTING TRAINING TOOLS SUPPORT FOR SOFTWARE TESTING Chapter 6 immaculateres 1.
SchoolSuccess for Coordinators
Welcome POS Synchronize Concept 08 Sept 2015.
Gunter Folger / CERN MC-PAD, DESY/Hamburg Jan 2010
How to Contribute to System Testing and Extract Results
EGEE Middleware Activities Overview
Using SHOPFLOOR with QUALITY control
fields of possible improvement
Offline shifter training tutorial
Generator Services planning meeting
LCG Software Quality Assurance
LCGAA nightlies infrastructure
Software Testing With Testopia
User Documents and Examples I
Testing Key Revision Points.
PIC + TransNet.
Program Management Portal (PgMP): Catalog and the Client
After Transmission to the IRS
Introducing Automation in Traditional Software Testing Best Practices.
Roberta Roth, Alan Dennis, and Barbara Haley Wixom
Design and Programming
GDSS – Digital Signature
Gunter Folger Geant4 Genova Workshop 4-July-2001
Online Quotes New Feature Instructions
LESSON 01 Hands-on Training Execution
How to run and format reports for your Local Board of Health
Priorities and contents of releases
Approving Time in Kronos Manager/Supervisor Reference Guide
Framework Anil
Presentation transcript:

SYSTEM INTEGRATION TESTING Getting ready for testing shifts Gunter Folger CERN PH/SFT Geant4 Collaboration Workshop 2011 SLAC

Tasks of STT working group  Integration testing of proposed tags  Regular - ~ daily - testing of new code  Shifts planned for this work  Q/A for release  Using tools checking memory use,  Checking code for std::,...  Physics validation test suite  HEP oriented 19-Sep Gunter Folger - CERN PH/SFT

Contents  “shift duties”  Testing cycle  Tags management tool  Selecting, rejecting, accepting tags  Checking results  Logging results or decisions  Documentation 19-Sep Gunter Folger - CERN PH/SFT

Shift Duties  Routine work within collaboration should be shared across collaboration  Daily running of system integration testing is one of these routine work items  like reviews for papers  Not included  Integrating more tests  Managing configuration 19-Sep Gunter Folger - CERN PH/SFT

Who does what? What is integration testing 19-Sep Gunter Folger - CERN PH/SFT

Regular integration testing Developer propose Tags Tester selects Tags Testing in “nightlies” Tester analyse results Tester accepts / rejects tags 19-Sep Gunter Folger - CERN PH/SFT

Steps – prepare testing  Tag(s) are proposed by developer  Tags are selected by STT tester  Use “full interface” of geant4tags  Taking into account dependencies on co-working tags  Avoiding sets of tags risking confusing results Hints: NO: development in different (sub)categories Yes: bug fixes in a category, or tags in tests and examples 19-Sep Gunter Folger - CERN PH/SFT

Geant4tags – full interface 19-Sep Gunter Folger - CERN PH/SFT

Selecting tags  Mark tags and click “Change Status”  Check and adapt settings in pop-up, and click “Change”  Taglist is a grouping of tags  Tags selected in one day  Optional name 19-Sep Gunter Folger - CERN PH/SFT

Starts short past midnight at CERN Lasts few hours Where? How? What ? Wait for testing 19-Sep Gunter Folger - CERN PH/SFT

19-Sep Gunter Folger - CERN PH/SFT

Testing - How 19-Sep-2011 Gunter Folger - CERN PH/SFT 12  Testing using SPI nightlies is automatic – unless there is a rare problem  Configure environment using CMT environment variables, compiler Geant4 specific setup in Geant4 CMT requirements file  Checkout automatic – querying db for tags  Compile/Build using Geant4 build system  Testing uses QMtest tool run tests as defined in test suite, setup in geant4/tests/tools/qmtest Failure defined as sizeof(stderr) != 0, or crash. In case of crash, core file is created, but is local to testing machine  Results of build & test copied to web  Crash require manual intervention  Core file can only be inspected on testing machine  No trace back generated

Testing - What 19-Sep-2011 Gunter Folger - CERN PH/SFT 13  We test different configurations in ‘slots’  Testing selected tags using Normal GNU-make build & test Cmake build & test using GNUmake and QMtest Internal CLHEP & Energy conservation checks in G4Hadronic Process enabled  Testing reference/patch tag using GNU-make build  And using differing OS/compiler combinations  Linux, Mac, Windows Sometimes several compilers  Sunday is Large_N test suite is run

Nightlies result page 19-Sep Gunter Folger - CERN PH/SFT

Analyse Results  Reject tag causing compilation error or warning  Any (new) tests failing in any configuration  If clear which tag causes this, reject tag  If unclear, one can postpone a tag  Run debugger on testing machine  Record result in online document 19-Sep Gunter Folger - CERN PH/SFT

Accept/reject tags  Important Do not let increase the number tests failing  Accept tags only if no  Compilation error or warning Do reject any tag with compilation warning or error  No test newly failing Unless a change in one category uncovers an old problem somewhere else. 19-Sep Gunter Folger - CERN PH/SFT

How to Reject Tags  Geant4tags web page, as for selection  Rejecting:  Select tag to reject, one by one  Change status  in popup, select reject and in new field fill in reason and click “Change” 19-Sep Gunter Folger - CERN PH/SFT

How to Accept tags  From full interface in Geant4tags  Mark all tags you want to accept, and click change Can do this on the tags tab Or on taglist tab, select taglist, and mark tags Click in header row will mark all  In popup, check that ‘Accepted’ is selected and click change 19-Sep Gunter Folger - CERN PH/SFT

Documentation  Working group web page has links to  Documentation on test procedures Needs improvements, please give feedback  List of tests and examples being used in testing Actual list is in log file of test Web page listing exists Sep Gunter Folger - CERN PH/SFT

Shift duties - Summary  For about one week, once per day: 1. Checks results and document results 2. Reject or accept tags 3. Select proposed tags 19-Sep Gunter Folger - CERN PH/SFT

Shift duties - Summary WhatTime CERNSLAC ( -9 )Japan ( +8 ) Checks results and document results Morning >~ 8:00 MorningLate afternoon >~ 16:00 Reject or accept tagsAs above Select proposed tagsBefore midnight Before 3:00 pmFrom 9:00 until 8:00 next day 19-Sep Gunter Folger - CERN PH/SFT For about one week of shift, once per day

Proposal  Start with testing shifts during “normal” weeks  Release phase depending on experience  Shifts of two testers, one novice, one expert  During release period, local expert will join  Need close collaboration with release manager 19-Sep Gunter Folger - CERN PH/SFT

Summary  Shift system for regular integration testing possible  Tools have been refined  No failing tests on supported platforms  Propose to start shifts now, and gather experience  Steering board first?  Improve documentation with experience and feedback  Andrea will continue to show  tools to document results  Subscribe for shifts 19-Sep Gunter Folger - CERN PH/SFT