ISO/IEC Software Testing The New International Software Testing Standard By Tafline Murnane and Stuart Reid ISO/IEC JTC1/SC7 WG26 Software Testing.

Slides:



Advertisements
Similar presentations
Software Quality Assurance Plan
Advertisements

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International.
International Standards for Software & Systems Documentation Ralph E. Robinson R 2 Innovations.
Ossi Taipale, Lappeenranta University of Technology
ISO – plans and progress towards the revised international standard for thesauri Stella G Dextre Clarke Project Leader, ISO NP
Miles Shepherd Chairman ISO Technical Committee 258.
1 Software Requirement Analysis Deployment Package for the Basic Profile Version 0.1, January 11th 2008.
Stepan Potiyenko ISS Sr.SW Developer.
COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Design & Documentation Adrian Marshall.
SECURITY SIG IN MTS 28 TH JANUARY 2015 PROGRESS REPORT Fraunhofer FOKUS.
COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.
Quality Manual for Interoperability Testing Morten Bruun-Rasmussen Presented by Jos Devlies, Eurorec.
RC14001 ® Update GPCA Responsible Care Committee September 23, 2013.
ISO/IEC Software Testing
ISO 9001:2015 Revision overview - General users
Software testing standards ISO/IEC and 33063
Test Design Techniques
ISO 9001:2015 Revision overview - General users
Leaders in Asset Management ISO Asset Management Systems 1 Prepared by Jim Dieter, MIAM CPPM CF NPMA Executive Vice President Director of Strategic.
Pre-Project Activities Text Chapters 5 and 6. Pre-Project Activities 1.Contract Review 2.Development Plan 3.Quality Plan.
Safety Driven Performance Conference 2013 The future of managing asset-intensive businesses John Keefe APM/RBMI Technical Manager Asset Integrity Services.
Laboratory Biorisk Management Standard CWA 15793:2008
Software Testing Lifecycle Practice
Implementing ISO/IEC in your workplace (And why should technical communicators care about standards, anyway?) George Hayhoe Mercer University School.
Alistair Walker, Alphonse Phillipe, Anne Mette Hass, Jon Hagar, Matthias Daigl, Qin Liu Stuart Reid, Sylvia Veeraraghavan,Tafline Ramos (Murnane),
CO2403 and CO3808 – Quality Management Systems Quality process definition, administration and accreditation.
Conformity Assessment and Accreditation Mike Peet Chief Executive Officer South African National Accreditation System.
Introduction to the ISO series ISO – principles and vocabulary (in development) ISO – ISMS requirements (BS7799 – Part 2) ISO –
Project Management Practices Vesa Tenhunen University of Eastern Finland LUMA Centre
D1.HGE.CL7.01 D1.HGA.CL6.08 Slide 1. Introduction Design, prepare and present reports  Classroom schedule  Trainer contact details  Assessments  Resources:
INFO 637Lecture #101 Software Engineering Process II Review INFO 637 Glenn Booker.
P1516.4: VV&A Overlay to the FEDEP 20 September 2007 Briefing for the VV&A Summit Simone Youngblood Simone Youngblood M&S CO VV&A Proponency Leader
Information Security 14 October 2005 IT Security Unit Ministry of IT & Telecommunications.
Requirements Management with Use Cases Module 10: Requirements Across the Product Lifecycle Requirements Management with Use Cases Module 10: Requirements.
P3 Business Analysis. 2 Section F: Project Management F1.The nature of projects F2. Building the Business Case F4. Planning,monitoring and controlling.
Models of Security Management Matt Cupp. Overview What is Security Management? What is Security Management? ISO/IEC ISO/IEC NIST Special Publication.
Report to IEEE CS SESC of IEEE CS/SC7 Liaison1 Report to IEEE CS SESC: Report of IEEE CS / SC7 Liaison James W. Moore Liaison Representative IEEE.
Software Engineering — Software Life Cycle Processes — Maintenance
Introduction for the Implementation of Software Configuration Management I thought I knew it all !
ISO/IEC Software Testing
Jürgen Großmann, Fraunhofer FOKUS
BIL 424 NETWORK ARCHITECTURE AND SERVICE PROVIDING.
IEEE Computer Society Category A Liaison to JTC 1/SC 7: Status Report
2012 Spring Simulation Interoperability Workshop
James W. Moore Liaison Representative IEEE Computer Society June 2004
ISO/IEC JTC 1/SC 7 Working Group 42 - Architecture Johan Bendz
Security SIG in MTS 05th November 2013 DEG/MTS RISK-BASED SECURITY TESTING Fraunhofer FOKUS.
Chapter 10 Software Quality Assurance& Test Plan Software Testing
Global Inventory of Statistical Standards
ISO/IEC Software Testing
TechStambha PMP Certification Training
ISO TC268 SC1 - Smart Community Infrastructures -
Evolution of the IECEx Scheme – Ex TAG Training Workshop Shanghai
Frances Cleveland Convenor WG15
Standards Development: An Overview
Status of EQ-5D-5L Valuation Using Standardized Valuation Methodology
Activities affecting HSSC
Engineering Processes
Software Measurement Process ISO/IEC
[Project Name] Project Report
Standards.
[Project Name] Project Report
Safety Management System Implementation
X-DIS/XBRL Phase 2 Kick-Off
Global Inventory of Statistical Standards
Software Testing Lifecycle Practice

Transport Safety Standards Committee 38th Meeting
DRAFT ISO 10013:20XX Overview Quality management – Guidelines for quality management system documented information ISO/TC176 TG 01.
38th Nuclear Safety Standards Committee 37th Radiation Safety Standards Committee 38th Waste Safety Standards Committee Joint Session 26 – 27 November.
Presentation transcript:

ISO/IEC Software Testing The New International Software Testing Standard By Tafline Murnane and Stuart Reid ISO/IEC JTC1/SC7 WG26 Software Testing

Scope  Overview of ISO/IEC  Applicability  Recent developments  Timeline  Future work

Motivation for ISO/IEC  Conflicts in definitions & procedures Plethora of standards to be replaced by one  IEEE 829, IEEE 1008, BS /-2, IEEE 1028  Missing in current standards: Organizational Test Policy & Strategy Project Test Management Common System & Acceptance Testing techniques Non-functional testing

BS BS IEEE 829 IEEE 1008 BS Documentation Part 3 Testing Techniques Part 4 Processes Part 2 Concepts & Vocabulary Part 1 ISO – Scope & Structure

Part 1: Concepts & Vocabulary  Software testing concepts Introduction to software testing Relationship between testing, development & maintenance Implications of lifecycle models Approaches to testing  Testing vocabulary

Part 2: Testing Processes Test Management Processes Static Test Processes Organisational Test Process Dynamic Test Processes

Instantiating Testing Processes

Published Test Specification [Issues identified or Scheduled review due or Major organizational change] [No issues identified with Test Specification] Develop test specification Gain consensus on test specification Approved Test Specification Publish test specification Review test specification [Major revision required] Draft Test Specification [No change required] Monitor and control use of test specification Updated Test Specification Update test specification [Minor revision required] Organisational Test Process

Test Management Processes Static Test Processes Test Planning Test Monitoring & Control Test Completion Organisational Test Documentation Feedback on Organisational Test Documentation Test Plan Updates Test Plan Test Completion Report Test Measures Dynamic Test Processes Test Plan, Control Directives Test Management Processes Test Plan, Test Completion Report, Test Measures Test Plan, Control Directives Test Management Processes

Organise Test Plan Development Identify & Analyze Risks Design Test Strategy Determine Staffing and Scheduling Document Test Plan Schedule, Staffing Profile Test Strategy Analysed Risks Scope Identify Risk Treatment Approaches Gain Consensus on Test Plan Approved Test Plan Draft Test Plan Publish Test Plan Understand Context Treatment Approaches Test Planning Process

Test Monitoring & Control Process Monitor Report Measures ControlSet-Up [Testing Complete] [Testing Incomplete] Control Directives Test Progress Information Test Progress Info Test Plan Test Measures Test Status Report Test Control Information...Test Processes... Dynamic/Static/Management

Dynamic Test Processes Test Specification [No Issues Noticed] Test Environment Readiness Report Test Environment Requirements Incident Report Test Design & Implementation Test Environment Set- up Test Execution Test Incident Reporting (Phase) Test Management Process [Issue Noticed or Retest Result] Test Results Control Directives Test Measures (Phase) Test Plan Dynamic Test Processes

Static Test Processes Preparation Follow-Up Review (Phase) Test Management Process Control Directives Test Measures (Phase) Test Plan Static Test Processes

Example Test Process Description  Described using a standard template (following ISO 24774), for example:  Purpose The purpose of the Test Design & Implementation Process is to derive test procedures that will be executed during the Test Execution Process. Test conditions, test coverage items and test cases are created as part of this process.  Outcomes As a result of the successful implementation of the Test Design & Implementation Process:  the features to be tested are decomposed into Feature Sets;  the Test Conditions are derived;  the Test Coverage Items are derived;  Test Cases are derived;  Test Sets are assembled;  Test Procedures are derived.  Activities and tasks The tester shall implement the following activities and tasks in accordance with applicable organization policies & procedures with respect to the Test Design & Implementation Process.  Identify Feature Sets (TD1) …  Derive Test Conditions (TD2) …  Derive Test Coverage Items (TD3) …  Derive Test Cases (TD4) …  Assemble Test Sets (TD5) …  Derive Test Procedures (TD6) …  Information Items As a result of carrying out this process, the following information items shall be produced:  Test specifications (test design specifications, test case specifications and test procedure specifications) and related traceability information;  Test data requirements;  Test environment requirements.

Part 3: Documentation  Organisational Test policy Test strategy  Project Project test plan Test completion report  Appendices examples of documents at each level of testing  Testing Test specification Test results Incident reports Test environment report Test status report Test completion report

Part 4: Techniques  Test case design techniques Static  Inspections, walkthroughs, etc. Dynamic  Black-box, white-box, etc. Non-functional  Security, performance, usability, etc.  Test measurement (e.g. coverage)  Appendices Examples Effectiveness

Applicability  Generic standard to support all domains Some are more standards-oriented (safety, telecoms) Some will become more regulated (e.g. financial)  Needs trialing in all domains & project types Waterfall, iterative, agile New & maintenance projects Information systems & embedded systems  When wouldn’t you be selecting and using techniques? When wouldn’t you plan?

When can I start using 29119?  Typical ISO standards take over 7 years ISO conceived in 1988, published in 1995, represents 17,000 person hours (8.5 person years)  BS /2 took 8 years to develop  IEEE estimates 2-4 years to develop a standard, at a cost of between $2,000 and $10,000 per page [1998]

May 08 May 09 May 10 May 11 May 12 … Timelines Working Draft (WD) Committee Draft (CD) Final Committee Draft (FCD) Final Draft International Standard (FDIS) Final International Standard (FIS) Parts 2 & 3 Parts 1 & 4 WD CD FCD FDIS FIS WD CD FCD FDIS FIS

Do you want to be involved?  Join the working group through your national standards body 6 day meetings, every 6 months contribute between meetings  Join the national working group in your country to: contribute materials review drafts

Current status  Drafts of parts 1, 2 and 3 are under review  Draft of part 4 is underway  18 nations represented: Australia, Canada, China, Columbia, Denmark, Finland, France, Germany, Hong Kong, India, Japan, Poland, Russian Federation, South Africa, South Korea, Spain, United Kingdom, USA  Last meeting – Niigata, Japan, May 2010  Next meeting – Washington, October 2010

Conclusions  International standard will provide practitioners with guidelines for testing that cover all aspects of life cycle Consistent definitions, processes, procedures & techniques for testing  Will be adopted by IEEE, ISO, Standards Australia and other national standards bodies  Currently has representation from 18 nations & is being reviewed by software testing professionals world-wide  We need to ensure widespread applicability

For more information…  Keep up to date on our progress  ISO/IEC website  access to official documents