By Eva Freund, The IV&V Group, Inc.

Slides:



Advertisements
Similar presentations
Common Core State Standards for Mathematics: Rigor
Advertisements

Active Method Management™ with Select Process Director David Piper Principal Consultant.
Ossi Taipale, Lappeenranta University of Technology
Automated Software Testing: Test Execution and Review Amritha Muralidharan (axm16u)
MODELING THE TESTING PROCESS Formal Testing (1.0) Requirements Software Design Risk Data Approved, Debugged, Eng. Tested Code Automated Test Tools Tested.
Software Quality Assurance Plan
Chpter#5 -part#1 Project Scope and Human Resource Planning
Enhancing Data Quality of Distributive Trade Statistics Workshop for African countries on the Implementation of International Recommendations for Distributive.
Prepared By: Certified Compliance Solutions, Inc. August 2012
July 11 th, 2005 Software Engineering with Reusable Components RiSE’s Seminars Sametinger’s book :: Chapters 16, 17 and 18 Fred Durão.
Chapter 6: Design of Expert Systems
Internal Control Concepts Knowledge. Best Practices for IT Governance IT Governance Structure of Relationship Audit Role in IT Governance.
Fundamentals of Information Systems, Second Edition
(c) 2007 Mauro Pezzè & Michal Young Ch 24, slide 1 Documenting Analysis and Test.
COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Design & Documentation Adrian Marshall.
SQM - 1DCS - ANULECTURE Software Quality Management Software Quality Management Processes V & V of Critical Software & Systems Ian Hirst.
Chapter 5: Project Scope Management
10.5 Report Performance The process of collecting and distributing performance information, including status reports, progress measurements and forecasts.
Project Management Methodology (PMM)
DITSCAP Phase 2 - Verification Pramod Jampala Christopher Swenson.
CSE Senior Design II Test Planning Mike O’Dell Based on an earlier presentation by Mike O’Dell, UTA.
SOFTWARE QUALITY ASSURANCE SOFTWARE QUALITY ASSURANCE  DEFINITIONS OF SQA  SOFTWARE STANDARDS  Process Quality Assurance  Product Quality Assurance.
ESC/EN Engineering Process Compliance Procedures August 2002.
Software Verification and Validation (V&V) By Roger U. Fujii Presented by Donovan Faustino.
Information Technology Audit
The BIM Project Execution Planning Procedure
1 Lecture 5.3: SEF Ch 4 Requirements Analysis Dr. John MacCarthy UMBC CMSC 615 Fall, 2006.
Introduction to Software Quality Assurance (SQA)
Test Organization and Management
Best Practices By Gabriel Rodriguez
RUP Implementation and Testing
Software Configuration Management (SCM)
Software System Engineering: A tutorial
NIST Special Publication Revision 1
FCS - AAO - DM COMPE/SE/ISE 492 Senior Project 2 System/Software Test Documentation (STD) System/Software Test Documentation (STD)
DEPARTMENT OF THE NAVY THE NAVY ASN(RD&A) Acquisition and Business Management Recycled Paper SECTION 3.3 Acquisition Strategy Becomes more definitive as.
© 2012 Cengage Learning. All Rights Reserved. This edition is intended for use outside of the U.S. only, with content that may be different from the U.S.
What is a Business Analyst? A Business Analyst is someone who works as a liaison among stakeholders in order to elicit, analyze, communicate and validate.
Software Engineering Principles Principles form the basis of methods, techniques, methodologies and tools Principles form the basis of methods, techniques,
University of Palestine software engineering department Testing of Software Systems Testing throughout the software life cycle instructor: Tasneem.
Software Engineering 2 Software Testing Claire Lohr pp 413 Presented By: Feras Batarseh.
Chapter 10 Analysis and Design Discipline. 2 Purpose The purpose is to translate the requirements into a specification that describes how to implement.
CONTENTS OF THE SRS REPORT. Software Requirements Specification (SRS) template The SRS document describes recommended approaches for the specification.
Software Architecture Evaluation Methodologies Presented By: Anthony Register.
PRJ566 Project Planning & Management Software Architecture.
Systems Accreditation Berkeley County School District School Facilitator Training October 7, 2014 Dr. Rodney Thompson Superintendent.
1 Software Engineering: A Practitioner’s Approach, 6/e Chapter 15a: Product Metrics for Software Software Engineering: A Practitioner’s Approach, 6/e Chapter.
ITC Software ITC FUNCTIONAL TESTING SERVICES.
Software Quality Assurance SOFTWARE DEFECT. Defect Repair Defect Repair is a process of repairing the defective part or replacing it, as needed. For example,
METHODS PLANNING. Methods Class 4 Agenda 1. Overview of Ontario Curriculum Documents 2. Introduce lesson plan formats – GPF & APF 3. Sequence for planning.
Software Production ( ) Lecture 3: Dr. Samer Odeh Hanna (PhD) office: 318.
SE513 Software Quality Assurance Lecture12: Software Reliability and Quality Management Standards.
Software Engineering — Software Life Cycle Processes — Maintenance
Introduction for the Implementation of Software Configuration Management I thought I knew it all !
SQA project process standards IEEE software engineering standards
Software Configuration Management
SQA project process standards IEEE software engineering standards
IEEE Std 1074: Standard for Software Lifecycle
Manfred Huber Based on an earlier presentation by Mike O’Dell, UTA
ENVIRONMENTAL SAFETY & HEALTH DEPARTMENT OF THE NAVY RESPONSIBILITIES
Chpter#5 -part#1 Project Scope and Human Resource Planning
UNIT-6 SOFTWARE QUALITY ASSURANCE
Engineering Processes
Test Planning Mike O’Dell (some edits by Vassilis Athitsos)
Measuring Data Quality and Compilation of Metadata
CHAPTER 9 (part a) BASIC INFORMATION SYSTEMS CONCEPTS
Software Requirements Specification (SRS) Template.
HHS Child Welfare National IT Managers' Meeting
Configuration Management
Speaker’s Name, SAP Month 00, 2017
Presentation transcript:

By Eva Freund, The IV&V Group, Inc. IEEE 829 – 2008 IEEE Standard for Software and System Test Documentation Presented to ASQ 0511 By Eva Freund, The IV&V Group, Inc. Vice-chair P829

Agenda PAST – The old 829 GAP – Unmet needs PRESENT – The new 829 FUTURE – Future needs Conclusion Q and A ASQ 0511 Presentation 1/21/2009

Perspective On The Old 829 Format and content for: Test Plan Test Design Specification Test Case Specification Test Procedure Specification Test Item Transmittal Test Log Test Incident Report Test Summary Report The standard (829-1998) only described the format and content of these documents. The documents were not placed in any context. ASQ 0511 Presentation 1/21/2009

Continued … Old 829 the old 829-1998 Focused solely on stand-alone test documentation Identified the same test documents and the same information for every project. Duplicated information for each level of testing if test documentation was generated for each level of testing (component, integration, system, acceptance) ASQ 0511 Presentation 1/21/2009

Unmet Needs IEEE needed : Standards to be process focused rather than document focused Standards to reflect the role of an activity (eg., test) throughout the SDLC Consistency among related standards Currently, there are only 2 standards that have been revised to meet these IEEE needs. The first to be approved was 1012-2004. The second to be approved is 829-2008. ASQ 0511 Presentation 1/21/2009

Continued … unmet needs Test management needed : Elimination of redundancy of information contained in various test documents A place to describe the management of large/complex projects with multiple test organizations and multiple layers of testing A way to determine how much testing is needed and which test tasks need to be executed Flexibility for various configurations of test documentation ASQ 0511 Presentation 1/21/2009

The New 829 To fill these gaps the new 829 adds: New directions/approach New processes New test documentation Key concepts ASQ 0511 Presentation 1/21/2009

The new 829 New directions Introduces the concept that the test effort has tasks to accomplish during the entire development life cycle not merely during the test activity. Moves from a document focus to a process focus. This is in keeping with the IEEE Standards Association direction. Moves away from stand-alone documents to various configurations. This standard recognizes that some projects may desire to have some stand-alone and some combined documents and allows for any combination of plan, design, test cases, and test procedures within test levels. ASQ 0511 Presentation 1/21/2009

Continued … The new 829 Document configuration example: Plan = [test plan] or [test plan + test design] Test cases = [test design + test cases + procedures] or [test cases + procedures] A particular project may decide to have a primary document that contains the test plan and the test design while having additional chapters that cover subsequent builds or functionalities. ASQ 0511 Presentation 1/21/2009

Continued … The new 829 New processes Introduces the concept of integrity levels. Provides a mechanism by which projects can identify their integrity level. The higher the integrity level the more test tasks that are recommended. Introduces the concept of test management. Describes tasks that are exclusive to those who manage a test effort. Adds a process for choosing appropriate documentation and contents. Introduces the concept of integrity levels. ASQ 0511 Presentation 1/21/2009

Continued … The new 829 New test related documentation Adds a Master Test Plan. This document governs the management of a large and/or complex test effort. Adds a Master Test Report. May summarize the results of the tasks identified in the Master Test Plan. May be used to consolidate results for multiple Level Test Reports. Adds a Level Interim Test Status Report. This is used during the test execution activity. Moves away from requiring identical documentation. This standard provides for documentation based on the integrity level of the project. Identifies minimum recommended tasks for the identified integrity level. ASQ 0511 Presentation 1/21/2009

Continued … new 829 Key concepts: Integrity Levels. Defines (example) four integrity levels to describe the importance of the software or system aspects to the user. The process of identifying the integrity level is the criticality analysis. Each project or organization identifies the system or software characteristic that is most important. For one it might be security and for another it might be reliability. For a third organization it may be the impact of failure. ASQ 0511 Presentation 1/21/2009

Continued … The new 829 Key concepts (continued): Recommended minimum testing tasks for each integrity level. Defines the recommended minimum testing tasks required for each of the four integrity levels. Includes a table of optional testing tasks for tailoring the test effort to meet project needs and application specific characteristics. Systems viewpoint. Includes recommended minimum test tasks to respond to system needs. A low integrity level project such as an internal bug-tracking program require fewer test tasks than would a high integrity level project such as one developing software/firmware for medical devices. ASQ 0511 Presentation 1/21/2009

Continued … The new 829 Key concepts (continued): Intensity and rigor applied to testing tasks. Introduces the notion that the integrity and rigor applied to testing tasks vary according to the integrity level. Higher integrity levels require the application of greater intensity and rigor. A high integrity level project such as one developing medical devices may execute a myriad of tests for each unit as well as for integration and system and acceptance tests. These tests will go to the depth of each test level looking for every conceivable deficiency. While a low integrity level project may do only acceptance testing against the primary functionalities rather than doing system testing against the specific requirements. ASQ 0511 Presentation 1/21/2009

Continued … key concepts Key concepts (continued): Detailed criteria for testing tasks. Defines specific criteria for each testing task including minimum recommended criteria for correctness, consistency, completeness, accuracy, readability, and testability. For each test task, includes a list of minimum inputs and outputs. Systems viewpoint Previously, a test manager would have to make a “best guess as to the adequacy and completeness of any test document. Now specific guidelines are provided to assist in this process. This standard recognizes that software does not exist in isolation and that much of current software development may actually be for software intensive systems or for embedded firmware. Thus the entire system needs to be taken into account when identifying the integrity level and the resultant test tasks, both the minimum recommended tasks and the optional tasks. ASQ 0511 Presentation 1/21/2009

Continued … The new 829 Key Concepts (continued): Selection of test documentation. Both the types of test documentation and the content topics within each documentation type need to be selected based on the testing tasks associated with the identified integrity level. The prior standard required every project to use the same test documents and to include the same information. The current standard provides for tailoring based on the integrity level. Thus a high integrity level project will require the full range of test tasks and test documentation as described in the standard. Conversely a low integrity level project may require only a minimum quantity of test plan information and a full range of test case and test procedure information. ASQ 0511 Presentation 1/21/2009

Continued … The new 829 Key Concepts (continued): Compliance with International and IEEE Standards. The standard is mapped to specific content requirements of IEEE/EIA 12207.0-1997 and IEEE/EIA 12207.0-1998. It is similarly mapped to IEEE/EIA 12207.1-1997 and IEEE/EIA 12207.1-1998. In addition it is in conformance with IEEE Std. 1012-2004 and is applicable for use with ISO 15288. ASQ 0511 Presentation 1/21/2009

Future Needs IEEE Needs User needs Evolving IEEE standards Synching IEEE standards with ISO standards User needs Evolving technologies ??? ASQ 0511 Presentation 1/21/2009

Conclusion The test activity is part of the overall engineering process The test tasks will reflect the overall test approach (strategy) and the development methodology If a waterfall methodology is being used then each applicable level of testing will be executed only one time (plus bug fixes). If it is being developed iteratively then each applicable level of testing will be executed multiple times. The test team may be doing acceptance test on the first iteration and be doing integration testing on a subsequent iteration. ASQ 0511 Presentation 1/21/2009

Continued … conclusion The new 829 guides the thinking for test planning The test documents are the culmination of the test planning activity not the beginning Tests are executed based on the applicable test documentation Test results are analyzed Test reports based on test execution and test results analysis are generated Test metrics are prepared and delivered to project management ASQ 0511 Presentation 1/21/2009

Questions - Comments QUESTIONS???? Comments???? ASQ 0511 Presentation 1/21/2009

Contact Information: Eva Freund, CSDP, CSQE Principal The IV&V Group, Inc. 703.573.7466 – office 703.731.8915 – cell www.ivvgroup.com ASQ 0511 Presentation 1/21/2009