Test Plan: Introduction o Primary focus: developer testing –Implementation phase –Release testing –Maintenance and enhancement o Secondary focus: formal.

Slides:



Advertisements
Similar presentations
Making the System Operational
Advertisements

Configuration Management
Software Testing. Quality is Hard to Pin Down Concise, clear definition is elusive Not easily quantifiable Many things to many people You'll know it when.
SPL/2010 Test-Driven Development (TDD) 1. SPL/
Test Execution and Defect management. 2 Module Objectives Introduction to Test Execution Checklist of Test Execution Defect management Defect Classification.
Automated Software Testing: Test Execution and Review Amritha Muralidharan (axm16u)
HP Quality Center Overview.
Chapter 4 Quality Assurance in Context
Verification and Validation: A Quick Introduction 1-2 Lectures.
Verification and Validation: A Quick Introduction Authors Massood Towhidnejad Massood Towhidnejad Mike Rowe Mike Rowe David Dampier David Dampier Sponsored.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 23 Slide 1 Software testing.
06/12/2007SE _6_12_Design.ppt1 Design Phase Outputs: Completed & Inspected SDS & Integration Test Plan Completed & Inspected System Test Plan.
Software Testing for Safety- Critical Applications Presented by: Ciro Espinosa & Daniel Llauger.
I n t e g r i t y - S e r v i c e - E x c e l l e n c e Business & Enterprise Systems Introduction to Hewlett Packard (HP) Application Lifecycle Management.
© 2006 Pearson Addison-Wesley. All rights reserved2-1 Chapter 2 Principles of Programming & Software Engineering.
Outline Types of errors Component Testing Testing Strategy
Introduction to Software Testing
Software Testing & Strategies
Issues on Software Testing for Safety-Critical Real-Time Automation Systems Shahdat Hossain Troy Mockenhaupt.
10th TTCN-3 User Conference, 7-9 June 2011, Bled, Slovenia AUTOSAR Conformance Tests - Feedback on their development and utilization Alain Feudjio-Vouffo,
Software Testing Verification and validation planning Software inspections Software Inspection vs. Testing Automated static analysis Cleanroom software.
University of Palestine software engineering department Testing of Software Systems Fundamentals of testing instructor: Tasneem Darwish.
Extreme Programming Software Development Written by Sanjay Kumar.
Testing. Definition From the dictionary- the means by which the presence, quality, or genuineness of anything is determined; a means of trial. For software.
University of Palestine software engineering department Testing of Software Systems Fundamentals of testing instructor: Tasneem Darwish.
INFO 637Lecture #81 Software Engineering Process II Integration and System Testing INFO 637 Glenn Booker.
CompSci 230 Software Design and Construction
© 2012 IBM Corporation Rational Insight | Back to Basis Series Chao Zhang Unit Testing.
CPIS 357 Software Quality & Testing
Software Testing.
Software Inspection A basic tool for defect removal A basic tool for defect removal Urgent need for QA and removal can be supported by inspection Urgent.
1 Debugging and Testing Overview Defensive Programming The goal is to prevent failures Debugging The goal is to find cause of failures and fix it Testing.
Understand Application Lifecycle Management
INT-Evry (Masters IT– Soft Eng)IntegrationTesting.1 (OO) Integration Testing What: Integration testing is a phase of software testing in which.
Testing Basics of Testing Presented by: Vijay.C.G – Glister Tech.
Ch6: Software Verification. 1 Decision table based testing  Applicability:  Spec. is described by a decision table.  Tables describe:  How combinations.
Moving into Implementation SYSTEMS ANALYSIS AND DESIGN, 6 TH EDITION DENNIS, WIXOM, AND ROTH © 2015 JOHN WILEY & SONS. ALL RIGHTS RESERVED.Roberta M. Roth.
Large Scale Software Systems Derived from Dr. Fawcett’s Notes Phil Pratt-Szeliga Fall 2010.
Testing. 2 Overview Testing and debugging are important activities in software development. Techniques and tools are introduced. Material borrowed here.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 22 Slide 1 Software Verification, Validation and Testing.
TEST-1 6. Testing & Refactoring. TEST-2 How we create classes? We think about what a class must do We focus on its implementation We write fields We write.
1 Phase Implementation. Janice Regan, Overview of Implementation phase Create Class Skeletons Define Implementation Plan (+ determine subphases)
What is Testing? Testing is the process of finding errors in the system implementation. –The intent of testing is to find problems with the system.
Software Maintenance Speaker: Jerry Gao Ph.D. San Jose State University URL: Sept., 2001.
Software Development Problem Analysis and Specification Design Implementation (Coding) Testing, Execution and Debugging Maintenance.
© 2006 Pearson Addison-Wesley. All rights reserved 2-1 Chapter 2 Principles of Programming & Software Engineering.
Software Engineering1  Verification: The software should conform to its specification  Validation: The software should do what the user really requires.
1 Software Testing Strategies: Approaches, Issues, Testing Tools.
Software Quality Assurance and Testing Fazal Rehman Shamil.
Unit 17: SDLC. Systems Development Life Cycle Five Major Phases Plus Documentation throughout Plus Evaluation…
 Software Testing Software Testing  Characteristics of Testable Software Characteristics of Testable Software  A Testing Life Cycle A Testing Life.
Dynamic Testing.
HNDIT23082 Lecture 09:Software Testing. Validations and Verification Validation and verification ( V & V ) is the name given to the checking and analysis.
T EST T OOLS U NIT VI This unit contains the overview of the test tools. Also prerequisites for applying these tools, tools selection and implementation.
CS223: Software Engineering Lecture 18: The XP. Recap Introduction to Agile Methodology Customer centric approach Issues of Agile methodology Where to.
Testing Overview Software Reliability Techniques Testing Concepts CEN 4010 Class 24 – 11/17.
SwCDR (Peer) Review 1 UCB MAVEN Particles and Fields Flight Software Critical Design Review Peter R. Harvey.
System Components Operating System Services System Calls.
Software Testing Strategies for building test group
Software Project Configuration Management
Software Verification and Validation
Chapter ? Quality Assessment
Configuration Management Why do we need it? What does it do?
One Digital – Integrated Digital Assurance Automation Framework
Lecture 09:Software Testing
Baisc Of Software Testing
Chapter 2: Operating-System Structures
Introduction to Operating Systems
Chapter 2: Operating-System Structures
Presentation transcript:

Test Plan: Introduction o Primary focus: developer testing –Implementation phase –Release testing –Maintenance and enhancement o Secondary focus: formal system verification –Addressed within current test plan via examples –Assumed to be primarily independent

Tests to be Performed o Bottom-up integration testing –Build a module, build a test to simulate how it is used o Black-box –Based on specification and ‘educated guess’ stress tests o White-box –Based on code inspection o Platform testing –Establish baseline capability of hardware/OS, detect differences o Performance –Per module, and peer to peer distributed costs o Test coverage –Coverage level dependent on resources, time, and cost of failure

Allocation of Testing Responsibilities o Assumption: development team performs –Internal (module-level) performance –Sample system performance (limited example federation) –Full white-box –Limited black-box (basic system functionality) o Assumption: external group performs –Independent, detailed system verification (Spec compliant) – Standardized performance tests o Test coverage –Coverage will be measured, analyzed for relative effectiveness –Coverage levels TBD

Testing Philosophy o Testing is not just pre-release! Continual process within development phase o Catch defects as early as possible o Make sure defects stay fixed o Track cause of defects: repair problem, do not keep re- patching the tire o Need support at both design level and implementation level to accomplish these goals

Continual Testing Process o Tests created during development o Central code repository: modules, test suites tied together –Tests are treated as live code to be maintained, not ‘once-offs’ –Test documentation: how to run, what is tested, and why o Revision control on both modules and tests o Modular development –System broken down into hierarchies of testable components o Automated, incremental integration testing –As code is developed, it is tested standalone, then incrementally within the confines of the system o Continual feedback into development, maintenance cycle –Weekly postings of performance results, current defect status

Design Support o Standard testing and debugging methods on each module / class (peek, dump, exercise, performance) o Self-checking code (pre and post parameter asserts, valid state calls) o Debug levels, controlled via runtime flags o Centralized logging mechanisms to collect distributed traces o Logs used in both developer testing and sample user traces from the field

Development Tool Support o Shadow development trees for automated and individual testing (ClearCase ‘views’) o Common testing tools, testing approach to simplify testing process, and to allow automated testing o Examples: –Standard test harnesses, method of invocation –Standard testing directories, makefile commands per module –Standard set of test-record-evaluate tools –Central I/O mechanisms to collect distributed test results –Standard system-level drivers –Sequential test harnesses and emulated distributed harnesses to emulate determinism during development

Levels of Testing per Module o Basic: used in initial development, later in porting –Minimal level of functionality to establish module operation –Simplicity is critical during development, porting o Detailed: used to verify module correctness before checking into current baseline –Does module meet interface contract –Tests for common coding errors o Regression: replicates previous use which caused failure –Used to verify defect has been corrected –Used to ensure defect does not re-occur over time o Performance: tracks the CPU usage per module and peer to peer performance across distributed components

Complex Functional Areas: Testing Examples o Memory: continual test for memory leaks and over-writes –Automated use of Purify within weekly test suites across all modules, all levels of the system o Threads: platform-specific performance and implementation variances must be established per platform –Standard set of tests which mimic system use of threads o Causality: complex areas of the system (such as zero- lookahead ordering) are difficult to establish correctness across all use cases (dropped packets, simultaneous time stamps with variable arrival times, cancelled events, …) –Detailed test scripts, executed in deterministic test harnesses with varying error conditions

Periodic Testing Activities o Code walkthroughs: encompass both system code and associated test suites. Testing focus: –Do the test suites sufficiently stress the module –Do the test suites still accurately represent the expected use of the module –Has the underlying platform changed, and has performance changed accordingly o Change Control Board: formal tracking process and tools to establish, record and monitor status of functional change requests, defect priority and status data