Download presentation
Presentation is loading. Please wait.
Published byBeverly Bryant Modified over 9 years ago
1
Testing
2
Today’s Topics Why Testing? Basic Definitions Kinds of Testing Test-driven Development Code Reviews (not testing) 1
3
Why Testing? Ideally, we’d prove that the code was correct using formal mathematics This is extremely difficult even for trivially small programs Not practical in most cases Not even for mission critical or safety code 2
4
Why Testing? Near ideal: use symbolic or abstract model testing Prove that model is correct Automatically extract mathematical model from code Prove properties of model for all possible cases In practice, can work well for simple properties E.g., “System will not crash in this way” Doesn’t work well for complex systems / data structures 3
5
Why Testing? We can actually run code to see if it works This is software testing Always necessary, even when you can prove correctness Proof seldom tied to actual code Testing is NOT a lost resort; it is critical part of sw dev 4
6
Why Testing? Testing all too often an afterthought Test before faults become too expensive to fix or pressing deadlines The more code written before fault is detected, the more code needs to be fixed… Better to detect fault(s) on a small prototype or “finished” product? 5
7
Waterfall 6 Requirements analysis Design Implementation Operation Testing Prototyping
8
Spiral 7 Draft a menu of requirements Establish requirements Plan Analyze risk & prototype Draft a menu of architecture designs Analyze risk & prototype Draft a menu of program designs Establish architecture Establish program design Implementation Testing Operation Plan
9
Agile 8 Customer provides “stories” (short requirement snippets) System and acceptance tests Do “spike” to evaluate & control risk Prioritize stories and plan Implement Operation Write/run/modify unit tests
10
Testing Saves Lives and Money NIST Report: “The Economic Impact of Inadequate Infrastructure for Software Testing” Inadequate software testing costs US $22 - $59 billion annually Better approaches could cut this is half 9
11
Testing Saves Lives and Money Major failures: Ariane 5 explosion Mars Polar Lander Intel Pentium FDIV bug THERAC-25 radiation machine: killed 5 10 Ariane 5- exception-handling bug forced self destruct on maiden flight (64-bit to 16-bit conversion: about $370 mill lost)
12
Today’s Topics Why Testing? Basic Definitions Kinds of Testing Test-driven Development Code Reviews (not testing) 11
13
Definitions: Testing Running a program to find faults (bugs) … in code … in specs … in documentation … in tests 12
14
Definitions: Test Case, Test Suite, Tester Test case: one execution of the program that may find a bug Test suite: a set of program executions, grouped together Made of up test cases Tester: program that generates tests 13
15
Definitions: Coverage Measures or metrics Abstraction of “what a test suite tests” Common measures: Statement coverage Decision coverage Path coverage Mutation coverage 14
16
Definitions: Coverage Statement coverage: aka: line or basic block coverage Which statements execute in test suite Decision coverage: Which boolean expressions in control structures evaluate to true and false during suite execution. 15
17
Definitions: Coverage Path coverage: Which paths through a program’s control flow graph are taken in the test suite Mutation coverage: Ability to detect random variations in the code 16
18
Today’s Topics Why Testing? Basic Definitions Kinds of Testing Test-driven Development Code Reviews (not testing) 17
19
Kinds of Testing White box Black box Unit Integration System Acceptance 18
20
White Box Testing Opens up the box aka glass box, clear box, structural testing Use source code to design test cases 19
21
Black Box Testing Testing does NOT look at source code or internal structures Sends program stream of inputs, observes outputs Abstracts away the internals Useful perspective for system and integration testing Sometimes you don’t have access to the source code 20
22
Testing Stages Unit testing: first phase; done by module develops Integration testing: combines unit tested modules, tests interactions System testing: test whole program to make sure reqs met Acceptance testing: tests by users to see if system meets reqs 21
23
Unit Testing First phase; mostly done by module developers Typically earliest type of testing Unit could be as small as single function or method Often relies on “stubs” to represent other modules or incomplete code Tools to support unit tests available for most popular languages, e.g., Junit ( http://junit.org )http://junit.org 22 ?
24
Integration Testing Combines unit-tested modules and tests interactions Relies on completed units After unit testing, before system testing Test cases focus on interfaces between components and assemblies of multiple components Often more formal than unit testing More detailed test plan(s) 23 ? ? ?
25
System Testing Tests whole program; make sure reqs met After integration testing Focuses on “breaking the system” Finds defects in completed product, not just component interactions Checks quality of requirements as well as system Often includes stress testing: beyond bounds of well-defined behavior 24
26
System Testing: Functional Testing Developer tests program from “user’s” perspective Different mindset than unit testing Analogy: home inspection vs. walk-through inspector => software engineer; looking for “hard facts” buyer => system user; looking for functionality 25
27
Acceptance Testing Tests by user to see if system meets use requirements Form of black box testing Done by end-users to determine if system really meets needs May revise requirements May find bugs in the system 26
28
Today’s Topics Why Testing? Basic Definitions Kinds of Testing Test-driven Development Code Reviews (not testing) 27
29
Test Drive Development Write test cases before code Make sure code is tested as early as possible Idea arises from Extreme Programming Often used in agile development Write automated test cases first Then write code to satisfy tests 28
30
Test Driven Development How to add a feature: 1.Add a test case that fails (but succeeds with new feature) 2.Run all tests, make sure only new test fails 3.Write code to implement new feature 4.Rerun all tests, make sure new test succeeds (no others fail) 29
31
30 Test Driven Development Cycle
32
Test Driven Development Benefits Results in lots of useful test cases A very large regression set Forces attention to controllable and observable sys behavior Only write code as needed to pass tests May get good coverage of paths through program Reduces temptation to tailor tests to idiosyncratic behaviors Testing is a first-class activity in this kind of development 31
33
Problems with Test Driven Development Needs institutional support Difficult to integrate with waterfall process Management may wonder why time is spent writing tests, not code Many test cases may create false confidence If developers have written all tests, there may be blind spots due 32
34
Exhaustive vs. Representative Testing Can we test everything? (exhaustive) Probably not! Permutations / combinations are exponential! Need to be thoughtful about creating representative tests Create tests at boundary conditions Want to “represent” the range of input values 33
35
Today’s Topics Why Testing? Basic Definitions Kinds of Testing Test-driven Development Code Reviews (not testing) 34
36
Code Reviews Code Review ≠ Testing Method for finding bugs and determining code quality Code walkthrough: developer leads review team through code Informal Code inspection: review team checks against concerns Prepared offline (in many cases) Led by a team moderator 35
37
Code Reviews Code inspections: found to be one of the most effective practices for finding faults Some experiments show removal of 67-85% of defects via inspections Some consider XP’s pair programming as a kind of “code review” process Can review/walkthrough requirements and design docs, not just code 36
38
Next Steps Homework 5 due Friday!! Vision paper due 11/21!! Midterm due 11/25!! 37
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.