Fall 2015CISC/CMPE320 - Prof. McLeod1 CISC/CMPE320 Presentations next week. Presentation stuff again. Finish Testing. Lots more jargon!

Slides:



Advertisements
Similar presentations
1 Integration Testing CS 4311 I. Burnstein. Practical Software Testing, Springer-Verlag, 2003.
Advertisements

Annoucements  Next labs 9 and 10 are paired for everyone. So don’t miss the lab.  There is a review session for the quiz on Monday, November 4, at 8:00.
Using UML, Patterns, and Java Object-Oriented Software Engineering Chapter 11: Integration- and System Testing.
Software Testing 3 Damian Gordon.
CMSC 345, Version 11/07 SD Vick from S. Mitchell Software Testing.
Chapter 9 Testing the System, part 2. Testing  Unit testing White (glass) box Code walkthroughs and inspections  Integration testing Bottom-up Top-down.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 23 Slide 1 Software testing.
Illinois Institute of Technology
Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s.
1 Testing. 2 About Testing  The reason the program is in testing is that it probably doesn’t work!  We test to find bugs before our users and hope that.
Outline Types of errors Component Testing Testing Strategy
Using UML, Patterns, and Java Object-Oriented Software Engineering Chapter 11: Integration- and System Testing.
Chapter 11: Testing The dynamic verification of the behavior of a program on a finite set of test cases, suitable selected from the usually infinite execution.
Software Testing & Strategies
BY RAJESWARI S SOFTWARE TESTING. INTRODUCTION Software testing is the process of testing the software product. Effective software testing will contribute.
Chapter 13 & 14 Software Testing Strategies and Techniques
Functional Testing Test cases derived from requirements specification document – Black box testing – Independent testers – Test both valid and invalid.
Estimation Wrap-up CSE 403, Spring 2008, Alverson Spolsky.
System/Software Testing
Testing Chapter 11. Dealing with Errors Verification: –Makes assumptions –Doesn’t always deal with real environment Testing (this lecture) –Testing is.
ECE 355: Software Engineering
Software Testing. Recap Software testing – Why do we do testing? – When it is done? – Who does it? Software testing process / phases in software testing.
Overview Integration Testing Decomposition Based Integration
SOFTWARE TESTING STRATEGIES CIS518001VA : ADVANCED SOFTWARE ENGINEERING TERM PAPER.
Object-Oriented Software Engineering, Ch. 9
CMSC 345 Fall 2000 Unit Testing. The testing process.
Prof. Mohamed Batouche Software Testing.
Chapter 8 – Software Testing Lecture 1 1Chapter 8 Software testing The bearing of a child takes nine months, no matter how many women are assigned. Many.
Overview of Software Testing 07/12/2013 WISTPC 2013 Peter Clarke.
INT-Evry (Masters IT– Soft Eng)IntegrationTesting.1 (OO) Integration Testing What: Integration testing is a phase of software testing in which.
1 Software testing. 2 Testing Objectives Testing is a process of executing a program with the intent of finding an error. A good test case is in that.
Testing Basics of Testing Presented by: Vijay.C.G – Glister Tech.
Software Testing. 2 CMSC 345, Version 4/12 Topics The testing process  unit testing  integration and system testing  acceptance testing Test case planning.
Software Testing Reference: Software Engineering, Ian Sommerville, 6 th edition, Chapter 20.
Chapter 8 Lecture 1 Software Testing. Program testing Testing is intended to show that a program does what it is intended to do and to discover program.
LECTURE 19 23/11/15 Software Quality and Testing.
Software Engineering1  Verification: The software should conform to its specification  Validation: The software should do what the user really requires.
1 CSE 403 Testing Reading: Code Complete, Ch. 22 (McConnell) Object-Oriented Software Engineering, Ch. 9 (B. Bruegge, A. Dutoit) These lecture slides are.
Fall 2015CISC/CMPE320 - Prof. McLeod1 CISC/CMPE320 RAD due Friday in your Wiki. Presentations week 6 – next week. Schedule on next slide. Today: –Operator.
CSC 480 Software Engineering Test Planning. Test Cases and Test Plans A test case is an explicit set of instructions designed to detect a particular class.
Integration testing Integrate two or more module.i.e. communicate between the modules. Follow a white box testing (Testing the code)
Fall 2015CISC/CMPE320 - Prof. McLeod1 CISC/CMPE320 RAD due Friday in your Wiki. Presentations week 6 – next week. Schedule on next slide. Today: –Operator.
Fall 2015CISC/CMPE320 - Prof. McLeod1 CISC/CMPE320 Project Wikis will be made public tonight. Presentations week 6 – next week. Slack Chat invites sent.
CS451 Lecture 10: Software Testing Yugi Lee STB #555 (816)
1 Software Testing Strategies: Approaches, Issues, Testing Tools.
Integration Testing Beyond unit testing. 2 Testing in the V-Model Requirements Detailed Design Module implementation Unit test Integration test System.
HNDIT23082 Lecture 09:Software Testing. Validations and Verification Validation and verification ( V & V ) is the name given to the checking and analysis.
Fall 2015CISC/CMPE320 - Prof. McLeod1 CISC/CMPE320 Assignment 4 is due Nov. 22 (this Sunday). Presentation stuff. Today, More Software Engineering: –System.
Testing Overview Software Reliability Techniques Testing Concepts CEN 4010 Class 24 – 11/17.
CS 325: Software Engineering February 16, 2016 Designing a Design Class Diagram Design Class Diagrams DCD: Restaurant Example DCD: ATM Example Software.
Software Testing Reference: Software Engineering, Ian Sommerville, 6 th edition, Chapter 20.
CSC 395 – Software Engineering Lecture 27: White-Box Testing.
Defect testing Testing programs to establish the presence of system defects.
Software Testing Kobla Setriakor Nyomi Faculty Intern (Programming II)
Software Testing Strategies for building test group
Integration Testing.
Rekayasa Perangkat Lunak Part-13
Chapter 13 & 14 Software Testing Strategies and Techniques
Lecture 09:Software Testing
Static Testing Static testing refers to testing that takes place without Execution - examining and reviewing it. Dynamic Testing Dynamic testing is what.
Software testing.
CSE 303 Concepts and Tools for Software Development
Chapter 10 – Software Testing
Test Case Test case Describes an input Description and an expected output Description. Test case ID Section 1: Before execution Section 2: After execution.
Software Testing “If you can’t test it, you can’t design it”
Chapter 11: Integration- and System Testing
Chapter 11: Integration and System Testing
CS410 – Software Engineering Lecture #11: Testing II
Software Testing Strategies
Chapter 13 & 14 Software Testing Strategies and Techniques 1 Software Engineering: A Practitioner’s Approach, 6th edition by Roger S. Pressman.
Presentation transcript:

Fall 2015CISC/CMPE320 - Prof. McLeod1 CISC/CMPE320 Presentations next week. Presentation stuff again. Finish Testing. Lots more jargon!

Week 12 Team Presentations Demonstrate of the fruit of your efforts! And, summarize (in any order you wish): –How work ended up being distributed and when it was completed. –Difficulties faced and overcome. –Difficulties not overcome. –Good/bad library use. –Work left to do, if any. –Team techniques that worked and those that did not. –What you would do differently if you had to do this again… Fall 2015CISC/CMPE320 - Prof. McLeod2

Presentation Schedule – Week 12 Tuesday, Dec. 1 – Basswood, Beech, Cherry, Tamarack. Thursday, Dec. 3, Lecture Time – Hickory, Maple, Oak, Birch. Thursday, Dec. 3, Tutorial Time (JEF 128) – Poplar, BalsamFir, Spruce, Cedar. Friday, Dec. 4 – Hemlock, JackPine, Walnut, WhitePine. Fall 2015CISC/CMPE320 - Prof. McLeod3

Peer Grading, Again. Sign-up sheets will be circulated again during presentations. Project Wikis will be make public on Nov. 25. As for last time – two Moodle surveys: –The Wiki: Is the SDD up-to-date and does it accurately describe the architecture? Is the Wiki looking better than the last time you saw it? –The Presentation: Organized, effective, met initial goals, good discussion of problems overcome? Fall 2015CISC/CMPE320 - Prof. McLeod4

Fall 2015CISC/CMPE320 - Prof. McLeod5 Unit Testing Common types of unit testing: –Equivalence testing –Boundary testing –Path testing –State-based testing

Fall 2015CISC/CMPE320 - Prof. McLeod6 Equivalence Testing A blackbox technique. –(Deal with the input/output of the component, leave the internal behaviour alone.) Minimal number of test cases. First, determine a set of equivalence classes. You assume that the component behaves the same for all members of the equivalence class.

Fall 2015CISC/CMPE320 - Prof. McLeod7 Equivalence Testing, Cont. Criteria for equivalence classes: –Coverage – every possible input belongs to one of the equivalence classes. –Disjointedness – no input belongs to more than one equivalence class. –Representation – if one member of an equivalence class causes an erroneous state then the same state can be caused by any other member of that class. For each class, select at least two test inputs – a valid input and an input that should exercise your exception handlers.

Fall 2015CISC/CMPE320 - Prof. McLeod8 Equivalence Testing, Cont. Example – suppose you are testing a class that stores dates, as given by a year and month. You would need six equivalence classes: –31 day months, non-leap years –31 day months, leap years –30 day months, non-leap years –30 day months, leap years –28 day month, non-leap years –29 day month, leap year Minimum of 12 tests!

Fall 2015CISC/CMPE320 - Prof. McLeod9 Boundary Testing Look at the “edges” of the equivalence classes. For example: –Leap year divisible by 400 (year 2000, for example) –Non-leap year divisible by 100 (year 1900, for example) –Invalid month (at negative edge of legal months): 0 –Invalid month (at positive edge of legality): 13

Fall 2015CISC/CMPE320 - Prof. McLeod10 Equivalence and Boundary Testing The problem with these techniques is that they do not explore many combinations of input data. Often a strange combination will cause an error, and this will take many more tests to locate. (My instincts say that there are just not enough tests!)

Fall 2015CISC/CMPE320 - Prof. McLeod11 Path Testing A whitebox technique. –(Focus on the internal structure of the component. Test every state of the component and every possible interaction of objects.) Exercise every possible path through the code at least once. –Exercise every branch of every if statement. –Run each loop – can you cause a loop not to run? –Throw each exception.

Fall 2015CISC/CMPE320 - Prof. McLeod12 Path Testing, Cont. The problem with this technique is that it cannot show where code is missing – it only tests existing code.

Fall 2015CISC/CMPE320 - Prof. McLeod13 State – Based Testing Developed for GUI systems. A new technique that is not fully fleshed out. Based on the idea that objects (like a button) can have different states under different conditions (like active, inactive, pressed, mouseover, etc.) See: WindowTester (for Java) from Google, for example: tools/wintester/html/index

Fall 2015CISC/CMPE320 - Prof. McLeod14 Polymorphism Testing OOP provides additional challenges to testing when a program takes advantage of polymorphism. You must make sure that every state of an object is tested. This can lead to many more tests especially when polymorphism is used by several objects in a component.

Fall 2015CISC/CMPE320 - Prof. McLeod15 Integration Testing Carried out after unit testing is complete. Start with the smallest combinations of components possible and then let the groups get larger as testing proceeds. Still need drivers and stub code. (Note that extreme programming requires that all drivers are written before components are developed!).

Fall 2015CISC/CMPE320 - Prof. McLeod16 Integration Testing, Cont. Four strategies: –Big Bang testing –Bottom-Up testing –Top-Down testing –Sandwich testing (Who thinks up these names…)

Fall 2015CISC/CMPE320 - Prof. McLeod17 Big Bang Testing Once unit testing is complete, test full system. You don’t need any more drivers, but it is very hard to pinpoint the source of an error.

Fall 2015CISC/CMPE320 - Prof. McLeod18 Bottom-Up Testing Start at the lowest layer (independent classes). Combine classes one at a time – double testing, triple testing, quadruple testing, etc. No stub code is required, only drivers to simulate the level above the one being tested. Sounds nice and safe to me!

Fall 2015CISC/CMPE320 - Prof. McLeod19 Bottom-Up Testing, Cont. Interface faults are easily found. The problem with Bottom-Up is that UI components are tested last. A correction in this important top layer may require changes all the way back down…

Fall 2015CISC/CMPE320 - Prof. McLeod20 Top-Down Testing Opposite to Bottom-Up (that’s a surprise!) Uses stub code for the lower layers, as required. No drivers required. Starts with the UI – the most important part of the system.

Fall 2015CISC/CMPE320 - Prof. McLeod21 Sandwich Testing Combination of bottom-up and top-down. Break the system down into three layers (to form the sandwich). Top, Bottom and Middle. Apply both bottom-up and top-down at the same time. You will not need drivers and stubs for the outer layers. “Modified Sandwich Testing”: Test the layers independently and then test all together.

Fall 2015CISC/CMPE320 - Prof. McLeod22 System Testing Test the entire system to make sure in complies with the functional and non-functional requirements. Carry out: –Functional testing –Performance testing –Pilot testing –Acceptance testing –Installation testing

Fall 2015CISC/CMPE320 - Prof. McLeod23 System Testing, Cont. Functional testing: –Also called requirements testing. –Compare the system to the functional requirements. –Blackbox technique. –Tests are extracted from the use case models. –The tester should choose the tests that are of the most importance to the user and the most likely to demonstrate a failure. –Test cases are created as they were with equivalence and boundary testing.

Fall 2015CISC/CMPE320 - Prof. McLeod24 System Testing, Cont. Performance testing: –Examine the difference between the system and the design goals. –Extract tests from the SDD or the RAD. –Examples: Stress testing Volume testing Security testing Timing tests Recovery tests

Fall 2015CISC/CMPE320 - Prof. McLeod25 System Testing, Cont. Pilot testing: –See “Alpha Testing” Acceptance testing: –The client prepares benchmark tests. –You might test the new system vs. the legacy system. Installation testing: –The testing of the system after installation. –Probably repeat many of the earlier tests. –(Can’t take too long!)

Fall 2015CISC/CMPE320 - Prof. McLeod26 Post Release Testing This brief discussion above was all about pre- release testing. Alpha Testing is carried out by a small team of potential customers on site. Beta Testing is all the rage these days. Release to the public, but on a narrow basis. Must have some way of getting feedback from the beta testers in return for getting free software ahead of other people.

Fall 2015CISC/CMPE320 - Prof. McLeod27 Corrections What to do when you find a fault? How do you reduce the odds of introducing new faults into the repaired component? (A big topic!) Techniques that can be used: –Problem tracking –Configuration management –Regression testing –Rationale maintenance

Fall 2015CISC/CMPE320 - Prof. McLeod28 Measuring Software Testing (Wikipedia) Common “metrics” used to quantify the state of the software: –Bugs found per tester per unit time (day/week/month). –Total bugs found in a release. –Total bugs found in a module or feature. –Bugs found or fixed per build. –Number of customer reported bugs. –(Continued…)

Fall 2015CISC/CMPE320 - Prof. McLeod29 Measuring Software Testing, Cont. –Bug trend over the period in a release. (Bugs should converge towards zero as the project gets closer to release. It is possible that there are more cosmetic bugs found closer to release - in which case the number of critical bugs found is used instead of total number of bugs found.) –Number of test cases executed per person per unit time. –Percent of test cases executed so far, total pass, total fail.

Fall 2015CISC/CMPE320 - Prof. McLeod30 Bugs in Vista Beta 2 Release From Robert McLaws of Longhorn Blogs (2006): –An average of 81 unique bugs per day were reported for Vista. –The count of bugs per day was increasing, not decreasing. –Around 200 bugs are reported within the first 24 hours of a new release. –Over 20,000 bugs have been closed so far where closed holds a status of "Closed" or "Resolved“. –When Microsoft released the Start Orb, 353 bugs were added to Connect during the first day and 338 during the second. –One fifth of the total bugs submitted were still open in 2008.

Windows 10? Do a Google search for “Windows 10 bugs”… Fall 2015CISC/CMPE320 - Prof. McLeod31