WARNING These slides are not optimized for printing or exam preparation. These are for lecture delivery only. These slides are made for PowerPoint They may not show up well on other PowerPoint versions. You can download PowerPoint 2010 viewer from here.here These slides contain a lot of animations. For optimal results, watch in slideshow mode. 30 min

But it worked in my machine! Our tester Your software

Quality Assurance: Testing and Beyond CS2103/T, Lecture 9, Part 1, [Oct 18, 2013]

ValidationVerification [Quality Assurance]

ValidationVerification [Quality Assurance] Are we building the correct product? Are we building the product correctly?

ValidationVerification [Quality Assurance] Are we building the correct product? Are we building the product correctly?

ValidationVerification [Quality Assurance] Are we building the correct product? Are we building the product correctly? FAIL!

ValidationVerification [Quality Assurance] Are we building the correct product? Are we building the product correctly? FAIL! FAIL!

ValidationVerification [Quality Assurance] Buggy requirements Buggy code FAIL! FAIL!

ValidationVerification [Quality Assurance] Two QA activities for a PC version of a ‘Snakes and ladders’ game: a) Ensure that the game allows players to decide who throws the die first. b) When more than one piece is on a square, ensure that it is the piece that belongs to the current player that will be moved. (Are we building the correct product?) (Are we building the product correctly?) snakes {a|b} e.g. snakes a snakes {a|b} e.g. snakes a tinyurl.com/questionSMS Which one is more related to verification than validation?

ValidationVerification [Quality Assurance]

ValidationVerification [Quality Assurance]

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance]

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] Product

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] Product

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] Product

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] Product But it worked in my machine!

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] Product

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] Product System specification Requirements specification

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] Product System specification Requirements specification

ValidationVerification [Quality Assurance] Product E.g. Acceptance testingE.g. System testing

ValidationVerification [Quality Assurance] Product E.g. Acceptance testingE.g. System testing

ValidationVerification [Quality Assurance] Product E.g. Acceptance testingE.g. System testing

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] Product

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] GUI testing

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] GUI testing

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] GUI testing

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] GUI testing GUI Logic Automated API tester Manual testing (or automate)

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] GUI testing GUI Logic Automated API tester Manual testing (or automate)

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] GUI testing

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance] GUI testing

ValidationVerification E.g. Acceptance testingE.g. System testing Back-end GUI [Quality Assurance]

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested?

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested?

Using EclEmma coverage tool on TEAMMATES

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested?

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? Statement coverage Function coverage Condition coverage Exit/entry coverage Branch coverage Path coverage

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? Statement coverage Path coverage

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? Statement coverage Path coverage void readInput(){ while ( hasInput() ) input += readFromKeyboard(); }

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? Statement coverage void readInput(){ while ( hasInput() ) input += readFromKeyboard(); }

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? void readInput(){ while ( hasInput() ) input += readFromKeyboard(); } Path coverage

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? void readInput(){ while ( hasInput() ) input += readFromKeyboard(); } Path coverage

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? void readInput(){ while ( hasInput() ) input += readFromKeyboard(); } Path coverage

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? void readInput(){ while ( hasInput() ) input += readFromKeyboard(); } Path coverage

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? void readInput(){ while ( hasInput() ) input += readFromKeyboard(); } Path coverage

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? Path coverage void readInput(){ while ( hasInput() ) input += readFromKeyboard(); } Control Flow Graphs

ValidationVerification E.g. Acceptance testingE.g. System testing [Quality Assurance] ← How much of this is being tested? Statement coverage Function coverage Condition coverage Exit/entry coverage Branch coverage Path coverage

ValidationVerification E.g. Acceptance testing [Quality Assurance] E.g. System testing just testing?

ValidationVerification E.g. Acceptance testing [Quality Assurance] E.g. System testing Other QA techniques just testing?

[Quality Assurance] Other QA techniques

ValidationVerification E.g. Acceptance testing [Quality Assurance] Code reviews static analysis formal methods … Other QA techniques

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods …

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods …

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods …

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods …

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods … Using CodeProAnalytix static analysis tool on TEAMMATES

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods …

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods … SUT doubleAmount(int):int SUT doubleAmount(int):int

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods … Text case 1: Input: 23 Expected output: 46 Text case 2: Input: 10 Expected output : 20 Text case n: Input: -2 Expected output : -4 … Testing SUT doubleAmount(int):int SUT doubleAmount(int):int

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods … Mathematically prove if input = x then, output = 2x Testing SUT doubleAmount(int):int SUT doubleAmount(int):int Text case 1: Input: 23 Expected output: 46 Text case 2: Input: 10 Expected output : 20 Text case n: Input: -2 Expected output : -4 …

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods …

ValidationVerification E.g. Acceptance testing [Quality Assurance] Other QA techniques Code reviews static analysis formal methods …

Have the ability to relate these concepts to the project: validation, verification, system testing, acceptance testing, code coverage, GUI testing, other QA techniques. For example, the ability to explain how the above was done in the project, or why they were not done and how you could have done them. You are welcome to, but not required to, use Automation of GUI testing Using tools to measure coverage Path analysis Apply other QA techniques

If all tests pass and those tests achieve100% path coverage (all possible execution paths through the code have been tested), can the software still have bugs? (Discuss with neighbors before submitting) buggy {yes|no} e.g. buggy yes buggy {yes|no} e.g. buggy yes tinyurl.com/questionSMS