1. Black box (Finite State Machine) testing Design for testability Coverage measures Random testing Constraint-based testing Debugging and test case minimization Using model checkers for testing Coverage revisited (“small model property”) Topics in Testing We’ll Cover

2. Testing Project Start with: interface and some definitions You – Implement (w ith intentional bugs) – Provide some (possibly VERY minimal) unit tests – Exchange programs with two other students – Submit a test plan for colleagues’ code – Present & defend test plan (bonus points) – Build a tester for your colleagues’ code Must include some random testing or SPIN testing by final checkin – Test! (and submit bug tickets) – Submit bug fixes, evaluations, mini-reports at milestones – Turn in final test report

3. Testing Project Your submitted code should contain two “subtle” bugs – Make sure your unit tests do not find the bug – Mail me: test cases and descriptions for the bugs Subject MUST be: CS362W11 BUGS [onidid] – You get to debug programs in lots of classes and in real life – in this class you get to bug a program intentionally, for once in your life I will see which testers find which bugs (over a sample of submitted programs)

4. Testing Project Grading criteria – Design/implementation project/tester – Effectiveness of the tester – Presentation of test plan (extra credit) – Quality of test/bug reports Can I figure out how you tested the system? Can I figure out what wasn’t tested? – Coverage, plus information on input restrictions Can I figure out how reliable you think the code is? What are key strengths/weaknessess? – How “interesting” and hard-to-find your intentional bugs are

5. Testing Project Expectations You can program in C You can figure out possibly poorly written specifications (a key software engineering skill) Or know when to ask someone who can! You can (learn to) use makefiles / build system “he who learns to play the harp learns to play by playing it” - Aristotle, Metaphysics, Book IX

6. Testing Project Oh, right. What are we implementing and testing? A card game Well defined notion of correctness “Specifications” not intended for implementation, but intended to be unambiguous Games are a good example of formal specifications: no one wants to have to “make up” the rule or interpret ambiguity in the middle of playing a game

7. Testing Project What are we implementing and testing? Dominion (+ expansions) Card game published by Rio Grande games Players start with their own draw decks Buy cards to add to decks (using cards) Some cards worth points if in deck Play this cards & you can draw a card and gain two actions (playing a card is an action)

8. Schedule April 7: First code checkin – Initial dominion implementation, expected to perhaps be buggy/incomplete; not frozen April 21: Test plan due April 28: BUGS DUE May 5: First test report (brief) due May 26: Second test report (brief) due June 2: “Late Midterm” June 7: FINAL TEST REPORT DUE, PROJECT COMPLETED, FINAL CODE FREEZE, TESTER CODE DUE

9. Grading & Other Admin Stuff Project: 75% Late midterm: 15% (Possibly-in-class) quizzes/exercises/homework: 10% My office hours: Tuesday, 10-11 KEC 3067 TAs: Sheela Surisetty surisets @onid.oregonstate.edu surisets @onid.oregonstate.edu Office Hours: W 3-4

10. Assignment 1 Get on beaversource (use your onid account) – http://beaversource.oregonstate.edu – Become my “friend” – Ask to join the class project (CS362S11) I’ll show you how, now

11. Assignment 1 Using svn, check out the code in the dominion directory – Compile it using make all – Look over the code, and the specifications (the rules in the rules subdirectory) Due as of first class next week