cs498dm Software Testing Darko Marinov January 15, 2008

Teaching Staff Instructor: Darko Marinov – –Office: SC 3116, –Office hours: after classes or by appointment TA: Vilas Shekhar Bangalore Jagannath – –Office hours: by appointment

Course Overview Introduction to software testing –Systematic, organized approaches to testing –Based on models and coverage criteria –Testing is not (only) about finding “bugs” –Improve your testing (and development) skills –Not focused on research (unlike cs598dm) Five problem sets and a project –Centered around testing refactoring engines

Refactoring Engines: One Slide Next lecture will be on refactoring engines –Guest lecturer: Jeff Overbey You’ll learn a lot about refactoring engines in this class Refactorings are program transformations that change code but not its behavior Refactoring engines are tools that automate refactorings

Administrative Info Lectures: TR 3:30pm-4:45pm, 1304 SC Credit: –3 undergraduate hours –3 or 4 graduate hours (a larger project for 4) Prerequisites: recommended software engineering (cs427) and programming languages (cs225, cs421) –Consent of instructor (must if not senior)

Grading Points –Project (25%) –Problem sets (5*15%) Grades –A*(90%), B*(80%), C*(70%), D*(60%), F(<60%) –For more details, see the syllabus –The instructor may lower the point limits

Project Testing a part of refactoring engine –Eclipse –NetBeans Deliverables –Proposal (due in three weeks) –Progress report (around midterm) –Final report (by the grade submission deadline) –Bug reports (hopefully you’ll find some bugs)

Collaboration You must individually write solutions for the problem sets You can collaborate on everything else (unless explicitly stated not to collaborate!) –Discuss problem sets –Do projects in groups, preferably two or three students Testing is a social activity –Communication matters

Course Communication Web site Wiki Mailing list Instructor: Darko – –Office: SC 3116, –Office hours: after classes or by appointment TA: Vilas

Signup Sheet Name address Program/Year Interests: what would you like to learn about testing? Experience: what testing did you do? To do: please also sign up on Wiki

Textbook “Introduction to Software Testing” by Paul Ammann and Jeff Offutt Cambridge University Press, Jan Books should be in the bookstore soon We won’t use book before the end of Jan. If necessary, you will get PDF files Book draft already used in cs{4,5}98dm

This Lecture: Introduction to “Bugs” Why look for bugs? What are bugs? Where they come from? How to detect them?

Some Costly “Bugs” NASA Mars space missions –Priority inversion (2004) –Different metric systems (1999) BMW airbag problems (1999) –Recall of 15,000+ cars Ariane 5 crash (1996) –Uncaught exception of numerical overflow – Your own favorite examples?

Some “Bugging” Bugs Two example bugs on my laptop –“Jumping” file after changing properties Put a read-only file on the desktop Change properties: rename and make not read-only –“Jumping” rectangle after move Put up a slide with two rectangles Move one within the textbox Your own favorite examples?

Economic Impact “The Economic Impact of Inadequate Infrastructure for Software Testing” NIST Report, May 2002 $59.5B annual cost of inadequate software testing infrastructure $22.2B annual potential cost reduction from feasible infrastructure improvements

Estimates Extrapolated from two studies (5% of total) –Manufacturing: transportation equipment –Services: financial institutions Number of simplifying assumptions “…should be considered approximations” What is important to you? –Correctness, performance, functionality

Some Motivation for Testers An article from SD Times, a magazine for software development managers: “Improving Software Quality” by Lindsey Vereen (page 34/68) A slide from Debra Richardson, a professor at UC Irvine: “Analysis and Testing are Creative” (page 26/48)

Terminology Anomaly Bug Crash Defect Error Failure, fault G… …

Dynamic vs. Static Incorrect (observed) behavior –Failure, fault Incorrect (unobserved) state –Error, latent error Incorrect lines of code –Fault, error

“Bugs” in IEEE Fault –Incorrect lines of code Error –Faults cause incorrect (unobserved) state Failure –Errors cause incorrect (observed) behavior Not used consistently in literature!

Next Lecture Thursday, January 17, 3:30pm, 1304 SC –Topic: refactorings What they are? Why you need them? How to perform them? –Guest lecturer: Jeff Overbey Assignments –Sign up on Wiki –Try out Eclipse and/or NetBeans