First delivery of the course Software Quality and Testing Katerina Zdravkova, Anastas Mišev

Syllabus of M.Sc. course  Introduction to testing fundamentals  Structural testing  Functional testing  Foundation for combining formal methods and testing  Assertion based testing  Model based formal methods  Finite state machine based testing  Testing from a process algebra  Testing with UML's dynamic models  Temporal logic, model checking and their role in testing  Tools for automatic testing and continuous integration  The management process of software testing

Study pack  Exists  Slides are nice  Incomplete  No lecture notes  No supportive literature, or electronic version of a book  And, we had no experience in the area

CSII Course Sophomore Course Software Engineering Course Senior Course MS Course PhD Course Industry

Syllabus of the course (stream 4)  Introduction  Graph Testing  Logic Testing  Input Space Partitioning  Syntax Testing  Practical Considerations

Syllabus ~ book topics Four Structures for Modeling Software Graphs Logic Input Space Syntax Use cases Specs Design Source Applied to DNF Specs FSMs Source Input Models Integ Source

Grading ActivityQtyPointsTotal% to pass Homework Project150 Mid term Extra points201 ActivityQtyPointsTotal% to pass Homework Project150 Practical exam Theoretical exam Extra points201 PointsPercentageGrade < 175< 50%5 (fail) 175 – 21050% - 60%6 211 – 24961% - 71%7 250 – 28772% - 82%8 288 – 32583% - 94%9 >326> 95%10

Sample of a student assignment  Provide reachability conditions, infection conditions, propagation conditions, and test case values to kill mutants 1- 6 in the figure below MutantReachabilityInfectionPropagation

Students project Given the method Proekt1 (in C#) that returns: (a + b) / (a * b) if c = true and d = true square root (a*b)if c = false and d = true square root(a/b)if c = true and d = false -1 otherwise public double Proekt1(int a, int b, Boolean c, Boolean d) { if (c && d) { return (a + b) / (a * b); } else if (!c && d) return Math.Sqrt(a * b); else if (c && !d) return Math.Sqrt(a / b) ; else return -1; } 1. Code and compile in a language of choice 2. Partition the input space with respect to the values of the variables a and b, whether the method raises an error. The blocks should be both complete and disjoint. 3. Choose test values from each block and derive the tests that satisfy All Combinations Coverage (ACoC) 4. Identify the faults and produce a method Project1_no_Faults 5. Derive a new method from the Project_no_Faults, called Project_1_if, such that each if statement tests only one variable 6. Write a program that will execute all the test from 3, over the Project_if and Project_no_Faults methods.

Statistics AssignmentSubmissionsAvg Introduction to testing Introduction to testing Graph coverage Finite state machines Logic coverage Logic coverage Input space partitioning Input space partitioning Mutation testing Grammar mutation Project52 (so far)/ (not graded)

Exams and mid term exams  The preferred way was through mid term exams  45 students took the mid term exams(both)  43 of them passed  The rest of the students took the exams  Only 3 left for the September exams  3 students dropped out

Conclusions about the course  The course is in the 8 th semester, and it had to start for the first time at out Institute  Keti, Anastas and Saško and had no choice but to start the course  Slides by Ammann and Offutt are fantastic  We find the book fantastic too  It demystifies the topics well  The approach contains all the theoretical knowledge

Conclusions  Although it was hard to prepare the lectures, the A&O’s approach was exciting, and the effort was worthy  We learned a lot about quality and software testing  We plan to introduce tool and practical testing in the next delivery  Any suggestions how to improve the course are welcome  We are ready to deliver the same course in the framework of the Joint M.Sc. studies wherever needed

Thank you for your attention