1. Presenter: Miguel Garzon Torres CrUise Lab - SITE SQL Coverage Measurement for Testing Database Applications María José Suárez-Cabal University of Oviedo Javier Tuya University of Oviedo

2. 2 Glossary DBMS:DBMS: Database Management System SQL:SQL: Structured Query Language UML:UML: Unified Modeling Language

3. 3 Context A software application that has a Database SQL Queries are embedded in the application code –Queries are part of the application’s business logic

4. 4 Goal Goal: We want to achieve the highest possible value of coverage of queries that access to the database. In other words, we want to test the database and obtain the highest SQL Coverage possible. HOW ? –Database test data is manipulated in order to get a better coverage

5. 5 Motivation Testing is a very expensive process in software development –30% of resources committed to this Most applications use a database for accessing the contents that they offer. We need to test the database –Because SQL queries are embedded into the code, we need to test in a similar way we test the rest of the code –Tests should cover all query situations and obtain maximum possible coverage

6. 6 Introduction The main aims of the research are: –Define a measurement of coverage of SQL Select queries in databases loaded with test data –Present an algorithm that automates the calculation of coverage –Guide the expert in the manipulation of test data to obtain better results in terms of coverage

7. 7 Related Work –Research Microsoft Research: Valid SQL statements were generated and run on several systems that contain identical databases –Goal: Evaluate DBMS and compare output AGENDA: Test data is generated, SQL queries are executed and the state of the database is verified. Other studies did not consider database schema and the database integrity constraints

8. 8 Related Work - Commercial Generators of database instances: TestByte, TestBase, DataTect generate random information depending on fields Users must know the database structure in order to use these tools SQLUnit is used for testing of stored procedures and SQL queries (XML)

9. 9 Software testing with DBs Testing is very difficult: –Selection of the initial instance of the test database is one of the most important steps –Combinations between tuples must be taken into account to cover all the SQL query situations –Test data should be useful for the greatest possible number of queries

10. 10 Simple Example sup_codesup_name 001sup1 002sup2 ord_numsup_code 1001 2 3002 1 2ord_numsup_name 1sup1 2 2sup2 SELECT sup_name, ord_num FROM Supplier s LEFT JOIN Order o ON (s.sup_code=o.sup_code)

11. 11 Detecting faults in SELECT queries Problems detected: –There may be suppliers with no orders, but this situation is not represented by the test data (all suppliers have at least one order) –Impossible to know whether the query always returns the information required –So, an adequate SELECT coverage is not obtained with these test data.

12. 12 Detecting faults in SELECT queries (cont’d) sup_codesup_name 001sup1 002sup2 003sup3 ord_numsup_code 1001 2 3002 ord_numsup_name 1sup1 2 2sup2 sup3 SELECT sup_name, ord_num FROM Supplier s LEFT JOIN Order o ON (s.sup_code=o.sup_code)

13. 13 Coverage of SELECT Queries 1. SELECT Query:

14. 14 Coverage of SELECT Queries 2. Coverage tree and evaluation of conditions: Evaluate the conditions of SELECT queries that are in the FROM clause, when they include JOIN, and in the WHERE clause Null values of fields will be verified as the conditions are evaluated

15. 15 Coverage of SELECT Queries – Coverage Tree Each level represents a condition of the query: –begins with conditions of JOIN clause (if it exists) –then those of WHERE clause –In same order as they appear in the query Each node of the tree will store: –Whether the condition is true for values of the fields;

16. 16 Coverage of SELECT Queries – Coverage Tree (cont’d) During the evaluation of a condition: –Each value in the first field must be compared with each one in the second field –Each value in the second field with each one in the first

17. 17 Coverage of SELECT Queries – Coverage Tree (cont’d) The evaluation of a condition: The grammar: A condition will be true if it is verified for a pair of values from the fields to compare. A condition will be false from left to right, if none of the values from the second field verifies the condition with a value from the first field. A condition will be false from right to left, if none of the values from the first field verifies the condition with a value from the second field.

18. 18 Coverage of SELECT Queries – Coverage Tree (cont’d) The evaluation of a condition: COVERAGE TREE: A condition will have null values when a value from the first field is null. A condition will have null values when a value from the second field is null. A condition will have null values when a value from the first and second fields are null.

19. 19 Coverage of SELECT Queries – Coverage Tree (cont’d) Coverage Tree

20. 20 Evaluation of the coverage tree The evaluation is made by crossing over the tuples of the tables participating in the conditions at each level of the tree. –Finishes when 100% coverage or no more values Two different coverage measures are established: –Theoretical coverage: Takes into account every possible situation at every node –Schema coverage: Takes into account the database schema constraints

21. 21 Theoretical Coverage It is not usually possible to reach 100% theoretical coverage, because of forbidden null values or referential integrity constraints. Solution: Schema coverage

22. 22 Schema Coverage Due to referential integrity, an order always had a supplier. Fr, Nb, and Nr will never be verified (X).

23. 23 Tool Demonstration Generation of Full Predicate Logic Coverage Rules 1 2 3 Enter Database Schema Enter SQL Query Complete the test data from Rules generated

24. 24 Tool Demonstration – Other Tool

25. Case Study The company’s department is responsible for managing the lamination rolls used in the rolling mills for the manufacture of steel sheets. Rolls are arranged in boxes. A mill may consist of one or more boxes. Company manages thousands of rolls, as well of dozens of boxes and mills.

26. Case Study - The problem All information about rolls, mills and boxes are maintained in a database. The database used for testing has about a thousand rolls and twenty boxes and mills.

27. Case Study - Queries Several SELECT queries that contain distinct tables of the system have been analyzed with the tool.

28. Case Study – Data model UML Model Database Tables

29. Case Study – Coverage Tree

30. Case Study – Measuring coverage After running the algorithm, we obtain: –Theoretical coverage = 19.23% Next steps: –Simplification of tuples based on constraints –Completing test data to complete our testing

31. 31 Future Work Including parameters in SQL QueryIncluding parameters in SQL Query Support for stored proceduresSupport for stored procedures More case studies demonstrating the conceptsMore case studies demonstrating the concepts

32. 32 Questions? Miguel Garzón - mgarz042@uottawa.ca