1. A Case Study in Database Reliability: Component Types, Usage Profiles, and Testing Colin Taylor, IBM Toronto Lab, Canada Mechelle Gittens, IBM Toronto Lab, Canada Andriy Miranskyy, University of Western Ontario, Canada

2. Outline Motivation Database under test Relevant terms Case study Conclusion Future work

3. Motivation Improved DBMS reliability Efficient assignment of DBMS testing effort

4. Database Under Test Distributed enterprise relational DBMS >20 MLOC Has evolved over >20 years and >10 major versions

5. Relevant Terms LOC LOCC Component call frequency Usage profiles: FT and ST Component types: Development and Architectural Defects

6. Case Study Case study presented as an analysis of our perceptions –Contour plots –Tabular summary of contour plots –Correlation tables

7. Case Study – Perceptions 1.Number of defects ~ LOC 2.Number of defects ~ Component call frequency 3.For the Development component type, FT Profile more strongly correlated to number of defects than ST Profile

8. Case Study – Analysis

9. Conclusion Weak support for Perceptions 1 and 2 Analysis of Perception 3 inconclusive Component size, the size of changes, and call frequency are variables that impact DBMS reliability Testing effort should be assigned according to relational DBMS architecture

10. Future Work Static code complexity variable Component overlap

11. Colin A. Taylor colint@ca.ibm.com

12. Etcetera

13. Related Work Majority of studies deal with test tool implementation; only two papers consider the relationships between variables that influence DBMS reliability –Chays, D., Dan, S., Frankl, P. G., Vokolos, F. I., and Weber, E. J. 2000. A framework for testing database applications. –Deng, Y., Frankl, P., and Wang, J. 2004. Testing web database applications. –Suárez-Cabal, M. J. and Tuya, J. 2004. Using an SQL coverage measurement for testing database applications. –Wu, X., Wang, Y., and Zheng, Y. 2003. Privacy preserving database application testing. –Gittens, M., Lutfiyya, H., Bauer, M., Godwin, D., Kim, Y. W., and Gupta, P. 2002. An empirical evaluation of system and regression testing. –Sullivan, M., Chillarege, R. A comparison of software defects in database management systems and operating systems.