Transcript A Case Study in Database Reliability: Component Types, Usage Profiles, and Testing

A Case Study in Database
Reliability: Component Types,
Usage Profiles, and Testing
Colin Taylor, IBM Toronto Lab, Canada
Mechelle Gittens, IBM Toronto Lab, Canada
Andriy Miranskyy, University of Western
Ontario, Canada
Outline
•
•
•
•
•
•
Motivation
Database under test
Relevant terms
Case study
Conclusion
Future work
Motivation
• Improved DBMS reliability
• Efficient assignment of DBMS testing
effort
Database Under Test
• Distributed enterprise relational DBMS
• >20 MLOC
• Has evolved over >20 years and >10
major versions
Relevant Terms
•
•
•
•
•
LOC
LOCC
Component call frequency
Usage profiles: FT and ST
Component types: Development and
Architectural
• Defects
Case Study
• Case study presented as an analysis of
our perceptions
– Contour plots
– Tabular summary of contour plots
– Correlation tables
Case Study – Perceptions
1. Number of defects ~ LOC
2. Number of defects ~ Component call
frequency
3. For the Development component type,
FT Profile more strongly correlated to
number of defects than ST Profile
Case Study – Analysis
Conclusion
• Weak support for Perceptions 1 and 2
• Analysis of Perception 3 inconclusive
• Component size, the size of changes,
and call frequency are variables that
impact DBMS reliability
• Testing effort should be assigned
according to relational DBMS
architecture
Future Work
• Static code complexity variable
• Component overlap
Colin A. Taylor
[email protected]
Etcetera
Related Work
• Majority of studies deal with test tool implementation; only two
papers consider the relationships between variables that
influence DBMS reliability
– Chays, D., Dan, S., Frankl, P. G., Vokolos, F. I., and Weber, E. J.
2000. A framework for testing database applications.
– Deng, Y., Frankl, P., and Wang, J. 2004. Testing web database
applications.
– Suárez-Cabal, M. J. and Tuya, J. 2004. Using an SQL coverage
measurement for testing database applications.
– Wu, X., Wang, Y., and Zheng, Y. 2003. Privacy preserving database
application testing.
– Gittens, M., Lutfiyya, H., Bauer, M., Godwin, D., Kim, Y. W., and
Gupta, P. 2002. An empirical evaluation of system and
regression testing.
– Sullivan, M., Chillarege, R. A comparison of software defects in
database management systems and operating systems.