Code coverage analysis in practice for large systems

Abstract

Large systems generate immense quantities of code coverage data. A user faced with the task of analyzing this data, for example, to decide on test areas to improve, faces a 'needle in a haystack' problem. In earlier studies we introduced substring hole analysis, a technique for presenting large quantities of coverage data in a succinct way. Here we demonstrate the successful use of substring hole analysis on large scale data from industrial software systems. For this end we augment substring hole analysis by introducing a work flow and tool support for practical code coverage analysis. We conduct real data experiments indicating that augmented substring hole analysis enables code coverage analysis where it was previously impractical, correctly identifies functionality that is missing from existing tests, and can increase the probability of finding bugs. These facilitate cost-effective code coverage analysis. © 2011 ACM.