Probabilistic regression suites for functional verification

Abstract

Random test generators are often used to create regression suites on-the-fly. Regression suites are commonly generated by choosing several specifications and generating a number of tests from each one, without reasoning which specification should be used and how many tests should be generated from each specification. This paper describes a technique for building high quality random regression suites. The proposed technique uses information about the probability of each test specification covering each coverage task. This probability is used, in turn, to determine which test specifications should be included in the regression suite and how many tests should be generated from each specification. Experimental results show that this practical technique can be used to improve the quality, and reduce the cost, of regression suites. Moreover, it enables better informed decisions regarding the size and distribution of the regression suites, and the risk involved.