next up previous contents
Next: Testers vs. Developers Up: Generalized Observations Concerning Previous: Reachability

Changing Test Suites

Are faults more or less likely to be hiding after assertions are added and the manner by which test cases are generated changes? We begin to answer this question by looking at the situation where the code has no assertions, and then we will consider the situation where assertions are embedded.

For program P without assertions, we showed that regardless of what added to test suite D, we know that Given a new test suite that is not identical to D and neither test suite is a subset of the other, what can we say about the relation between and ? Quite simply, we cannot say anything confidently without running a technique similar to sensitivity analysis with both D and . That is, we do not know whether or .

Now let represent program P with assertions which were designed to boost the fault revealing ability of D. Assertions increase the likelihood that the third condition of the fault/failure model happens. Then,

>From there, this is also true:

 

regardless if D and have common members. Our argument in support of this conjecture follows: if the assertions placed into P (that are based on D) are never exercised via inputs from , then ; if as little as one assertion is exercised by some input in , then it is possible that will be true.

What this suggests for assertions is straightforward. Since assertions directly affect propagation, if propagation is homogeneous for some , it is likely to be homogeneous for some other .gif Whether j is in D or is immaterial. In general then, assertions appear to have an impact on fault detectability that is without respect for whether the inputs to the software are from one test suite or another. Hence since assertions improve the propagation prospects for inputs from D by increasing the cardinality or dimensionality of the output space, they should also improve the propagation prospects for .

And finally, we suspect that it will generally be true that:

However there will be cases where

specifically when and/or the assertions placed into P are not exercised.



next up previous contents
Next: Testers vs. Developers Up: Generalized Observations Concerning Previous: Reachability



Roger Gima
Tue Jan 20 16:43:17 EST 1998