Mutation Testing

We can use mutation testing to figure out if the test case we write actually test proper functionality of the code. Mutations will result in the test failing.

The mutant is killed if different results are found. The test case is good as it detects the change (i.e., injected defect). The mutant is alive if the results are the same: the test case is NOT effective.

Important

So we want mutants to be killed.

Complete coverage equates to killing all non-equivalent mutants.

There are different kinds of mutants:

1. Stillborn Mutant: syntactically incorrect, killed by compiler
2. Trivial Mutant: killed by almost any test case
3. Equivalent Mutant: always produces the same output as the original program

None of the above are interesting from a mutation testing perspective. We want to have mutants that represent hard to detect faults.

Mutation score = 100 * D/(N-E)

• D = dead mutants
• N = total mutants
• E = equivalent mutants
  • these can be detected through an equivalence detector

Strong mutation: a fault must be reachable, infect the state, and propagate to output Weak mutation: a fault which kills a mutant need only be reachable and infect the state