Mutation testing: meaning and examples

Mutation testing judges a test suite by planting small deliberate faults, mutants, in the rule under test and checking whether the tests notice. A mutant that makes a test fail is killed; one that passes every test survives and exposes a gap.

A mutant is produced by one mechanical change: swapping an operator, removing a negation, replacing a literal. If the test suite is good, every mutant that changes behaviour is caught by at least one test. The mutation score, killed mutants over non-equivalent mutants, measures the suite rather than the code. Mutants equivalent to the original cannot be killed and must be recognised, which for Boolean rules the engine can do exactly, by equivalence check on every row. The engine mutates a rule with operator, negation and literal mutants and runs the rule's test suite against each.

The artefact on this page is one mutant compared with its original as a formula pair, not a mutation run: original = (member OR invited) AND NOT banned and negation_mutant = (member OR invited) AND banned, the negation removed. The two columns differ on rows 2 to 7, and the equivalence check is violated with the counterexample member = FALSE, invited = TRUE, banned = FALSE, on which the original is TRUE and the mutant FALSE. Any test with that input and the expected outcome TRUE kills this mutant; a suite with no such test lets it survive.

Mutation testing is the standard answer to “are my tests any good?”, used in compiler and library projects and increasingly for business rules, where the rule is small and the mutants are few. Its cost is running the suite once per mutant. The term is confused with fuzzing (random inputs rather than altered code), with property testing, and with regression testing, which compares versions written by people rather than generated faults.

Example

original := (member OR invited) AND NOT banned · outputnegation_mutant := (member OR invited) AND banned · output

Exact checks

Always true (tautology)? No Counterexample: member = F, invited = F, banned = F · Exhaustively checked and solver cross-checked · 1 rows

Always false (contradiction)? No Counterexample: member = F, invited = T, banned = F · Exhaustively checked and solver cross-checked · 3 rows

Can be true? Yes Example assignment: member = F, invited = T, banned = F · Exhaustively checked and solver cross-checked · 3 rows

Are original and negation_mutant equivalent? No Counterexample: member = F, invited = T, banned = F · Exhaustively checked and solver cross-checked · 3 rows

Truth table (8 rows)

Mutation testing: meaning and examples
#	member	invited	banned	original	negation_mutant
0	FALSE	FALSE	FALSE	FALSE	FALSE
1	FALSE	FALSE	TRUE	FALSE	FALSE
2	FALSE	TRUE	FALSE	TRUE	FALSE
3	FALSE	TRUE	TRUE	FALSE	TRUE
4	TRUE	FALSE	FALSE	TRUE	FALSE
5	TRUE	FALSE	TRUE	FALSE	TRUE
6	TRUE	TRUE	FALSE	TRUE	FALSE
7	TRUE	TRUE	TRUE	FALSE	TRUE

Plain-language reading

Beginneroriginal is true exactly when both of the following hold:
1. at least one of member or invited is true (both may be true)
2. banned is false
In every other case, original is false.
About the operators:
- AND is true only when all of its parts are true.
- OR is inclusive: it is true when at least one part is true, including when several are.
- NOT reverses a value: true becomes false and false becomes true.

Intermediateoriginal is true exactly when both of the following hold:
1. at least one of member or invited is true (both may be true)
2. banned is false
In every other case, original is false.

Advancedoriginal is true exactly when both of the following hold [conjunction ∧]:
1. at least one of member or invited is true (both may be true) [inclusive disjunction ∨]
2. banned is false [negation ¬]
In every other case, original is false.
Formal notation: (member ∨ invited) ∧ ¬banned

The common slip

Counting a surviving mutant as a bug in the rule. A survivor means the tests did not distinguish the mutant from the original; the rule may be correct and under-tested. The fix is a new test on a differing row, such as the counterexample row on this page, not a change to the rule.

Questions

What is an equivalent mutant?

A mutant whose change does not alter the function, such as swapping the operands of an AND. No test can kill it, so it is excluded from the score. For Boolean rules the engine detects it exactly: the equivalence check holds on every row and the mutant is reported as equivalent rather than surviving.

Glossary

Generated from the Tangibool engine, semantics tangibool-semantics/1, engine 1.0.0, build 2026-11-07. Every table, map, simplification and verdict on these pages was computed by the Tangibool engine when the site was built, and every claim was re-checked by the same exact checker the app uses. Limits are stated where they apply. © 2026 Foreranger LTD.