Regression test: meaning and examples

A regression test checks that behaviour which used to be right is still right after a change. For a Boolean rule, the sharpest regression test is an equivalence check between the old and new versions, which names every row where they differ.

Regression tests are the tests you keep: cases recorded when a rule was known to be correct, re-run after every edit. Their job is to detect unintended change, so a failing regression test after an intended change is information, not necessarily an error; it means the change reached that case. For rules small enough to tabulate, the whole table is the regression suite, and comparing two versions row by row shows exactly what changed, which no sample of cases can promise.

The artefact on this page compares v1 = (member OR invited) AND NOT banned with v2 = (member OR invited OR staff) AND NOT banned, which adds a fourth input. Over the 16-row table, the two columns differ on exactly one row, row 1, and the equivalence check is violated with that counterexample: member = FALSE, invited = FALSE, banned = FALSE, staff = TRUE, where v1 refuses and v2 admits. Every other row is unchanged, so the edit did what it meant to and nothing else; a staff member who is banned is still refused in v2.

Regression testing is the daily practice of every software team, usually as a suite run in continuous integration. For rules, the row-level comparison on this page is the stronger form: it finds the one changed row in sixteen without anyone having to think of it. The term is confused with re-testing (checking that a fix fixed the bug) and with the statistical sense of regression, which is unrelated.

Example

v1 := (member OR invited) AND NOT banned · outputv2 := (member OR invited OR staff) AND NOT banned · output

Exact checks

Always true (tautology)? No Counterexample: member = F, invited = F, banned = F, staff = F · Exhaustively checked and solver cross-checked · 1 rows

Always false (contradiction)? No Counterexample: member = F, invited = T, banned = F, staff = F · Exhaustively checked and solver cross-checked · 3 rows

Can be true? Yes Example assignment: member = F, invited = T, banned = F, staff = F · Exhaustively checked and solver cross-checked · 3 rows

Are v1 and v2 equivalent? No Counterexample: member = F, invited = F, banned = F, staff = T · Exhaustively checked and solver cross-checked · 2 rows

Truth table (16 rows)

Regression test: meaning and examples
#	member	invited	banned	staff	v1	v2
0	FALSE	FALSE	FALSE	FALSE	FALSE	FALSE
1	FALSE	FALSE	FALSE	TRUE	FALSE	TRUE
2	FALSE	FALSE	TRUE	FALSE	FALSE	FALSE
3	FALSE	FALSE	TRUE	TRUE	FALSE	FALSE
4	FALSE	TRUE	FALSE	FALSE	TRUE	TRUE
5	FALSE	TRUE	FALSE	TRUE	TRUE	TRUE
6	FALSE	TRUE	TRUE	FALSE	FALSE	FALSE
7	FALSE	TRUE	TRUE	TRUE	FALSE	FALSE
8	TRUE	FALSE	FALSE	FALSE	TRUE	TRUE
9	TRUE	FALSE	FALSE	TRUE	TRUE	TRUE
10	TRUE	FALSE	TRUE	FALSE	FALSE	FALSE
11	TRUE	FALSE	TRUE	TRUE	FALSE	FALSE
12	TRUE	TRUE	FALSE	FALSE	TRUE	TRUE
13	TRUE	TRUE	FALSE	TRUE	TRUE	TRUE
14	TRUE	TRUE	TRUE	FALSE	FALSE	FALSE
15	TRUE	TRUE	TRUE	TRUE	FALSE	FALSE

Plain-language reading

Beginnerv1 is true exactly when both of the following hold:
1. at least one of member or invited is true (both may be true)
2. banned is false
In every other case, v1 is false.
About the operators:
- AND is true only when all of its parts are true.
- OR is inclusive: it is true when at least one part is true, including when several are.
- NOT reverses a value: true becomes false and false becomes true.

Intermediatev1 is true exactly when both of the following hold:
1. at least one of member or invited is true (both may be true)
2. banned is false
In every other case, v1 is false.

Advancedv1 is true exactly when both of the following hold [conjunction ∧]:
1. at least one of member or invited is true (both may be true) [inclusive disjunction ∨]
2. banned is false [negation ¬]
In every other case, v1 is false.
Formal notation: (member ∨ invited) ∧ ¬banned

The common slip

Keeping the old expected results after an intended change. When v2 deliberately admits staff, the recorded expectation for row 1 must change with it; leaving it makes the suite fail forever or, worse, tempts someone to delete the test. Record the differing rows the equivalence check reports and update exactly those.

Questions

Does adding an input break the comparison with the old version?

No. The table is built over the union of the inputs, and v1 simply ignores staff on every row. The one differing row is where staff alone would admit someone, which is the whole intended change.

Glossary

Generated from the Tangibool engine, semantics tangibool-semantics/1, engine 1.0.0, build 2026-11-07. Every table, map, simplification and verdict on these pages was computed by the Tangibool engine when the site was built, and every claim was re-checked by the same exact checker the app uses. Limits are stated where they apply. © 2026 Foreranger LTD.