Consistency Testing

Consistency Testing is a model validation approach that verifies whether a model produces logically consistent predictions across related inputs — checking that the model's outputs satisfy domain constraints, monotonicity requirements, and logical coherence.

Types of Consistency Tests

• Monotonicity: If feature $x$ increases and all else is equal, the prediction should increase (or decrease) monotonically if the relationship is known to be monotonic.
• Transitivity: If A > B and B > C, the model should predict A > C.
• Symmetry: If the relationship between A and B should be symmetric, $f(A,B) = f(B,A)$.
• Boundary: At known boundary conditions, predictions should match known physical limits.

Why It Matters

• Physical Plausibility: Inconsistent predictions indicate the model has not learned the underlying physics.
• Edge Cases: Consistency tests often catch failures at extremes of the input space.
• Trust: Engineers won't trust a model that violates known engineering relationships, even if average accuracy is high.

Consistency Testing is checking the model's logic — verifying that predictions satisfy known constraints, monotonic relationships, and domain rules.