numerical-validation

What problem does it solve?

Subtle numerical errors in scientific algorithms can lead to incorrect research conclusions, wasted effort, and unreliable software. This Skill provides a rigorous process to verify the mathematical correctness and numerical accuracy of your scientific code after any changes, ensuring the integrity of your results.

Core Features & Use Cases

• Baseline Comparison: Capture and compare outputs before and after code changes to detect any numerical differences.
• Mathematical Invariant Verification: Explicitly check critical properties like probabilities summing to one or matrices being stochastic.
• Property-Based Testing: Use advanced testing techniques to verify invariants across a wide range of random inputs and edge cases.
• Golden Regression Testing: Compare against validated reference outputs using real scientific data to catch subtle changes affecting scientific conclusions.
• Use Case: After optimizing a JAX-based Hidden Markov Model (HMM) filter for performance, use this Skill to confirm that the faster code produces numerically identical results to the original, ensuring no accuracy was sacrificed for speed.

Quick Start

Follow the numerical validation checklist:

1. Capture baseline outputs before change

2. Make the code change

3. Capture new outputs after change

4. Compare numerical differences

5. Verify mathematical invariants

6. Run property-based tests

7. Run golden regression tests

8. Generate full analysis report

9. Present analysis and request approval (if differences found)