bayesflow-validation

What problem does it solve?

This Skill addresses the critical need to validate the performance and calibration of BayesFlow models, ensuring they accurately represent the underlying data-generating process.

Core Features & Use Cases

• Simulation-Based Calibration (SBC): Verifies that posterior credible intervals achieve nominal coverage, a key indicator of model correctness.
• Condition Grid Validation: Tests model performance across the entire design space, not just training conditions, to prevent overfitting.
• Coverage Metrics & Quality Gates: Provides quantitative metrics like calibration error and C2ST deviation, enabling automated pass/fail criteria for training.
• Use Case: After training a new posterior estimator, use this Skill to run an SBC analysis, check coverage metrics against predefined thresholds, and ensure the model is reliable before deployment.

Quick Start

Use the bayesflow-validation skill to run the validation pipeline on the current workflow with the pre-computed validation data.