skills.rest
VMAFx
VMAFxOFFICIAL
Explore all skills by this author →
WebsiteGitHub

bisect-model-quality

Locate the first ONNX checkpoint that drops below a PLCC, SROCC, or RMSE quality gate.

3|1|Updated May 28, 2026
Search Tags:#checkpoints#onnx#binary search#rmse#model regression#plcc#srocc
One-click install
npx skills add https://github.com/VMAFx/vmafx --skill bisect-model-quality
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: bisect-model-quality
Source: https://github.com/VMAFx/vmafx/tree/main/.claude/skills/bisect-model-quality
Command: npx skills add https://github.com/VMAFx/vmafx --skill bisect-model-quality

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you pinpoint the first model checkpoint that fails a quality gate, so you can stop guessing where a regression began and focus on the exact training step that broke performance.

Core Features & Use Cases

  • Binary search over checkpoints: Evaluates an ordered list of ONNX models efficiently to localize the first failing checkpoint.
  • Quality-gate testing: Checks held-out data against PLCC, SROCC, or RMSE thresholds to determine whether each checkpoint passes or fails.
  • Operational guardrails: Validates that you have enough models, a parquet file with the required target column, and a clean working tree before running.
  • Use case: A training pipeline suddenly produces worse video-quality scores; use this Skill to identify the earliest checkpoint that crossed the regression boundary.

Quick Start

Run the bisect-model-quality skill on your ordered ONNX checkpoints and a held-out parquet with a MOS column to localize the first checkpoint that falls below your chosen PLCC, SROCC, or RMSE gate.

Frequently Asked Questions about bisect-model-quality

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find the first ONNX checkpoint that fails a quality gate?▼

To find the first failing ONNX checkpoint, you can binary search across an ordered list of models, testing held-out parquet data against PLCC, SROCC, or RMSE thresholds to localize the exact step where performance dropped.

What do I need to bisect model regressions across PLCC or RMSE thresholds?▼

Bisecting model regressions requires at least two ordered ONNX checkpoints, a held-out parquet file containing a mos target column, and exactly one defined gate flag to evaluate whether each checkpoint passes or fails.

Can I use binary search to evaluate ONNX model checkpoints on held-out data?▼

Yes, binary search efficiently evaluates ordered ONNX model checkpoints against held-out parquet data, using deterministic ORT inference to identify the first checkpoint that drops below your specified quality gate.

What quality metrics are supported for testing ONNX checkpoint regressions?▼

Supported quality metrics for testing ONNX checkpoint regressions include PLCC, SROCC, and RMSE thresholds, which are calculated against a mos target column in your held-out parquet evaluation dataset.

Why does checkpoint bisection require a clean working tree and multiple models?▼

Checkpoint bisection requires a clean working tree and at least two models as operational guardrails to ensure deterministic ORT inference and validate that sufficient data exists to localize the first failing checkpoint accurately.

Does the checkpoint bisection process support JSON output for regression analysis?▼

Yes, the checkpoint bisection process supports optional JSON output, providing structured results of the binary search over ONNX models to help you analyze exactly where the model regression began.

Related Skills

Explore More Software Engineering Skills→
rocklambrosrocklambros
2

auditing-rlhf-reward-hacking

Audit RLHF, DPO, and RLAIF checkpoints for reward hacking by comparing reward win-rates with preference win-rates.

Community
Advanced
gabrielfruetgabrielfruet
3

pytorch-nan-debugging

Identifies root causes of non-finite values in PyTorch training runs.

Community
Advanced
HopeArtOrgHopeArtOrg
8

onnx-rust-integration

Load and execute ONNX models in Rust with the ort crate.

Official
Intermediate
VYRE-StudiosVYRE-Studios

segment-anything-model

Generate binary object masks from images via zero-shot segmentation.

Official
Advanced