train-visual-qa

What problem does it solve?

Ensures visual-qa analysis prompts reliably detect layout and spacing defects in mobile UI screenshots by automating evaluation against labeled test cases and preventing regressions caused by ad-hoc prompt edits.

Core Features & Use Cases

• Automated Test Harness: Loads captured test cases with expected signals and known false positives, runs the visual-qa prompt against each screenshot, and collects outputs for scoring.
• Deterministic Scoring: Compares outputs to ground truth to mark FOUND/MISSED signals and AVOIDED/FLAGGED false positives, applying an 80%+ pass threshold and zero flagged known false positives as pass criteria.
• Iterative Prompt Tuning: Guides structural prompt changes, re-runs the full suite to detect regressions, and repeats until results stabilize or iteration limits are reached.
• Use Case: Fine-tune a visual-qa analysis prompt to consistently find spacing and layout defects across a library of mobile screenshots before deploying to production QA pipelines.

Quick Start

Run the visual-qa trainer to evaluate the visual-qa prompt against the training test cases and iterate until pass criteria are met.