eval-model-comparison

What problem does it solve?

This Skill enables rigorous benchmarking of OCR, transcription, and LLM form-filling accuracy against gold references to guide model selection and benchmarking reports.

Core Features & Use Cases

• Benchmarking: Compare across OCR, transcription, and LLM form-filling pipelines using field-level metrics and hallucination checks.
• Model Selection: Provide per-field confusion summaries and a model-comparison report template to inform choices among models and settings.
• Use Case: Build a benchmark with typed notes as references to evaluate system reliability for medical notes or structured forms.

Quick Start

Run the eval-model-comparison workflow on a sample dataset to generate the initial model comparison report.