evaluator

Evaluate rendered UGC videos and generate eval.json and eval-report.md.

118|12|Updated May 5, 2026
One-click install
npx skills add https://github.com/alecs5am/ralphy --skill evaluator-alecs5am
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluator
Source: https://github.com/alecs5am/ralphy/tree/main/.agents/skills/evaluator
Command: npx skills add https://github.com/alecs5am/ralphy --skill evaluator-alecs5am

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Quickly assess the quality and integrity of rendered UGC videos by producing a structured evaluation report that highlights scene segmentation, audio, captions, and visual consistency. The fixer agent can read the report to apply targeted fixes.

Core Features & Use Cases

  • Automated quality evaluation across render modes (structure, keyframe, native-video, deep-style) and generation of eval.json + eval-report.md for downstream agents.
  • Outputs deterministic findings and tailored handoffs for fixers, with references to scenario.json, BRIEF.md, STYLE_LOCK.md when present.
  • Supports triggering phrases like "evaluate this video" or "is this ready to ship" to kick off the evaluation workflow.

Quick Start

Run the evaluator on a rendered mp4 using ralphy eval video <path-to-mp4>, then review the generated eval.json and eval-report.md for the results.

Frequently Asked Questions about evaluator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate video quality evaluation for rendered mp4 files?▼

Automate video quality evaluation by running the evaluator on a rendered mp4 to generate an eval.json and eval-report.md highlighting scene segmentation, audio, captions, and visual consistency.

Can I run mid-render quality checks on UGC videos before final cuts?▼

Yes, you can run mid-render quality checks on UGC videos. The evaluator assesses quality across structure, keyframe, native-video, and deep-style render modes for both final cuts and intermediate renders.

Does the video evaluation report support handoffs to downstream fixer agents?▼

Yes, the video evaluation report supports downstream fixer agents by outputting deterministic findings and tailored handoffs in a machine-readable eval.json contract for targeted automated fixes.

What project context files can I use to improve UGC scene analysis accuracy?▼

You can improve UGC scene analysis accuracy by optionally incorporating project context from scenario.json, BRIEF.md, and STYLE_LOCK.md files when evaluating rendered video outputs.

What is the best way to generate a machine-readable eval.json contract for artifact detection?▼

The best way to generate a machine-readable eval.json contract for artifact detection is using an automated evaluation tool that analyzes rendered video outputs and surfaces actionable findings.