evaluating-instruction-following

Scores whether a generated image satisfies the original natural-language instruction.

Updated Apr 8, 2026
One-click install
npx skills add https://github.com/ShinyGua/ComfyUI-Agent --skill evaluating-instruction-following-shinygua
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluating-instruction-following
Source: https://github.com/ShinyGua/ComfyUI-Agent/tree/main/.claude/skills/evaluating-instruction-following
Command: npx skills add https://github.com/ShinyGua/ComfyUI-Agent --skill evaluating-instruction-following-shinygua

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? After an image generation or editing step completes, there is no automatic way to know whether the result actually matches what the user asked for. This Skill provides a deterministic pass/fail gate that scores instruction adherence before a step is marked done, feeding failures back into the repair loop. ## Core Features & Use Cases - Heuristic Scoring: Combines goal-token matching, instruction-to-prompt token overlap, and denoise-range sanity checks into a 0-1 score with a 0.5 pass threshold. - Structured JSON Contract: Reads {instruction, result_meta} on stdin and emits {pass, score, reasons[], evaluator} on stdout, so it can be swapped for a vision-LLM evaluator later without interface changes. - Smoke Evals Included: Ships an eval runner covering pass, fail, and malformed-input cases. - Use Case: After a ComfyUI generation step produces an image for the instruction "generate a photo of a red fox in snow", run this evaluator to confirm the goal, prompt, and denoise settings align with the request before accepting the result. ## Quick Start Evaluate the last generation result against my original instruction and tell me whether it passes.

Frequently Asked Questions about evaluating-instruction-following

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if a generated image matches the user's instruction?▼

Pass the original instruction and the result metadata (goal, prompt, denoise) as JSON to the evaluator script on stdin. It returns a pass flag, a 0-1 score, and human-readable reasons explaining the decision.

How does the instruction-following score get calculated?▼

The score adds 0.3 when the goal token matches the instruction, up to 0.5 for token overlap between instruction and prompt, and 0.2 when the denoise value is sane for the goal. A score of 0.5 or higher passes.

Can I replace the heuristic evaluator with a vision-LLM?▼

Yes, the stub is designed for that swap. Keep the same stdin/stdout JSON contract and route a vision-LLM call through the host LLM provider, and downstream gating logic continues to work unchanged.

What happens when the evaluator receives malformed or empty input?▼

Invalid JSON or an empty instruction causes the script to exit with a non-zero code and output pass=false with a score of 0.0 and a reason describing the problem, so callers can treat it as a failed evaluation.

What are the limitations of a heuristic image evaluation approach?▼

Heuristics only inspect metadata like goal, prompt text, and denoise value, not the actual pixels, so they cannot detect visual mismatches. Semantic or visual correctness requires the planned vision-LLM evaluator in v2.