planning-visual-tasks

Decomposes natural-language visual instructions into typed Plan envelopes for ComfyUI orchestration.

Updated Apr 8, 2026
One-click install
npx skills add https://github.com/ShinyGua/ComfyUI-Agent --skill planning-visual-tasks-shinygua
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: planning-visual-tasks
Source: https://github.com/ShinyGua/ComfyUI-Agent/tree/main/.claude/skills/planning-visual-tasks
Command: npx skills add https://github.com/ShinyGua/ComfyUI-Agent --skill planning-visual-tasks-shinygua

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Turning a free-form image or video instruction into an executable multi-step ComfyUI workflow requires deciding which capabilities to invoke, in what order, and how outputs chain between steps. This Skill automates that planning: it converts a natural-language instruction into a structured Plan envelope (matching contracts/agent/plan.schema.json) of typed sub-goals that the host Orchestrator executes. ## Core Features & Use Cases - LLM-driven planning (v1.2): propose_plan.py wraps the host Planner to emit Plans with sub-goal kinds like workflow_search, model_resolve, comfyui_execute, llm_transform, evaluate, checkpoint, and wait_user, with cross-step state://, template://, and model:// references. - Deterministic fallback router: when no LLM provider is configured, legacy keyword-based decompose.py and emit_dag.py split instructions on connectives, map clauses to capability skills, and emit a validated DAG with provenance and trust tiers. - Tool discovery and plan validation: discover_tools.py lists executor kinds, models, templates, and MCP tools; validate_plan.py checks schema conformance, DAG acyclicity, and reference resolution. - Use Case: A user says "generate a rainy alley, then inpaint a detective, then apply noir style while preserving the face" — the Skill emits a four-step DAG with dependencies, identity-preservation evaluators, and per-step provenance. ## Quick Start Plan the instruction "generate a cyberpunk portrait, then replace the background with a neon street" into an executable ComfyUI workflow DAG.

Frequently Asked Questions about planning-visual-tasks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I turn a text instruction into a ComfyUI workflow plan?▼

Pipe a JSON payload with the instruction into propose_plan.py, which calls the host LLM planner and returns a Plan envelope of typed sub-goals. If no LLM provider is configured, the legacy decompose.py and emit_dag.py scripts produce a keyword-routed DAG instead.

How does the planner handle multi-step instructions like generate then inpaint?▼

Instructions are split on connectives such as "then" and "after that", and each clause is mapped to a capability skill. Each step's output is wired as the next step's image input, and the DAG is validated for acyclicity before execution.

What happens when no LLM provider is configured?▼

The host falls back to the legacy v1 keyword router, a pure-stdlib Python script with no LLM dependency. It uses regex keyword matching to map clauses to skills and fails loudly with a no_match error if any clause cannot be resolved.

Can the planner handle unusual requests with no matching keyword?▼

Yes, if the caller supplies search_candidates from the searching-comfyui-solutions skill. Each candidate must declare a valid skill and goal; candidates without them are rejected and the plan fails rather than guessing.

How is face or identity preservation handled across steps?▼

Explicit constraint markers like "while preserving the face" arm a sticky flag that adds preserve:[face] and the identity-preservation evaluator to subsequent edit steps. Incidental mentions of "face" in a clause do not trigger this propagation.

Why does plan validation reject forward references between sub-goals?▼

Cross-step references using state://, template://, model://, or artifact:// schemes must point to a sub-goal that appears earlier in the list. validate_plan.py enforces this so the executor can resolve inputs in topological order.