benchmark-agents

Runs interactive Claude Code eval sessions via WezTerm to verify Vercel plugin skill injection.

Updated Sep 5, 2026
One-click install
npx skills add https://github.com/dsgalkar/dnyaneshwar_portfolio --skill benchmark-agents-dsgalkar
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: benchmark-agents
Source: https://github.com/dsgalkar/dnyaneshwar_portfolio/tree/main/.agents/plugins/vercel/.claude/skills/benchmark-agents
Command: npx skills add https://github.com/dsgalkar/dnyaneshwar_portfolio --skill benchmark-agents-dsgalkar

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Testing whether a Claude Code plugin correctly injects skills and fires hooks requires real interactive sessions, since print-mode runs never trigger PreToolUse, PostToolUse, or UserPromptSubmit hooks. This Skill provides the exact commands, monitoring queries, and verification checks to run reliable end-to-end evals of the Vercel plugin. ## Core Features & Use Cases - Interactive Eval Launch: Spawns WezTerm panes running Claude Code sessions with the plugin installed via npx add-plugin, using timestamped directories and debug logging. - Injection Monitoring: Checks skill claim directories and debug logs to count injected skills, hook firings, and PostToolUse validation catches. - Code Verification: Greps generated projects for correct patterns such as withWorkflow config, AI Gateway usage, gemini image models, and AI Elements components. - Use Case: Run three parallel eval sessions with natural-language prompts (e.g., a durable content pipeline), then produce a coverage report showing which of the 43 plugin skills triggered and which code patterns need fixing before release. ## Quick Start Launch three benchmark eval sessions for the Vercel plugin using the tarot-deck, interior-designer, and superhero-origin scenarios and report which skills were injected.

Frequently Asked Questions about benchmark-agents

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test whether a Claude Code plugin injects skills correctly?▼

Launch an interactive Claude Code session in a WezTerm pane with the plugin installed via npx add-plugin, then inspect the session's claim directory in the temp folder. Each injected skill appears as a file in the vercel-plugin-<session-id>-seen-skills.d directory.

Why doesn't claude --print work for testing plugin hooks?▼

Print mode generates text without executing tools, so PreToolUse, PostToolUse, and UserPromptSubmit hooks never fire. No files are created, no session_id exists, and dedup, profiler, and claim files do not work.

How do I run multiple Claude Code eval sessions in parallel?▼

Create timestamped directories, install the plugin in each with npx add-plugin, then spawn a separate WezTerm pane per session using wezterm cli spawn with /bin/zsh -ic. Each pane runs an independent interactive session with its own debug log.

How do I check which hooks fired during a Claude Code session?▼

Grep the session's debug log in ~/.claude/debug for patterns like SessionStart success, executePreToolHooks, provided additionalContext, and VALIDATION. Setting VERCEL_PLUGIN_LOG_LEVEL=debug at launch enables this output.

What prompt style triggers Vercel plugin skills during evals?▼

Describe products in natural language rather than naming technologies, since the plugin matches prompt signals like multi-step pipeline or streams progress. Explicitly naming SDKs or config options bypasses the pattern matching being tested.