create-mcp-eval
OfficialGenerate eval tests for MCP servers.
AuthorMCPJam
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill automates the creation of comprehensive evaluation tests for MCP servers, ensuring tool accuracy and LLM reasoning capabilities are rigorously assessed.
Core Features & Use Cases
- Automated Test Generation: Creates Jest or Vitest compatible eval tests using the
@mcpjam/sdk. - LLM-Driven & Deterministic Patterns: Supports both mock-based (deterministic) and LLM-driven test scenarios.
- Comprehensive Validation: Includes tools for matching tool calls, arguments, and counts, along with multi-turn conversation testing.
- Use Case: A developer needs to ensure their new MCP server's tools are correctly invoked by an LLM. This Skill generates a test suite that prompts the LLM with various requests and verifies the appropriate tools are called with the right parameters, reporting results directly to MCPJam for analysis.
Quick Start
Generate eval tests for your MCP server using the create-mcp-eval skill.
Dependency Matrix
Required Modules
@mcpjam/sdkvitesttypescriptjestts-jest
Components
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: create-mcp-eval Download link: https://github.com/MCPJam/inspector/archive/main.zip#create-mcp-eval Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.