mcs-eval
OfficialEvaluate Copilot Studio agent performance.
Software Engineering#automation#testing#agent evaluation#copilot studio#direct line api#mcs native eval
Authormicrosoft
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill automates the evaluation of Copilot Studio agents, ensuring they meet quality and boundary standards before deployment. It identifies and reports on performance issues, enabling targeted fixes.
Core Features & Use Cases
- Automated Testing: Runs predefined evaluation sets against your agent using Direct Line API or MCS Native Eval.
- Mode Flexibility: Supports both fast, automated Direct Line testing and robust MCS Native Eval for agents with complex tools.
- Detailed Reporting: Writes test results directly into
brief.jsonfor dashboard visibility and generates CSVs for reference. - Use Case: After building a new customer support agent, use this Skill to run a suite of tests covering common queries, edge cases, and quality metrics. The results will highlight any areas where the agent fails to provide accurate or appropriate responses.
Quick Start
Run all evaluation sets for the project 'MyProject' and agent 'MyAgent'.
Dependency Matrix
Required Modules
@microsoft/teamsfx-clinode
Components
scriptsreferencesassets
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: mcs-eval Download link: https://github.com/microsoft/MCS-Agent-Builder/archive/main.zip#mcs-eval Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.