evaluator

What problem does it solve?

This skill addresses the challenge of improving AI tools without compromising user privacy. It provides a transparent, opt-in framework for collecting anonymous usage data and feedback, enabling developers to enhance skills based on real-world interactions and user needs. It eliminates guesswork in feature prioritization and bug fixing, ensuring that AI works better for everyone.

Core Features & Use Cases

• Privacy-First Telemetry: Collects anonymous usage events (skill invoked, success/failure, duration) with no PII, ensuring user trust and compliance.
• GitHub-Native Feedback: Leverages GitHub Issues and Projects for structured feedback, bug reports, and improvement suggestions, streamlining the feedback loop.
• Skill-Specific Metrics: Tracks detailed metrics for integrated skills like Oracle (query success rate), Guardian (suggestion acceptance), and Summoner (subagent spawn frequency), providing actionable insights.
• Use Case: A skill developer wants to understand which features of their new Claude Skill are most used and where users encounter errors. They integrate Evaluator to collect anonymous data, which then informs their development roadmap and helps prioritize bug fixes, leading to a more robust and user-centric skill that truly serves its users.

Quick Start

To enable anonymous telemetry for your project: /evaluator enable To submit feedback on a skill: /evaluator feedback