validate-rubrics

What problem does it solve?

This Skill addresses the critical issue of unreliable or vulnerable evaluation rubrics that can lead to incorrect, incomplete, or biased results. It proactively identifies and rectifies these weaknesses.

Core Features & Use Cases

• Adversarial Stress-Testing: Simulates edge cases and attack vectors to uncover rubric vulnerabilities.
• Vulnerability Analysis: Identifies specific ways a rubric might fail, rating severity and likelihood.
• Revision and Validation: Proposes concrete improvements and validates the hardened rubric against identified edge cases.
• Use Case: A team developing an AI model for content moderation uses this Skill to ensure their rubric for evaluating harmful content is robust and doesn't miss nuanced cases or produce biased classifications.

Quick Start

Use the validate-rubrics skill to stress-test the rubric file located at rubrics/prompt.md.